A Document-grounded Matching Network for Response Selection in Retrieval-based Chatbots

1 Introduction

Human-machine conversation is a long-standing goal of artificial intelligence. Recently, building a chatbot for open domain conversation has gained increasing interest due to both availabilities of a large amount of human conversation data and powerful models learned with neural networks. Existing methods are either retrieval-based or generation-based. Retrieval-based methods respond to human input by selecting a response from a pre-built index [Ji et al., 2014; Yan and Zhao, 2018], while generation-based methods synthesize a response with a natural language model [Shang et al., 2015; Li et al., 2015]. In this work, we study the problem of response selection for retrieval-based chatbots, since retrievalbased systems are often superior to their generation-based counterparts on response fluency and diversity, are easy to evaluate.

A key step in response selection is measuring the matching degree between a context (a message with a few turns of conversation history) and a response candidate. Existing methods [Wu et al., 2017; Zhou et al., 2018b] have achieved impressive performance on benchmarks [Lowe et al., 2015;

∗Equal Contribution.

†Corresponding author: Rui Yan (ruiyan@pku.edu.cn).

Wu et al., 2017], but responses are selected solely based on conversation history. Human conversations, on the other hand, are often grounded in external knowledge. For example, in Reddit, discussion among users is usually along the document posted at the beginning of a thread which provides topics and basic facts for the following conversation. Lack of knowledge grounding has become one of the major gaps between the current open domain dialog systems and real human conversations. As a step toward bridging the gap, we investigate knowledge-grounded response selection in this work and specify the knowledge as unstructured documents that are common sources in practice. The task is that given a document and a conversation context based on the document, one selects a response from a candidate pool that is consistent and relevant with both the conversation context and the background document. Table 1 shows an example from PERSONA-CHAT, a data set released recently by Facebook [Zhang et al., 2018], to illustrate the task: given two speakers' profiles as documents and a conversation context, one is required to distinguish the true response from the false ones 1 .

Intuitively, both documents and conversation contexts should participate in matching. Since documents and contexts are highly asymmetric in terms of information they convey, and there exists complicated dependency among sentences in the documents and utterances in the contexts, challenges of the task include (1) how to ground conversation contexts with documents given that utterances in the contexts are not always related to the documents due to the casual nature of open domain conversation (e.g., the greetings in Table 1); (2) how to comprehend documents with conversation contexts when information in the documents are rather redundant for proper response recognition (e.g., the description regarding to B's hobby in her profile in Table 1); and (3) how to effectively leverage both information sources to perform matching. To overcome the challenges, we propose a document-grounded matching network (DGMN). DGMN encodes sentences in a document, utterances in a conversation context, and a response candidate through self-attention, and models context grounding and document comprehension by constructing a document-aware context representation and a context-aware document representation via an attention mechanism. With the rich representations, DGMN distills matching information from each utterance-response pair and each sentenceresponse pair, where whether an utterance needs grounding, which parts of the document are crucial for grounding and matching, and which parts of the context are useful for representing the document are dynamically determined by a hierarchical interaction mechanism. The final matching score is defined as an aggregation of matching signals from all pairs.

1 For space limitation, we only show one false response here.

We conduct experiments on two public data sets: the PERSONA-CHAT data [Zhang et al., 2018] and the CMU Document Grounded Conversation (CMUDoG) data [Zhou et al., 2018a]. Evaluation results indicate that on both data sets, DGMN can significantly outperform state-of-the-art methods. Compared with Transformer, the best performing baseline on both data, absolute improvements from DGMN on r@1 (hits@1) are more than 13% on the PERSONA-CHAT data and more than 5% on the CMUDoG data. Through both quantitative and qualitative analysis, we also demonstrate the effect of different representations to matching and how DGMN grounds conversation contexts with documents.

Our contributions in this work are three-fold: (1) proposal of a document-grounded matching network that performs response selection according to both conversation contexts and background knowledge; (2) empirical verification of the effectiveness of the proposed model on two public data sets; and (3) new state-of-the-art on the PERSONA-CHAT data without any pre-training on external resources.

2 Document-GroundedMatching Network

In this section, we first formalize the document-grounded matching problem, and then introduce our model from an overview to details of components.

2.1 ProblemFormalization

Suppose that we have a data set D = {(Di, ci, yi, ri)}Ni=1 where Di = {di,1, · · · , di,mi } is a document that serves as background knowledge for conversation with di,k the k-th sentence, ci = {ui,1, · · · , ui,ni } is a conversation context following Di with ui,k the k-th utterance, ri is a response candidate, and yi ∈ {0, 1} is a label with yi = 1 indicating that ri is a proper response given ci and Di, otherwise yi = 0. The task is to learn a matching model g(·, ·, ·) from D, and thus for a new triple (D, c, r), g(D, c, r) returns the matching degree between c and r under D.

2.2 Model Overview

We define g(D, c, r) as a document-grounded matching network. Figure 1 illustrates the architecture of the model. In brief, DGMN consists of an encoding layer, a fusion layer, a matching layer, and an aggregation layer. The encoding layer represents D, c, and r via self-attention, and feeds the representations to the fusion layer where D and c are fused into the representations of each other as a document-aware context representation and a context-aware document representation. Based on the representations given by the first two layers, the matching layer then lets each utterance in c and each sentence in D interact with r, and distills matching signals from the interaction. Matching signals in all pairs are finally aggregated as a matching score in the aggregation layer.

2.3 Model Details

We elaborate each layer of the document-grounded matching network in this section.

Encoding Layer

Given an utterance ui in a context c, a sentence dj in a document D, and a response candidate r, the model first embeds ui, di, and r as Eui = [eui,1, · · · , eui,lu ], Edj = [edj,1, · · · , edj,ld ] and Er = [er,1, · · · , er,lr ] respectively by looking up a shared embedding table pre-trained with Glove [Pennington et al., 2014] on the training data D, where eui,k, edj,k and er,k are representations of the k-th words in ui, dj and r respectively, and lu, lr, and ld are lengths of the three sequences. Eui , Edj and Er are then processed by an attentive module to encode long-term dependency among words into the representations.

The attentive module simplifies the multi-head attention module in Transformer [Vaswani et al., 2017], and consists of a scaled dot-product attention component and a feed-forward component. Without loss of generality, let Q ∈ RnQ×d , K ∈ RnK×d , and V ∈ RnV ×d denote embedding matrices of a query, a key, and a value respectively, where nQ, nK, and nV are numbers of words in the input sequences, and d stands for embedding size. The scaled dot-product attention component is then defined as:

Intuitively, each entry of V is weighted by a relevance score defined by the similarity of an entry of Q and an entry of K, and then an updated representation of Q is formed by linearly combining the entries of V with the weights. In practice, we often let K = V, and thus Q is represented by similar entries of V. The feed-forward component takes Attention(Q,K,V) as input, and transforms it to a new representation by two non-linear projections. A residual connection [He et al., 2016] and a row-wise normalization [Ba et al., 2016] are applied to the result of each projection. For ease of presentation, we denote the whole attentive module as fATT(Q,K,V). ui, dj and r are then represented by attending to themselves through fATT(·, ·, ·):

Fusion Layer

The fusion layer grounds the conversation context by the document and fuses the information of the context into the document, which results in a document-aware context representation and a context-aware document representation. Formally, the document-aware representation of ui is given by ˆUi = [ ˆUi,1, · · · , ˆUi,m], where m is the number of sentences in the document, and ∀j ∈ {1, . . . ,m}, ˆUi,j can be formulated as

Similarly, the context-aware representation of dj is defined as ˆDj = [ ˆDj,1, · · · , ˆDj,n], where n is the number of utterances in the context, and ∀i ∈ {1, . . . , n}, ˆDj,i is calculated by

In ˆUi,j, information in dj provides grounding to ui, and correlations between dj and ui will be distilled to enhance the original representation of ui. The grounding is performed on a sentence-level rather than on a document-level (i.e., attention with a document vector). This is motivated by the intuition that sentences in a document are differentially important to represent the semantics of an utterance in a context, and the importance should be dynamically recognized through interaction with a response in the matching step. In a similar sense, by letting dj attend to ui in ˆDj,i we attempt to highlight important parts of dj through their correlation with ui, and thus achieve better document understanding in matching.

As we have analyzed before, utterances in a context are not always related to the background document in chat. To model this intuition, we append Ui to ˆUi as ˜Ui = [Ui, ˆUi,1, · · · , ˆUi,m] and determine if an utterance needs grounding with the guide of response r in the following matching layer. Ideally, if an utterance does not need grounding, then only Ui should participate in matching since other entries of ˜Ui are noisy. The weights of the entries of ˜Ui will be learned from training data.

Matching Layer

The matching layer pairs Ui, ˜Ui, ˆDj with R as {Ui,R}, {˜Ui,R} and {ˆDj,R} respectively, and extracts matching information from the pairs. Different from existing matching models that are solely based on conversation contexts, ˜Ui and ˆDj now contain grounding information from multiple sentences (utterances). Thus, the model needs to dynamically select important sentences (utterances) for grounding and even determine if grounding is necessary. To tackle the new challenges, we propose a hierarchical interaction mechanism. Take {˜Ui,R} as an example. For ease of presentation, we define Ui = ˆUi,0. Let rj denote the j-th entry of R, then the first level interaction of ˜Ui and R happens between rj and each ˆUi,k, ∀k ∈ {0, . . . ,m}, and transforms ˆUi,k into hi,j,k through

where ˆui,k,t is the t-th entry of ˆUi,k, and wa, va, and ba are parameters. Through Eq. (9), the first level interaction tries to play emphasis on important words in each ˆUi,k with respect to rj. The second level interaction of ˜Ui and R then summarizes [hi,j,0, . . . , hi,j,m] as hi,j by

where w′ a, v′ a, and b′ a are parameters. In the second level interaction, sentences in the document that can bring valuable grounding information for matching will play an important role in the formation of hi,j. As a special case, when α′i,j,0 is much bigger than other weights, the model judges that ui does not need grounding from the document. Finally, matching information between ˜Ui and R is stored in a matrix ˜Mi = [mi,1, · · · ,mi,lr ]. ∀j ∈ {1, . . . , lr}, mi,j is calculated by

where wp and bp are parameters, and ⊙ refers to elementwise multiplication.

Following the same procedure, we obtain ˆMj as a matching matrix for {ˆDj,R} where utterances in the context that are helpful for representing dj are highlighted by r. Since Ui is only made up of word representations (i.e., one-layer structure), the matching matrixMi for {Ui,R} is calculated by one level interaction parameterized in a similar way as Eq. (7)-(9) and the same function as Eq. (13).

Aggregation Layer and Learning Method

The aggregation layer accumulates matching signals in {Mi}ni=1 , { ˜Mi}ni=1 , and { ˆMj}mj=1 as a matching score for (D, c, r). Specifically, we construct a tensor from {Mi}ni=1 , and then apply a convolutional neural network [Ji et al., 2010] to the tensor to calculate a matching vector t. Similarly, we have matching vectors ˆt and ˜t for { ˆMj}mj=1 and { ˜Mi}ni=1 , respectively. The matching function g(D, c, r) is defined as

where wo and bo are wights, and σ(·) is a sigmoid function.

Parameters of g(D, c, r) are estimated from the training data D by minimizing the following objective:

-

3 Experiments

We test our model on two public data sets.

3.1 Experimental Setup

The first data we use is the PERSONA-CHAT data set published in [Zhang et al., 2018]. The data is collected by requiring two workers on Amazon Mechanical Turk to chat with each other according to their assigned profiles. Each profile is presented in a form of a document with an average of 4.49 sentences. The profiles define speakers' personas and provide characteristic knowledge for dialogues. For each dialogue, there are both original profiles and revised profiles that are rephrased from the original ones by other crowd workers to force models to learn more than simple word overlap. A revised profile shares the same number of sentences with its original one, and on average, there are 7.33 words per sentence in the original profiles and 7.32 words per sentence in the revised ones. The data is split as a training set, a validation set, and a test set by the publishers. In all the three sets, 7 turns before an utterance are used as conversation history, and the next turn of the utterance is treated as a positive response candidate. Besides, each utterance is associated with 19 negative response candidates that are randomly sampled by the publishers. More statistics of the three sets are shown in Table 2. Following the insights in [Zhang et al., 2018], we train models using revised profiles and test the models with both original and revised profiles.

In addition to PERSONA-CHAT, we also conduct experiments with CMUDoG data set published recently in [Zhou et al., 2018a]. Conversations in the data are collected from workers on Amazon Mechanical Turk and are based on movie-related wiki articles in two scenarios. In the first scenario, only one worker has access to the provided document, and he/she is responsible for introducing the movie to the other worker; while in the second scenario, both workers know the document and they are asked to discuss the content of the document. Since the data size for an individual scenario is small, we merge the data of the two scenarios in the experiments and filter out conversations less than 4 turns to avoid noise. Each document consists of 4 sections and these sections are shown to the workers one by one every 3 turn (the first section lasts 6 turns due to initial greetings). On average, each section contains 8.22 sentences and 27.86 words per sentence. The data has been divided into a training set, a validation set, and a test set by the publishers. In each set, we take 2 turns before an utterance as conversation history and the next turn of the utterance as a positive response candidate. Since the data does not contain negative examples, we randomly sample 19 negative response candidates for each utterance from the same set. Detailed statistics of the data is given in Table 2.

We employ r@k as evaluation metrics where k ∈ {1, 2, 5}. For a single context, if the only positive candidate is ranked within top k positions, then r@k = 1, otherwise, r@k = 0. The final value of the metric is an average over all contexts in test data. Note that in PERSONA-CHAT, r@1 is equivalent to hits@1 which is the metric used by [Zhang et al., 2018] for model comparison.

3.2 Baseline Models

The following models are selected as baselines. These models are the ranking models in [Zhang et al., 2018] and [Mazare et al., 2018] which perform much better than the generative models in [Zhang et al., 2018] on the PERSONA-CHAT data.

Starspace. A supervised model in [Wu et al., 2018] that learns the similarity between a conversation context and a response candidate by optimizing task-specific embedding via the margin ranking loss. The similarity is measured by the cosine of the sum of word embeddings. Documents are concatenated to conversation contexts.

Profile Memory. The model in [Zhang et al., 2018] that lets a conversation context attend over the associated document to produce a vector which is then combined with the context. Cosine is used to measure the similarity between the output context representation and a response candidate.

KV Profile Memory. The best performing model in [Zhang et al., 2018] which considers keys as dialogue history and values as the next dialogue utterances and uses a conversation Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

context as input to perform attention over the keys in addition to the documents. The past dialogues are stored in memory to help influence the prediction for the current conversation.

Transformer. A variant of the model proposed by [Vaswani et al., 2017] for machine translation. The model exhibits state-of-the-art performance on the PERSONA-CHAT data as reported in [Mazare et al., 2018].

All baseline models are implemented with the code shared at https://github.com/facebookresearch/ParlAI/tree/ master/projects/personachat and tuned on the validation sets. We make sure that the baselines achieve the performance on the PERSONA-CHAT data as reported in [Zhang et al., 2018] and [Mazare et al., 2018]. Note that we do not include models pre-trained from large-scale external resources, such as the FT-PC model in [Mazare et al., 2018], as baselines, since the comparison is unfair. On the other hand, it is interesting to study if pre-train the proposed model on those large-scale external data (e.g., the Reddit data in [Mazare et al., 2018] with over 5 million personas spanning more than 700 million conversations) can further improve its performance. We leave the study as future work.

3.3 Implementation Details

We set the size of word embedding as 300. In PERSONACHAT, the number of sentences per document is limited to 5 (i.e., m ≤ 5). For each sentence in a document, each utterance in a context, and each response candidate, if the number of words is less than 20, we pad zeros, otherwise, we keep the latest 20 words (i.e., lu = lr = ld = 20). In CMUDoG, we set m ≤ 20 and lu = lr = ld = 40 following the same procedure. In the matching layer of DGMN, the number of filters of CNN is set as 16, and the window sizes of convolution and pooling are both 3. All models are learned using Adam [Kingma and Ba, 2015] optimizer with a learning rate of 0.0001. In training, we choose 32 as the size of mini-batches. Early stopping on validation data is adopted as a regularization strategy.

3.4 Evaluation Results

Table 3 reports evaluation results on the two data sets. We can see that on both data sets, DGMN outperforms all baselines over all metrics, and the improvement is statistically significant (t-test, p-value < 0.01). Improvement fromDGMN over Transformer on the CMUDoG data is smaller than that on the PERSONA-CHAT data. The reason might be that Transformer can benefit from the wiki documents in CMUDoG that are longer and contain richer semantics than those handcrafted ones in PERSONA-CHAT.

3.5 Discussions

In this section, we investigate how different representations affect the performance of DGMN by an ablation study, visualize the example in Table 1 to illustrate how contexts are grounded by documents in DGMN, and check how the performance of DGMN changes with respect to document length.

Ablation study. First, we calculate a matching score only with the self-attention based context representation and response representation and denote the model as DGMN(t) which means only t is kept in Eq. (14). Then, we take the document-aware context representation into account, and denote the model as DGMN(t+˜t) in which both t and ˜t are used in Eq. (14). Based on t+˜t, we further examine if the special configuration for utterances that do not need grounding matters to the performance of DGMN by removing Ui from ˜Ui. The model is denoted as DGMN(t+˜t-NoGround). Finally, the context-aware document representation is considered, and we have the full model of DGMN. Table 3 reports evaluation results on the two data sets. We can conclude that (1) all representations are useful for matching; (2) some effect of the context-aware document representation might be covered by the document-aware context representation, as adding the former after the latter does not bring much gain; and (3) although simple, the special configuration for utterances that do not need grounding cannot be removed from DGMN.

Visualization. Second, to further understand how DGMN performs context grounding, we visualize the attention weights in formation of the document-aware context representation (i.e., ˆUi,j) and the weights in the second level of interaction (i.e., α′i,j,k in Eq. (11)) with the example in Table 1 in Introduction. Due to space limitation, we only visualize the last utterance of the context. Figure 2 shows the results. It is interesting to see that words like "work" and "education" are highly correlated in the graph, and at the same time, weights between the utterance and irrelevant sentences in the profile, such as "I am very social and love those close to me", are generally small. Moreover, in the second level interaction, while most function words and punctuation point to the utterance itself (i.e., u), the word "student" indicates that information from "i use all of my time for my education." is useful to recognize the relationship between the response candidate and the context. The example explains why DGMN works well from one perspective.

Performance analysis in terms of document length. Finally, we study the relationship between the performance of DGMN and document length by binning text examples in both data into different buckets according to the document length. Table 4 reports the evaluation results. On the PERSONA-CHAT data, both short profiles and long profiles lead to performance drop, while on the CMUDoG data, the longer the documents are, the better the performance of DGMN is. The reason behind the difference might be that profiles in the PERSONA-CHAT data are handcrafted by crowd workers, and thus semantics among different sentences are relatively independent, while documents in the CMUDoG data come from Wikipedia, and there is rich semantic overlap among sentences. Therefore, short profiles contain less useful information and long profiles contain more irrelevant information, and both will make the matching task more challenging. On the other hand, the longer a wiki document is, the more relevant information it can provide to the matching task.

4 RelatedWork

There are two groups of methods for building a chatbot. The first group learns response generation models under an encoder-decoder framework [Shang et al., 2015; Vinyals and Le, 2015] with extensions to suppress generic responses [Li et al., 2015; Mou et al., 2016; Xing et al., 2017; Tao et al., 2018]. The second group learns a matching model of a human input and a response candidate for response selection. Along this line, early work assumes that the input is a single message [Wang et al., 2013; Hu et al., 2014]. Recently, conversation history is taken into account in matching. Representative methods include the dual LSTM model [Lowe et al., 2015], the deep learning to respond architecture [Yan et al., 2016], the multi-view matching model [Zhou et al., 2016], the sequential matching network [Wu et al., 2017], the deep attention matching network [Zhou et al., 2018b], and the multirepresentation fusion network [Tao et al., 2019]. Our work belongs to the second group. The major difference we make is that in addition to conversation contexts, we also incorporate external documents as a kind of background knowledge into matching.

Before us, a few recent studies have considered grounding open domain dialogues with external knowledge. For example, Ghazvininejad et al. (2018) generalize the vanilla Seq2seq model by conditioning responses on both conversation history and external "facts". Zhang et al. (2018) release a persona-based conversation data set where profiles created by crowd workers constrain speakers' personas in conversation. Mazare et al. (2018) further increase the scale of the persona-chat data with conversations extracted from Reddit. Zhou et al. (2018a) publish a data set in which conversations are grounded in movie-related articles from Wikipedia. Dinan et al. (2018) release another document-grounded data set with wiki articles covering broader topics. In this work, we study grounding retrieval-based open domain dialog systems with background documents and focus on building a powerful matching model with advanced neural architectures. On the persona-chat data published in Zhang et al. (2018) and the document-grounded conversation data set published in Zhou et al. (2018a), the model improves upon state-of-theart methods with large margins.

5 Conclusions

We propose a document-grounded matching network to incorporate external knowledge into response selection for retrieval-based chatbots. Experimental results on two public data sets consistently show that the proposed model can significantly outperform state-of-the-art methods.

Acknowledgments

This work was supported by the National Key Research and Development Program of China (No. 2017YFC0804001), the National Science Foundation of China (NSFC Nos. 61672058 and 61876196).