Enhancing Text Generation via Multi-Level Knowledge Aware Reasoning

1 Introduction

Text generation aims to produce realistic and reasonable textual content that is indistinguishable from human-written text, which is helpful for question answering, story generation [Mostafazadeh et al., 2016], etc.

Existing works generally ground the generative models on word-level knowledge [Zhou et al., 2018; Ji et al., 2020] and successfully generate informative text. However, these methods may produce unreasonable text, because word-level knowledge cannot express multi-word text units. For example, in Figure 1, the existing state-of-the-art (SoTA) model GRF [Ji et al., 2020] utilizes the word-level knowledge to generate an unreasonable sequence. This is because the wordlevel knowledge cannot reflect the actual semantics of the multi-word expression "scaredy cat".

Real-world knowledge exists at the word-level, the eventlevel, etc. It is inadequate for a text generation system to just ground on word-level knowledge. To make high-quality text generation, event-level knowledge also makes sense. An event is a tuple containing a subject, a verb, an object, and some additional token(s) [Radinsky et al., 2012]. As the basic semantic unit of natural language, an event carries richer information than a single word, hence event knowledge might help the event-based content planning, and conditioned on event knowledge may help reasonable and diverse text generation. However, event knowledge is very sparse, which brings about the difficulty to make the reasonable contentplanning. To solve this problem, we split a coarse-grained event into fine-grained word components to obtain the wordlevel knowledge among event components. The word-level knowledge models the interaction among event components, which makes it possible to reduce the sparsity of events.

Based on the event-level and the word-level knowledge, we devise a novel Multi-Level Knowledge aware Reasoning (MKR) framework for text generation. Our framework generates text through the following three steps. First, it uses a graph-based module to learn structure-aware embeddings for words and events (§ 3.2). This step models the interaction between event components, which makes it possible to reduce the sparsity of events. Following, it utilizes the event-level knowledge to make event-based content planning (§ 3.3), i.e., select reasonable event sketches conditioned by the input text. We use the selected event sketches as guidance for text generation. Finally, we combine the word-level knowledge with the selected event sketches (§ 3.4), to make full use of multilevel knowledge for text generation.

Our contributions can be summarized as follows: (1) We propose to split a coarse-grained event into fine-grained word components to obtain the multi-level knowledge. The multilevel knowledge models real-world knowledge in the eventlevel and the word-level, as well as models the interplay between event components; (2) We propose a novel multi-level knowledge aware reasoning framework for text generation. It utilizes both the event-level and the word-level knowledge, therefore it can generate high-quality text.

Experimental results on two widely used datasets demonstrate the effectiveness of our framework. We also make an in-depth analysis of event-level knowledge and word-level knowledge. The combination of two kinds of knowledge leads to the best generation results, which indicates that there is complementarity between the two kinds of knowledge.

2 RelatedWork

2.1 Knowledge-Aware Text Generation

Existing text generation methods focus on utilizing wordlevel knowledge to augment the limited textual information. [Zhou et al., 2018] enriched the representations of the input text with neighbouring concepts on ConceptNet. [Zhang et al., 2019a] explicitly models the conversation flow with the commonsense knowledge and guides the conversation flow in the latent concept space. [Ji et al., 2020] integrates external knowledge into pretrained language models to generate more informative texts. Nevertheless, these methods just utilize word-level knowledge. Different from those works, we introduce multi-level knowledge to enhance text generation.

2.2 Text Generation with Content Planning

Text generation with content planning first identifies pertinent information to present, and then realizes the pertinent information into surface text [Holmes-Higgin, 1994]. [Puduppully et al., 2019] incorporates the end-to-end content planning to data-to-text generation. [Xu et al., 2020] enhanced the dialogue system by making the content planning grounded on narrative chains. [Goldfarb-Tarrant et al., 2020] makes a plotguided content planning and demonstrates its use in longform story generation. In this paper, we make the content planning based on the event-level knowledge. In addition, we introduce the multi-level knowledge graph to reduce the sparsity of events, which is less studies in previous works.

3 Method

3.1 Preliminary

BuildingMulti-Level KnowledgeGraph Given a context, we first construct a multi-level knowledge graph G for it. To obtain event-level knowledge for text generation, we use ATOMIC [Sap et al., 2019] and COMeT [Hwang et al., 2020]

as event knowledge base. ATOMIC is a repository of inferential if-then knowledge. COMeT is a transformer model trained on ATOMIC that generates nine kinds of inferences of events in natural language. Given the input context, we first extract central events from the context and feed the central events into COMeT to generate one-hop events with corresponding relations. The one-hop events are then fed into COMeT to generate two-hop events. After that, we obtain many event paths. Several heuristic rules are applied to filter low-quality paths. For example, if an event in a path contains less than 2 words, the path will be discarded. There are still many event paths left, we randomly select at most 80 paths. We next split each event into word components. For each component, we expand at most three word-level relations from ConceptNet 1 . Finally, we look up word-level relations among all word pairs. In this way, originally unrelated events can interact through word-level relations between event components. Figure 2 illustrates the construction process. We heuristically use the word-overlap ratio to label words and events. That is, a word is labeled as positive if it is contained by the gold sequence. An event is labeled as positive if 70% of event components are contained by the gold sequence. The event labels are used as supervision for selecting event sketches (§ 3.3), and word labels are used as supervision for gate control (§ 3.4) in the decoding phase.

Task Definition The input X = (x1, x2, · · · , xM) is a text sequence which may consist of several sentences. We aim to generate another sequence Y = (y1, y2, · · · , yN). According to X, we extract a multi-level knowledge graph G. There are two types nodes in G: (1) word nodes Vw; (2) event nodes Ve. Each event ei ∈ Ve contains several word components ei = {wi1, · · · ,wik}, (wi1, · · · ,wik ∈ Vw). Also, There are two types of edges in G: (1) event-event relation (he, re, te) indicating the head event he has a relation re to the tail event te; (2) word-word relation (hw, rw, tw) indicating the head word hw has a relation rw to the tail word tw, where hw and tw may be the components of different events. We decompose the text generation task into two steps. The first step uses the event knowledge to make a event-based content planning, i.e., select reasonable event sketches Ek conditioned by the X. In the second step, we use word-level knowledge to generate the text sequence given X and Ek. Our model maximizes the following conditional probability:

3.2 Multi-Level Knowledge Graph Encoding

We use Relational Graph Convolutional Network (RGCN) [Schlichtkrull et al., 2018] to encode our multi-level knowledge graph to learn structure-aware embedding of words, events, and relationships. Given a RGCN with L layers, for each word tw ∈ Vw, we update its word embedding at the l + 1-th layer by aggregating its local neighbours N(tw) including pairs of the word hw and the connected relation rw:

1 In ConceptNet, each relation is assigned with a weight. We choose 3 relations with the highest weight.

where h0tw is initialized by pretrained embedding and h0rw by randomly, Ztw = |N(tw)| is the number of neighbors of tw, Wls and Wla are trainable specific to the l-th layer, σ is the RELU activation. We also update embeddings of word-word relations via another linear transformation hl+1 rw = Wlr hlrw . After L iterations, words embeddings hLhw , hLtw and relations embeddings hLrw are obtained. Each event embedding is obtained through pooling on its word components:

3.3 Event Content Planning via Event Reasoning

Given the context X and the graph G, we perform the eventbased content planning to produce the event sketches.

We first encode X = (x1, x2, · · · , xM) into the hidden state hX ∈ Rd (d is the hidden size):

where hxm is the vector of the m-th token, Wx is trainable.

Then we perform the event-level reasoning to select event sketches. Because how to realize reasoning is not the focus of this paper, we directly adopt the existing approach: multihop reasoning flow [Ji et al., 2020]. We iteratively compute the relevance scores of one-hop events and two-hop events in accordance with the context, as shown in Figure 3. In each iteration, the module parallelly computes relevance scores of events which are in the same hop. For the event te, the score cs(te) between te and X is computed by aggregating evidence from its neighbours Nte including pairs of the connected event he and the connected edge re:

where γ (0.5 by default) is a discount factor that controls the score flow from the previous hops. Initially, zero-hop events (central events) are given a score of 1, while other events are assigned with 0. R(·) is the relevance of the relation (he, re, te) under the context hX, which is calculated by

According to the scores of all events, we select the top-k events Ek = topki (cs(ei)), which are used as guidance for text generation. We also explore the influence of k in the experiments.

3.4 Generation by Combining Event Sketches and Word Level Knowledge

We use the selected events Ek as guidance for text generation. To make full use of multi-level knowledge, we also use word knowledge in generation process. Specifically, taking the context X, the multi-level graph G and the event sketches Ek as input, the generation module first encodes X and Ek to obtain the guided context vectors HC = BARTEncoder([X;Ek]) where [·; ·] denotes concatenation operation, HC ∈ Rc×d (c is the total length of [X;Ek]). The hidden state of t-th time step of the target sequence hyt is computed by:

The word distribution of t-th time-step over the standard vocabulary V is

The generation module then performs word-level reasoning to consider the word level knowledge according to the current decoder state hyt . For the word tw, the module computes the score cs(tw) between tw and [X;Ek] by aggregating evidence from its neighbours Ntw including pairs of the connected word hw and the connected edge rw:

where R(hw, rw, tw) is computed by:

Initially, the zero-hop words, which are existed in X, are given a score of 1, while other words are assigned with 0. After H iterations, the distribution over all words in G is:

where yw t ∈ Vw is the selected word at the t-th time step.

The final token distribution combines the distribution over word nodes in Vw and the distribution over the standard vocabulary with a soft gate gt = sigmoid(Wghyt ), which determines whether to copy words from G when generation:

3.5 Model Training

To train the event planning module, we minimize the crossentropy loss of selecting positively-labeled events by:

where p(ei) = sigmoid(cs(ei)) denotes the probability that the event ei is selected, li is the label of ei, Z is the number of events in G.

To train the text generator, we minimize the NLL loss of generating the gold sequence:

where Ek is the selected event sketches (§ 3.3). We additionally add the gate loss Jg to supervise the training of gt, the loss takes the form of binary cross-entropy. The final loss of the generator is JG = JNLL + αJg where α is set to 0.5 by default. The final loss is J = JP + JG. The event planner and the text generator are jointly training.

4 Experiments Settings

4.1 Datasets

Story Ending Generation (SEG) is to generate a reasonable ending given a four-sentence story context. The stories come from ROCStories [Mostafazadeh et al., 2016] corpus. Following [Yao et al., 2019], we randomly split the dataset into 8:1:1 for training, validating and testing.

Abductive NLG (αNLG) is to generate an explanatory hypothesis given two observations: O1 as the cause and O2 as the consequence. We use the official data split.

4.2 Baselines and Implementation Details

GPT2-FT is a GPT-2 model fine-tuned on the task-specific dataset with initialization from [Radford et al., 2019]. T5FT is a T5 model fine-tuned on the task-specific dataset with initialization from [Raffel et al., 2019]. BART-FT is a BART model fine-tuned on the task-specific dataset with initialization from [Lewis et al., 2019]. GPT2-OMCS is a commonsense-enhanced GPT-2 model first post-trained on the Open Mind Common Sense (OMCS) corpus5 from which the ConceptNet is constructed. The model is then fine-tuned on the task-specific dataset. CALM-T5 [Zhou et al., 2020], which utilizes the self-supervised contrastive learning to inject word-level (nouns and verbs) knowledge into pre-trained T5 model. GRF-GPT2 [Ji et al., 2020], the current SoTA model on the used datasets, is a GPT2 based model which generates text with multi-hop reasoning on structural knowledge graphs. For the sake of fairness, we also use the BART pretrained model to reproduce GRF and name it GRF-BART.

Our method employs the base version of the BART model, and a 2-layer RGCN module. To train the model, we use the Adam optimizer with β1 = 0.9, β2 = 0.999, ϵ = 10-6 and linearly decrease learning rate to zero with no warmup. We search for the best hyper-parameters according to BLEU-2 on the development set of each dataset. At the inference stage, we adopt beam search decoding with a beam size of 3 for our model and all the baselines we produce.

5 Results and Analysis

5.1 Automatic Evaluation

Metrics include: BLEU-n [Papineni et al., 2002], METEOR [Banerjee and Lavie, 2005], BertScore [Zhang et al., 2019b] and Distinct-n [Li et al., 2015] .

SEG

Our Method vs. Baselines

The result is shown in Table 1. In the used pretrained models, BART performs best, so we use BART as our backbone. Our implementation of GRF (GRF-BART) performs better than the original one (GRF-GPT2), the improvement is caused by the BART model. GRF-BART performs better than GPT2OMCS and CALM-T5. This indicates that explicitly performing reasoning on structural knowledge graphs is more effective than implicitly injecting knowledge into PLMs. This coincides with [Ji et al., 2020]. Compared with GRF-BART, our model achieves a notable improvement. This is because GRF-BART only utilizes the word-level knowledge, and our method utilizes the event-level knowledge as well. The distinct score of our method is more significant than baselines, which indicates that our method can generate more diverse text. This may be because the multi-level knowledge provides diverse background information for text generation. The overall result indicates that our framework is effective for improving the quality of text generation.

Ablation Study

Settings To investigate the effectiveness of different components, we further conduct ablation study to compare our model with the following variants: (1) "w/o Jp" means we do not supervise the event content planning module; (2) "w/o MKE" means we remove the multi-level graph encoding module, that is we do not model interaction between event components; (3) "w/o ECP" means we do not utilize event knowledge and only use word knowledge when decoding stage; (4) "w/o GWK" means we do not utilize word knowledge when decoding stage.

SEG

Result The result is shown in Table 2. Each component contributes to text generation. "w/o MKE" leads to a performance drop, this is because the model does not model the interaction between event components, and the sparsity of events damages the event content planning. "w/o Jp" leads to a larger drop compared to "w/o MKE". This coincides with human intuition that the event labels help to make a more reasonable event sketches selection. In addition, we have the following observations. (1) Compared with word-level knowledge, event-level knowledge is more important for text generation. For example, the removal of event knowledge ("w/o ECP") leads to a huge performance drop. This makes sense, because event knowledge is naturally more diverse than word knowledge and carries richer information than word knowledge. Hence event guidance enables the model to generate more diverse and high-quality text. (2) Although not as important as event knowledge, word knowledge is also useful for text generation. The best results (our full model) are obtained by combining the word-level knowledge and the event -level knowledge. The reasons lie in two aspects: (1) wordlevel knowledge helps event representation learning, which is helpful for making reasonable event sketches planning; (2) Being aware of multi-level knowledge, a model has the ability to generate high-quality text.

5.2 Manual Evaluation

Criteria includes informativeness and reasonability. When evaluating informativeness, the annotators are required to assess whether the generated text produce unique and nongenetic information that is specific to the input context. When evaluating reasonability, annotators are asked to focus on evaluating the causal and temporal relevance of the generated results and the contexts. We carried out pairwise comparison with BART-FT, GRF-BART, and two ablated models "w/o ECP" and "w/o GWK". We randomly sample 100 cases from the two testsets respectively for each pair of models. Three annotators are recruited to make a preference among win, tie and lose given the input context and two outputs generated by our model and a baseline respectively. The annotators are research students from the field of text generation to make sure they have a fair judgement of used metrics. The result is shown in Table 3. Our full model outperforms all compared models. We calculate the Fleiss's kappa reliability as the inter-rater agreement. For αNLG, the agreement of informativeness and reasonability is 0.497 and 0.459, respectively. And for SEG, the two values are 0.424 and 0.451.

5.3 Deeper Analysis of Multi-Level Knowledge

We then investigate how multi-level knowledge helps text generation. The compared models include GRF-BART, our full method MKR, and two ablated versions: "w/o ECP" and "w/o GWK". We divide the test samples of the SEG task into four groups, according to whether the multi-level knowledge graph of a test sample contains at least a positively-labeled event or a positively-labeled word 2 . The number of samples in each group is (A) 374, (B) 1476, (C) 1369 and (D) 6596. In each group, we calculate the BLEU-2 score of generated text of different models. From Figure 4, we observes that:

(1) In group A and B, it is possible for "w/o GWK" to select a supporting event for generation, hence "w/o GWK" performs far better than "w/o ECP" and GRF-BART.

2 For simplicity, we call positively-labeled events or words as supporting events or words.

1. In group C, the result gap between "w/o ECP" and "w/o GWK" was the smallest. This is because it is possible for "w/o ECP" to select the supporting words for generation, but it is impossible for "w/o GWK" to select any supporting event. This reduces the performance gap between the two variants. Note that event knowledge is more sparse than word knowledge. When a multi-level knowledge graph does not contain supporting events, it may contain supporting wordlevel knowledge, which is also useful for text generation.

(3) In all groups, the best result is obtained after combining the word-level with event-level knowledge. This is because the model is aware of multi-level knowledge, so it is more easier to generate high-quality text.

5.4 Further Discussion

Impact of the training data size In Figure 5, we gradually reduce the training data size of αNLG, and test on the original testset. Our method achieve consistent performance gains, even when given extremely small training data (10%), but the result of GRF-BART drops more dramatically. This demonstrates the generalization ability of our model with the aid of multi-level knowledge.

Impact of the number of selected events We investigated the impact of the number of selected events on the generation quality on the αNLG testset, as shown in Figure 6. Most of metrics first increase, reach the maximum at k = 2, and then decrease. On the one hand, the greater the k, the more likely it is to select the support event. On the other hand, the more selected events, the more likely it is to select irrelevant events. When k = 2, the two aspects reach the good balance, so our method obtains the best performance. Therefore, we set k = 2 for all the main results of our model.

5.5 Case Study

We provide two cases which are from the αNLG testset, and present the generated sentences of corresponding models in Table 4. We find that: (1) Baseline models sometimes fail to generate reasonable sentences, i.e., the generated text of BART-FT in the first case. In contrast, our method generates reasonable sentences which are causally relevant to the context. (2) By utilizing the multi-level knowledge, our full model generates more informative sentences than the model "w/o GWK" which only utilizes the event-level knowledge.

We visualize a part of the event-level knowledge of the first case, as shown in Figure 7. The event "get nicotine patch" receives the highest score, follows by the event "buy nicotine patch". The two events are selected as event sketches and are used for generation.

6 Conclusion

We present a multi-level knowledge aware reasoning framework for text generation. It is a two-stage reasoning framework which utilizes both the event-level and the word-level knowledge. Experiments demonstrate that word-level knowledge and event-level knowledge complement each other, and both contribute to text generation.

Acknowledgements

This work is supported by Research Grants Council of Hong Kong(PolyU/15207920, PolyU/15207821), National Natural Science Foundation of China (62076212) and PolyU internal grants (ZVQ0, ZVVX).