Global Inference with Explicit Syntactic and Discourse Structures for Dialogue-Level Relation Extraction

1 Introduction

Dialogue-level relation extraction is a newly proposed task that seeks to infer the semantic relationships between the subject arguments and object arguments in a conversation [Yu et al., 2020], as exemplified in Fig. 1. Comparing with sentence-level RE [Katiyar and Cardie, 2016; Fei et al., 2020a] and document-level RE [Yao et al., 2019], DiaRE is much more challenging due to the characteristic of dialogues. Existing DiaRE studies handle multi-turn dialogues by concatenating all the utterances within it as a very long text sequence [Yu et al., 2020; Zhao et al. ], while recent studies model the dialogue context as various graphs so as to learn better feature representations [Chen et al., 2020; Xue et al., 2021; Nan et al., 2021; Qiu et al., 2021]. Unfortunately, current research still fails to dig into several pivotal issues of DiaRE, which hinders the task for further improvements.

∗Corresponding author

First of all, multi-party dialogue threads are scattered and entangled, and the semantics or topic consistency of each thread will be interrupted and damaged when simply concatenating the utterances into a long flat sequence. In fact, there could be a logical answering structure between utterances from different speakers (parties), as illustrated in the right part of Fig. 1. Second, the key to infer the relation of argument pairs lies in locating the crucial trigger clues in texts, for which the syntactic dependency tree features have been extensively and successfully exploited in regular RE [Miwa and Bansal, 2016; Fei et al., 2020d; Fei et al., 2020c]. However, the overall dialogue contexts are segmented into utterance pieces, which makes it intractable to directly apply the syntax structure information for DiaRE. Third, the speaker coreference ambiguity is not fully investigated in existing studies, i.e., the issue of first-person ('I'), second-person ('you') and zero pronoun of speakers in utterances would hinder the relation inference. Fourth, there could be complex relation inter-dependencies between the arguments in DiaRE, such as the multi-hop relations, implicit relations, reversed relations, etc. This requires an effective method for global-level argument reasoning.

In this work, we address all the above challenges for improving DiaRE. First, we model the entire conversation text into a hierarchical dialogue-level mixed dependency graph (D2G). As demonstrated in Fig. 2, the D2G is a directed acyclic graph by connecting 1) the inter-utterance structures including dialogue answering network & speaker coreference links, and 2) the intra-utterance structures including syntactic dependency tree & speaker-predicate links. D2G organizes the entire dialogue into a coherent dependency graph that explicitly integrates both the syntactic and discourse structures, which enables to more accurately capture the dialogue semantics and critical trigger clues for relation inference.

On the other hand, we perform end-to-end DiaRE with global relation reasoning. Conditioned on the argument mentions from D2G, we build a bidirectional argument reasoning graph (ARG) for direct argument relation inference. Overall, the system consistently refines argument representations at the global level based on the dual graph (i.e., both D2G and ARG), and finally outputs all the predicted relations between argument pairs via a triaffine decoder (cf. Fig. 3). Over both the English and Chinese DiaRE datasets, our system outperforms the current state-of-the-art (SoTA) models with big margins. Further analyses show the importances of constructing dialogue-level dependency structures and the global relation reasoning mechanism for DiaRE.

To sum up, this paper contributes mainly in three folds.

• ⋆We introduce a novel dialogue-level mixed dependency graph, D2G, which integrates syntactic and discourse structural information from various aspects. D2G enhances the overall semantic learning of dialogue contents and the feature retrieval of argument pairs.
• ⋆ We introduce an argument reasoning graph, ARG, for direct inference of the argument inter-dependencies. We aggregate the argument mentions in ARG from D2G via a conditional argument node normalization mechanism.
• ⋆ Our framework achieves new SoTA performances on benchmarks, and meanwhile yields explainable predictions. 1

2 RelatedWork

Relation extraction (RE) has long been a fundamental NLP task, aiming at discoversing argument relations in given texts [Katiyar and Cardie, 2016]. RE was upgraded from the initial sentence level to the document level, which recently has been introduced at the dialogue scenario, i.e., DiaRE [Yu et al., 2020]. The crux of RE is to deeply understand the context semantics and accurately retrieve the critical features for revealing the relations of argument pairs. Comparing with sentence-level and document-level RE, the relation inference in DiaRE could be much more difficult because of the nature of conversation form of texts, e.g., non-sequential order of discourse structure, scattered clues in different utterances, and speaker coreference.

1 Codes at https://github.com/scofield7419/DiaRE-D2G.

As a newly proposed task, currently DiaRE has received limited research attention. Initial DiaRE works [Yu et al., 2020; Zhao et al., 2021; Long et al., 2021] directly collapse the multi-turn dialogues into a document, i.e., transforming into the document-level RE. This however would disrespect the aforementioend conversation characteristics, and thus lead to suboptimal results. Very recent works consider constructing graph representations for DiaRE. For example, Chen et al., [2020] build a graph that connects the speaker, entity, type, and utterance nodes. Xue et al., [2021] and Nan et al., [2021] develop the latent graphs for DiaRE so as to better capture the key features for relation inference. Although improvements have been made, existing graph-based methods still fail to fully leverage the dialogue structural features, i.e., the dialogue discourse structure and the inner-utterance structure as we revealed previously.

This work also closely relates to the syntactic dependencybased RE methods [Xu et al., 2015; Fei et al., 2020b]. The external dependency structure provides intrinsic prior knowledge for mining the critical feature for relation inference from a low-level linguistic perspective [Fei et al., 2021b; Wu et al., 2021], which thus significantly promotes the RE performances [Miwa and Bansal, 2016; Song et al., 2019]. However, the syntax dependency information could not be directly applied to DiaRE task, as the dialogue contexts are segmented into utterance pieces with arbitrary order. In this work, we construct a novel hierarchical dialogue-level mixed dependency graph for DiaRE, representing the overall conversation as a coherent structure.

3 Dialogue-level Mixed Dependency Graph

We model a dialogue as a unified directed acyclic structure, i.e., dialogue-level mixed dependency graph as illustrated in Fig. 2. We formulate D2G as G=(V,E), where V is a set of nodes vi of words and speakers, and E is a set of labeled edges πu,v, with E=Eans∪Esco∪Edep∪Espd. Eans represents the dialogue answering edges between different utterances; Esco represents the speaker-coreference edges; Edep represents the sentence-level syntactic dependency edges; Espd represents the speaker-predicate edges. The former two types of edges refer to the inter-utterance structures while the latter two types of edges refer to the intra-utterance structures.

Dialogue answering edge. The dialogue answering structure can be seen as an inter-utterance dialogue discourse structure, ensuring a information flow from predecessors to successor with semantics consistency. Specifically, each conversational utterance ui will connect to a subsequent utterance uj (i<j) that either uj is a response to ui (i.e., crossspeaker case), or both ui and ui are yielded from a same speaker. For the cross-speaker case which is essentially a multi-turn response selection problem [Lu et al., 2019; Jia et al., 2020], we employ a well-trained off-the-shelf model to predict to which previous utterance the current one should link, and assign an 'ans' (answer) label for the edge. Also we directly link these utterances with same speakers and with a 'smsp' (same speaker) label.

Speaker coreference edge. Speaker coreference information should also be properly modeled. First, for the case of first-person pronoun of speaker (e.g., 'I', 'me', 'myself'), we directly link those pronoun words to the utterance speaker node with edge type 'fpp' (first-person pronoun). Then, for the second-person speaker pronoun (e.g., 'you'), we create 'spp' type of edges from the pronoun to the corresponding speaker(s) in the adjacent utterance(s) determined by the aforementioned 'ans' arcs. Note that here we do not consider the third-person pronoun case, due to its particular difficulty for disambiguation.

Syntactic dependency edge. We represent the innerutterance sentences by the syntactic dependency trees that are produced from a external third-party parser. In the syntactic dependency tree, each dependency edge links the head word to the dependent word with a specific syntactic label, e.g., 'Pheebs'nsubj 'help' shown in Fig. 2.

Speaker-predicate edge. We then connect the speaker to its corresponding utterance, so as to make the speaker&utterance a coherent unit, and meanwhile solve the zero pronoun issue (omitted reflexive pronoun). Technically, we create the speaker-predicate edges that an utterance speaker will link to the core predicate word of the utterance (with edge type of 'prd'). 2 When an utterance contains multiple sentences, we create multiple speaker-predicate edges, i.e., many predicates to one speaker projection.

4 DiaRE Framework

Task formalization. In DiaRE, a dialogue includes a sequence of utterances {un}Nn=1 , and a set of argument pairs A={(ai, aj )o}|A| o=1 . Each utterance is a sequence of words un={wn,1, · · · ,wn,m}, yielded by a corresponding speaker Sl ∈ {Sl}Ll=1 . An argument could either be an entity mention in utterances or a speaker. 3 Our system also creates a D2G for the corresponding dialogue texts. The target is to predict the relation label ri→j ∈ R between the subject argument ai and the object argument aj. 4 We also include a dummy label ϵ in R to represent no valid relation between ai and aj .

4.1 Base Encoding

Following the line of DiaRE works [Yu et al., 2020; Nan et al., 2021; Long et al., 2021], we also employ the pre-trained BERT language model [Devlin et al., 2019] as the underlying encoder to yield the contextualized representations for the words and speakers. We pack the utterance with its speaker as a group, and concatenate those groups into a whole (separated with SEP tokens), and feed into BERT encoder:

2 We note that mostly the core predicate of a sentence is the only child of the virtual Root node within the syntax tree.

3 89.9% of argument pairs have at least one speaker in the dataset.

4 The relation is directional, i.e., ri→j ̸= rj→i.

where hsl is the representation of speaker sl, and hwn,∗ is word representation, both of which will be used in the next module.

4.2 Global Relation Inference

Global relation reasoning module performs two learning targets: 1) critical feature mining for argument pairs, which is fulfilled based on the D2G encoder; 2) inter-dependencies inference for argument relations via the ARG encoder. Both of two graphs works collaboratively, performing the relation reasoning globally over the dialogue.

D2G encoding. Each D2G edge comes with a label. To encode D2G effectively, we here propose a novel label-wise graph convolutional network (LWGCN). In the graph G = (V,E), for each edge πi,j ∈ E from node vi to vj, we define πi,j=1 when there is an edge in between, and πi,j=0 vice versa. We additionally add a 'self ' label as the self-loop arc (i.e., πi,i=1) for each node vi to enrich the information aggregation. We also maintain the vectorial embedding xπi,j for each edge label. We denote the LWGCN hidden representation of node vi as ei:

(2) where hj is the node representation from BERT encoder (cf. Eq. 1), and γi,j is the linking distribution calculated via:

γi,j indicates the structural neighboring connecting strength globally, which will be dynamically updated during learning so that some important clues will be highly weighted and lead to more accurate relation detection.

ARG encoding. In DiaRE, the direct information exchanging of different arguments should be considered for sufficient relation inference (e.g., argument inter-dependencies). Thus we build an ARG, in which we create fully bidirectional connections between each argument pair, and the argument mentions are aggragated from D2G, 5 as depicted in Fig. 3. We introduce a novel conditional graph neural network (ConGNN) to encode ARG. Unlike the vanilla GNN that makes direct propagation among nodes, in ConGNN, the neuron's activity of argument mention node is normalized from D2G, so as to reduce the covariate shift problem that causes imbalanced inference [de Vries et al., 2017; Xiong et al., 2020]. Technically, ConGNN passes messages for each argument ai as:

where B is the connecting weight between an argument pair withBi,j = 1, and ¯D = ∑v Bi,j = 1. ˆ eai is the conditionally normalized node representations (ConNorm):

5 One argument entity is often mentioned more than once in D2G, and scattered broadly within the dialogue.

where eai,j is the j-th element of vector eai , µ and σ are the mean and standard deviation of the normalization. ConNorm generates α and β by aggregating the raw mention representation edi in D2G:

where V is the number of the mention representation edi,u that refers to the same argument ai.

Global inference with two graphs. The global relation reasoning assembles the above two graph encoders as a whole, consistently performing feature learning and refining the argument relations globally. Overall, we enable total K rounds of reasoning for a sufficient information propagation.

4.3 Prediction and Training

Based on the argument representations eai we finally perform end-to-end prediction for all the argument pairs. Most prior works simply concatenate two representations for prediction. However, this could inevitably lead to the order information loss between the subject and object arguments, as the DiaRE task is sensitive to the order of the argument pair. Also some global context information is not utilized in existing works. We thus employ a TriAffine decoder [Carreras, 2007] that makes decisions based on the two argument features (in order) as well as a global context feature:

where eas and eos are the subject and object argument representations fromARG, ec is global context representation:

where hCLZ is the BERT representation of 'CLZ' token, eD is the average pooling representation over all the last-layer of LWGCN node features {edi }. Only the predicted label that is valid (i.e., ri→j ̸= ϵ) will be output.

The training target of our system is to minimize the crossentropy loss L between the predicted and ground truth labels of all the relations.

5 Experimentation

5.1 Setups

We conduct experiments on the DiaRE benchmark data [Yu et al., 2020], which includes the English version (DialogREEN) and the Chinese translation version (DialogRE-CN). DiaRE data is split into Train&Dev&Test sets, and totally contains 1,788 dialogues and 10,168 relational triples, covering 36 relation types, with average of 13.1 utterance per dialogue and average 3.3 speaker per utterance. To yield dialogue answering edges Eans, we adopt the current SoTA multi-turn response selection model [Jia et al., 2020]. We employ the Stanford CoreNLP Toolkit 6 to obtain the dependency parse trees Edep. We load the base version BERT parameters.

6 https://stanfordnlp.github.io/CoreNLP/, v4.2.0 typed version.

All the BERT output representation h has 768 D. The edge label embedding (xπi,j ) size is 100. LWGCN hidden size, argument embedding size and ConGCN hidden size are all set as 300. We adopt the Adam optimizer with an initial learning rate of 4e-5. We set unfixed epochs with an early-stop training strategy instead. We mainly make comparisons with the existing DiaRE baselines. All the baselines use the same BERT-base embedding. We adopt the F1 score as the metric.

5.2 Results and Analyses

Main performances. In Table 1 we compare the main performances against baseline DiaRE systems. The first observation is that the RawBERT model that collapses the entire diagloue texts as a flat document also without using any other information source, presents comparatively weaker performances. In contrast, those baselines that either take the graph modeling of dialogue (i.e., HeterGraph, LatGraph), or make use of additional information (i.e., AuxPrd, End2End, Position, CoIN) achieve better results than RawBERT model.

Most importantly, our model outperforms the bestperforming baselines with big margins, e.g., 2.2%(74.5-72.3) on DialogRE-EN and 3.4%(69.8-64.4) test F1 on DialogRECN respectively over the CoIN model. We note that CoIN is the SoTA baseline because of the design of multiple learning constraints [Long et al., 2021]. However, CoIN becomes inferior to our system, largely due to the leverage of dialoguelevel dependency mixed graph and the global relation reasoning mechanism in our method.

Ablation study. We perform ablation experiments (cf. Table 2) to better understand the impact of each part of our proposed method. We first study the influence of the D2G by removing it, which substantially results in the biggest performance drops among all the other factors, i.e., -7.4% and -6.4% F1 scores on two datasets respectively. This reflects the importances to build a dialogue-level structure for DiaRE. Diving into the D2G, we remove each sub-structure one by one, and find that the syntactic dependency links are the most important element, followed by the dialogue answering edges. Besides, without encoding the edge label information, the results drop about 2 points.

Further, removing the ARG also hurts the overall performances to certain extent (not as significant as without D2G). When the condition normalization mechanism of argument nodes is not available, considerable drops are witnessed, which proves the necessity of its proposal. Finally, we find that stripping off the the global context features ec will lead to performance degradation. If further using a concatenation operation as a replacement for generating the feature representation, i.e., [eas ; eao ] and without considering the argument order, we can meet further performance decreases.

Influence of the global reasoning round. In Fig. 4 we study how the global reasoning round affects the model performances. We see that both the performance of English and Chinese data climbs to the peak when gradually stepping into the third iteration. This informs that K=3 is enough to ensure sufficient sentiment and context learning. Once over third rounds, the overall results are deteriorated rapidly, largely due to the overfitting by too many reasoning steps.

Influence of the number of speaker parities. In multiparty dialogues, more speakers will introduce more conversation threads, i.e., more complex dialogue semantics. In Fig. 5 we show the results under different numbers of speakers. We see that without leveraging the inter-utterance structure in D2G (i.e., Eans and Esco), our system could perform worse rapidly when handling multitudinous speakers, e.g., the speaker number is larger than 6. This evidently proves the necessity to model the cross-utterance information for DiaRE.

Influence of the relational argument number. Fig. 6 further plots the performances under different co-existed argument numbers in a dialogue. We notice that increasing the arguments causes worse overall results, since more arguments could lead to complicated relation inter-dependencies. In particular, without the integration of argument reasoning graph, the performances of our system on the bigger number of arguments (e.g., ≥11) are hurt dramatically, which demonstrates the efficacy of the ARG.

Influence of the distance of an argument pair. Sentencelevel syntactic dependency tree knowledge has been extensively verified effective on relieving the long-range dependence issue in sentence-level relation extraction [Xu et al., 2015; Fei et al., 2020b; Wu et al., 2021; Fei et al., 2021a]. Here we explore the results of different systems when handling the argument pairs in different distances in DiaRE scenario. As shown in Fig. 7, we see that our system equipped with the dialogue-level dependency structure can still perform well for those super-long argument pairs, where the other baselines fail to give competitive performances (e.g., AG- GCN). This implies the importance to construct the dialoguelevel mixed dependency graph for DiaRE.

Structural correlation discovery. Finally, we qualitatively investigate if our system can capture the intrinsic correlations between the dialogue dependency structures and the argument relations. We technically observe the connecting weights γi,j (in Eq. 4) and collect the weights of the correlated edges and the argument relation types, which are normalized and rendered in Fig. 8. Interestingly, via some patterns we can infer that our system has successfully learned some structural correlations, which accordingly explains the task improvements by our model. For example, the interutterance edges dialogue answering ('ans') show bigger influence to most of the relation types, while the relation types 'alternate names' and 'title' rely more on the speakerpredicate ('prd') edges. This also reveals that our model can achieve explainable predictions for DiaRE.

6 Conclusions

In this paper, we introduce a novel system for dialogue-level relation extraction (DiaRE) task. We first propose modeling the conversation texts as a dialogue-level mixed dependency graph for more accurate feature learning, in which we integrate both syntactic and discourse information. We then introduce an argument reasoning graph with a conditional argument node normalization mechanism for direct inference of the argument inter-dependencies. Our framework achieves new state-of-the-art results over best-performing baselines on two DiaRE benchmark datasets.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No.61772378, No. 62176187), the National Key Research and Development Program of China (No. 2017YFC1200500), the Research Foundation of Ministry of Education of China (No.18JZD015), the Youth Fund for Humanities and Social Science Research of Ministry of Education of China (No. 22YJCZH064), the General Project of Natural Science Foundation of Hubei Province (No.2021CFB385). This work is also supported by the Fundamental Research Funds for the Central Universities.