Neutral Utterances are Also Causes: Enhancing Conversational Causal Emotion Entailment with Social Commonsense Knowledge

1 Introduction

Conversation is the most omnipresent medium for people to express daily emotions. Capturing emotions is critical for machines to understand conversations, which leads to researchers [Majumder et al., 2019; Ghosal et al., 2019; Shen et al., 2021b] devoting into emotion recognition in conversations (ERC). However, further reasoning why an emotion is aroused in conversations is still under explored [Poria et al., 2021]. Understanding emotion causes is also a critical step for machines to understand conversations, which can be further utilized in other high-level tasks like empathetic generation [Kim et al., 2021]. Therefore, Poria et al. [2021] propose a new task called Recognizing Emotion Cause in Conversations (RECCON). RECCON involves an important subtask called Conversational Causal Emotion Entailment (C2E2). C2E2 detects causal utterances for a targeted utterance from conversations where the emotion is given.

∗ Joint work with Pattern Recognition Center, WeChat AI, Tencent Inc, China.

†Zheng Lin and Peng Fu are the corresponding authors.

Implied by the name, C2E2 is originally modeled as a "text pair" task by Poria et al. [2021]. In this way, a candidate utterance and the targeted utterance are paired plus their conversational context and the emotion. They are concatenated and then processed by a pretrained model. However, this entailment style ignores the explicit interactions between utterances, thus contextual information from conversations not fully captured. To explicitly model the interactions between utterance representations, works [Ghosal et al., 2019; Shen et al., 2021b] in ERC adopt various types of graphs for conversations, which can achieve decent performance.

As emotion plays a crucial role in C2E2, it is utilized in the entailment [Poria et al., 2021]. When conversations are constructed as graphs, it is also critical to consider emotion information. An intuitive way is to represent emotion as emotion embedding [Liang et al., 2021] and concatenate it with the node representation. Experiments show that emotion information can remarkably boost the performance. Specifically, we find that it can dramatically promote the detection of causal utterances whose emotion is the same as the targeted utterance. However, causal utterances with different emotions, especially neutral ones (neutral causal utterances occupy 87% of this kind of causes), is still hard to detect even with emotion information. We think the reason lies in that models are limited in reasoning causal clues and passing them between utterances. For example, in Fig. 1, the 2nd utterance u2 is a cause to the 3rd utterance u3. Models cannot reason why a neutral utterance about misdialing can evoke sadness to another utterance, leading to a wrong prediction.

To boost models' causal reasoning ability, we introduce social commonsense knowledge (CSK) [Hwang et al., 2021] and propose a graph-based structure to properly utilize CSK. Therefore, causal utterances with different emotions from the targeted utterance can be further detected. Specifically, social CSK can infer a person's social interactions like mental states and reactions. Back to Fig. 1, equipped with the knowledge, models can infer that u2 makes the speaker of u3 feel upset and then give a right prediction. To properly utilize knowledge, we propose a Knowledge Enhanced Conversation graph (KEC). KEC passes knowledge through edges. As the knowledge should align with speaker interactions and emotions, we therefore propose a speaker and sentiment realized knowledge selecting strategy, which can pick up suitable knowledge for utterances. Furthermore, to process the KEC, we extend the Directed Acyclic Graph networks (DAG) for ERC [Shen et al., 2021b] to build Knowledge Enhanced DAG networks. To verify our method, we conduct experiments on the RECCON dataset. Our method significantly outperforms the baseline of RECCON and other highly related methods. In addition, our method promotes the detection of causal utterances with different emotions from the targeted utterance.

2 RelatedWork

Conversational Emotion Recognition (ERC) is a highly related task to C2E2. ERC tends to predict the emotion of utterances in conversations. In ERC, a main stream of works [Majumder et al., 2019; Ghosal et al., 2019; Shen et al., 2021a] focuses on studying the speaker modelling. Although ERC is a hot-spot task for researchers to study conversations, further reasoning about emotions and understanding the causes in conversations is still lack of exploration.

Emotion Cause Extraction (ECE) is to extract the causal clause for an emotion clause from a document and Gui et al. [2016] construct the most popular ECE dataset. Ding et al. [2019]; Xia et al. [2019] introduce position embeddings to model causal clauses appeared near the emotion clause. As emotion and cause are highly related, Turcan et al. [2021] propose multi-task learning frameworks for emotion recognition and cause extraction. To simultaneously extract emotion and cause clauses from documents, Xia and Ding [2019] propose a new task called Emotion Cause Pair Extraction (ECPE) and handle it in a two-stage style. To overcome the problems brought by two-stage processing, some works [Ding et al., 2020a; Wei et al., 2020; Ding et al., 2020b] propose their novel joint frameworks. Both ECPE and ECE are only extract causes in articles, which does not extend to the scene of conversations. Conversations are more complicated due to the flowing emotion dynamics and speaker participation [Ghosal et al., 2019]. To study the causal factor of emotion in conversations, Poria et al. [2021] propose Recognizing Emotion Causes in Conversations (RECCON). RECCON involves two subtasks: Conversational Causal Span Extraction (C2SE) and Conversational Causal Emotion Entailment (C2E2). C2SE focuses on extracting causal spans from conversations, which is modelled as a machine reading comprehension task by Poria et al. [2021]. In this work, we focus on C2E2, the classification version of RECCON.

Commonsense Knowledge Utilization (CSK) can bring rational external knowledge for models. In ERC, some works [Zhong et al., 2019; Ghosal et al., 2020; Li et al., 2021] study how to use CSK to help understand conversation context and mental states of speakers. In ECE, Gui et al. [2021] propose Knowledge-Aware Graph using CSK to alleviate position bias brought by position embeddings. Turcan et al. [2021] use CSK to enhance ECE and Emotion Detection. In the task of Sentence Ordering, sequential commonsense knowledge is structurally utilized by Ghosal et al., [2021]. However, for C2E2, it is still under explored how to properly use CSK.

3 Method

3.1 Task Definition

C2E2 is to find causal utterances for a targeted utterance from the historical context. Given a conversation C = [u1, ..., uN ], with the corresponding emotion sequence E = [e1, ..., eN] and speaker sequence P = [p1, ..., pN], every utterance ui is paired with its contextual utterances uj (j ≤ i). If ui is a non-neutral utterance and uj is its causal utterance, the pair (ui, uj) is labeled with 1. Otherwise, (ui, uj) is labeled with 0. In our setting, neutral utterances are involved as targeted utterances, leading to more negative non-causal pairs and making the task more challenging.

3.2 Utterance Encoder

Similar as other conversation-related tasks (e.g., ERC), utterances in a conversation are first encoded by an utterance encoder to form utterance representations. We utilize the pretrained model RoBERTa [Liu et al., 2019] to encode an utterance ui = [w1, ...,wLu ] with Lu words. We then obtain the utterance representation by:

where si ∈ Rdu and du is the dimension of utterance representation.

3.3 Conversation Encoder

As only reviewing conversation history for utterances, the conversation can be modelled as a directed acyclic graph when not considering self-loop edges. Based on the Directed Acylic Graph (DAG) networks [Shen et al., 2021b], we introduce our knowledge enhanced conversation graph, which is illustrated in Fig. 2.

Knowledge Enhanced Conversation Graph

Knowledge Enhanced Conversation graph (KEC) is formulated as G = (V,Ac,Ak). V denotes the set of utterance nodes, Ac denotes the adjacency matrix of conventional utterance interactions, and Ak denotes the adjacency matrix of knowledge passing between utterances.

Algorithm 1 Construction of Knowledge adjacency matrix

Input: Dialog [u1, ..., uN], emotion [e1, ..., eN], speaker [p1, ..., pN], COMET CT, window wk, func split(·), e2s(·).

Output: Knowledge passing adjacency matrix Ak. 1: ∀i, j ∈ [1, ...,N], initializing Ak with Ak[i, j].item = 0, Ak[i, j].klg ="none". (i: target, j: source) 2: for j in [1,N] do 3: posKx, negKx, neuKx = split(CT(uj, xE/xR)) 4: posKo, negKo, neuKo = split(CT(uj, oE/oR)) 5: for i in [j,min(j +wk,N)] do 6: if pi == pj then 7: K = {pos: posKx, neg: negKx, neu: neuKx} 8: else 9: K = {pos: posKo, neg: negKo, neu: neuKo} 10: end if 11: if e2s(ei) ̸= neu then 12: Ak[i, j].item = 1 13: if e2s(ej) == neu then 14: Ak[i, j].klg = K[neu] + [sep] + K[e2s(ei)] 15: else 16: Ak[i, j].klg = K[e2s(ei)] 17: end if 18: end if 19: end for 20: end for 21: return Ak

an utterance ui and one of the relations, COMET takes them as the input and generates knowledge with the beam search of 5. We denote the generation as CT(ui, xE), where CT and xE are the abbreviations of COMET and xEffect respectively.

As COMET does not guarantee to generate emotionrelated knowledge, it is necessary to pick up the knowledge related to the emotion of the targeted utterance. Furthermore, knowledge passing should align with speakers' interactions. Therefore, we propose a knowledge selecting strategy, which is sentiment realized and fits the condition of conversations. The knowledge selection realizes two factors:

Speaker realization. If pi of the target ui equals to pj of the source uj , knowledge of xEffect and xReact is chosen, otherwise knowledge of oEffect and oReact picked.

Utterance nodes. V models all utterances as nodes vi in a conversation. vi contains the attribute rep to store the utterance representation (i.e., vi.rep= si).

Utterance interaction adjacency matrix. Conversational context modeling is crucial to conversation-related tasks, which can enrich targeted utterances with contextual messages passed from other utterances. As only the dialogue history is considered, we construct Ac as a lower triangle matrix. Ac contains two attributes: item and rel, where item stores 0 or 1 to denote the existence of an edge and rel stores the relation type of an edge.

For nodes vi, vj, Ac[i, j].item = 1 is present if there is an edge from vj to vi, otherwise Ac[i, j].item = 0. For a targeted utterance, it has been studied that local contextual information is more important than that in the remote context [Shen et al., 2021b]. Therefore, a context window with size of wc is sliding through conversations. In the window, every contextual utterance sends an edge to the targeted one. Speaker information is another crucial factor. Speaker interactions can be modelled as two relations: self-dependency (SD) and inter-speaker dependency (ID) [Ghosal et al., 2019]. For vi, vj , Ac[i, j].rel = SD means that the speaker identity pi is the same as pj , otherwise Ac[i, j].rel = ID.

Knowledge passing adjacency matrix. We pass the knowledge through edges to enhance the ability of inferring causes. Similar as Ac, Ak is also a lower triangle matrix. Ak contains two attributes: item and klg, where item plays the same role as Ac.item and klg stores the knowledge attached on an edge.

The commonsense knowledge (CSK) we utilize is the social CSK provided by the CSK graph ATOMIC-2020 [Hwang et al., 2021]. Hwang et al., [2021] train a generating model called COMET [Bosselut et al., 2019] on the CSK graph so that texts out of the scope of the CSK graph can be freely inferred. ATMOIC-2020 contains nine social-interaction relations while not all relations can be fit into our condition. According to the direction of inference and relevance, we pick four relations: xEffect, xReact, oEffect, and oReact. x(o)Effect reasons the effect on the speakerself (other speakers) after an utterance, and x(o)React describes how the speaker-self (other speakers) feels after an utterance. We exemplify these relations in Appendix A. Given Sentiment realization. For the generated set of knowledge, we need to first identify its emotion, and then pick up emotion-related pieces for ui and uj . As no standard emotion lexicon resource is provided, we replace the emotion with the sentimental polarity. We utilize sentiwordnet to compute the sentimental polarity of every piece of knowledge. As sentiwordnet rates every word with scores of neutral, negative, and positive, we compute the sentimental score of a piece of knowledge by:

where rpos is the average positive score of all words in the knowledge, and the knowledge is regarded as positiveness when rs > 0. According to the rs, we split knowledge into three sets and concatenate the knowledge in a set with the token [sep] (denoted as a function split(·)). If a set is empty, the text "none" will be assigned to it. After splitting knowledge, we further map the emotion of ui and uj into the sentiment (denoted as a function e2s(·)). The emotion happiness is mapped to positive, neutral to neutral, and other emotions in the dataset to negative. With the mapped sentiment, we can select the related knowledge for utterances.

The knowledge selection is processed simultaneously along the construction of Ak, which is illustrated in Algorithm 1. In line 14 of Algorithm 1, we add the neutral knowledge, because we think that it can bring some reasoning information to help understand causes.

Knowledge Enhanced DAG Networks

As the original DAG network considers no knowledge, we modify it and propose the Knowledge Enhanced DAG network to better process the KEC graph.

We first encode the knowledge into representations by using the shared RoBERTa of the utterance encoder and the same operation of encoding utterance. For an item of knowledge Ak[i, j].klg, it is encoded as ki,j ∈ Rdu .

For a targeted utterance ui, edge weights from neighboring nodes vj in the lth layer can be computed as:

where hl-1 i is the node representation of the (l - 1)th layer, hlj is the neighboring node representation that has been processed in the lth layer, Wlw ∈ R1×2du and Wlk ∈ Rdu×du are trainable parameters. For h0i ∈ Rdu , we initialize it by h0i = Linear([si||eembei ]), where eembei is the trainable embedding of the emotion ei and is initialized by the weight vector in the MLM head of RoBERTa. Furthermore, the contextual message and knowledge message are aggregated from neighboring nodes according to the edge weight αi,j :

where WAc[i,j].rel ∈ (WlSD,WlID) with the size of Rdu×du is a trainable parameter.

To capture the nodal and contextual information, DAG networks utilize two GRUs: nodal unit (GRUn) and contextual unit (GRUc). The information is obtained by:

To further consider the effect of knowledge from the context and the utterance itself, we employ two additional GRUs: contextual knowledge unit (GRUk) and self-loop knowledge unit (GRUs). Therefore, knowledge-related information can be obtained by:

Finally, the node representation of utterance ui in the lth layer is updated by summing the four types of information, i.e. hli = nodi + cxti + ckgi + skgi.

3.4 Cause Predictor

After the encoding of two-level encoders, the final utterance representation is computed by hi = ||Ll=0hli [Shen et al., 2021b]. Whether uj is the cause of ui is then computed by:

where MLP maps the concatenation of hi and hj from the dimension of 2du to 1.

We use the cross entropy loss to train our model, and the loss on a conversation is formulated as:

where yi,j ∈ {0, 1} is the cause label of the pair (ui, uj).

4 Experimental Setup

4.1 Dataset

We conduct experiments on the dataset call RECCON-DD [Poria et al., 2021], which is collected from the popular dyadic dialogue dataset DailyDialog (DD) [Li et al., 2017]. Preprocessing the original RECCON-DD, we only consider causes in the dialogue history for every utterance and remove the duplicated causal pairs. Statistics of the processed RECCON-DD are presented in Tab. 1.

4.2 ComparedMethods

For the reason that C2E2 is a new task proposed by Poria et al. [2021], there is only one baseline [Poria et al., 2021]. Therefore, we additionally compare our model with other SOTA methods from conversation and emotion cause related tasks.

Entail is the baseline of C2E2. It utilizes RoBERTa to process the concatenation of utterances and their emotion and history context. The concatenation is formed by "[cls] emotion [sep] ui [sep] uj [sep] history context [sep]". DAGERC [Shen et al., 2021b] is the SOTA method of Conversational Emotion Recognition (ERC). Our method is based on it. SKAIG [Li et al., 2021] is another SOTA method of ERC which utilize CSK to help the prediction. It passes knowledge through edges using graph transformers [Shi et al., 2021], while no knowledge selecting strategy is applied. For the fair comparison, we utilize the knowledge x/oEffect and x/oReact for SKAIG. KAG [Yan et al., 2021] is the SOTA method of Emotion Cause Extraction (ECE) using CSK to overcome the position bias. It connects emotional utterances with a contextual utterance if a path is matched in ConceptNet. It uses relational GCNs to process their proposed graph. Knowledge used in KAG only participates in the computation of edge weights. As other methods add no position embeddings, we do not add position embeddings in KAG. Adapted [Turcan et al., 2021] is another SOTA method of ECE. It directly concatenates CSK (xReact and oReact) after the utterance and utilizes RoBERTa to make prediction. For baselines from Emotion Cause Pair Extraction (ECPE), following and referring the results from Poria et al. [2021], we choose ECPE-2D [Ding et al., 2020a], ECPE-MLL [Ding et al., 2020b], and RankCP [Wei et al., 2020]. To consider emotion information, in the re-implementation, all graph-related baselines utilize emotion embedding.

4.3 Implementation

All compared methods and our model use base-size pretrained models (e.g., RoBERTa-base) from huggingface Transformers. We train our model using the AdamW optimizer with 40 epochs, learning rate of 3e-5, L2 regularization of 1e-4. The batch is set to 4 and we step the gradient with the accumulating step of 2. For Conversation Encoder, we set the dimension of utterance to 300, the hidden size of GRUs to 300, the number of layers of DAG networks to 5. For emotion embeddings, we utilize a linear unit mapping the dimension of 768 to 200. For Cause Predictor, dimensions of MLP is set to [600, 300, 300, 1] and the dropout rate is set to 0.1. For the window size of knowledge and that of context, we find that our model achieves better performance when they are the same, and we set them to 2. Following Poria et al. [2021], we report the F1 scores of both negative and positive causal pairs and the macro F1 score of them. Our experiments are conducted using 5 random seeds. 1

5 Results and Discussions

5.1 Main Results

The results of our method and baselines are listed in Tab. 2. We split the table into 4 rows: the 1st row for models of ECPE, the 2nd row for models using CSK, the 3rd row for models with no CSK, and the 4th row for our method.

From the 1st row, models of ECPE are not competitive to models in other rows. We think the reason is that the setting of ECPE is not suitable for C2E2. Compared with models in the 2nd row, our method can achieve better results. For KAG, we think the undesired performance can be attributed to that it is designed to resist position bias for news articles. Conversations in RECCON tends to be short and requires more sophisticated modeling than articles. In addition, knowledge is only used to compute edge weight in KAG, which does not propagate through edges like ours. For "Adapted", knowledge is directly concatenated after the utterance, and is not explicitly passed between utterances. For SKAIG, knowledge is also not selected, which leads to sentiment-nonaligned knowledge passed through the graph. On the contrary, our KEC selects proper knowledge to align with emotions, and explicitly models and propagates the knowledge in graphs. The comparison indicates that our method is more suitable for utilizing knowledge in C2E2.

As the baseline of RECCON, "Entail" is not comparable to the graph-based SKAIG and DAG-ERC, which demonstrates that explicit modeling of utterance interactions is critical for C2E2. DAG-ERC is the SOTA baseline for ERC with no knowledge involved. By introducing CSK and the knowledge selection, our method can achieve significant improvements against DAG-ERC. Therefore, it is critical to utilize proper knowledge for cause detection. To verify our knowledge enhanced DAG networks, we construct a variant that knowledge Wlkkj,i is directly added on hlj in eq. 4 and no knowledge-related GRUs are involved. The macro F1 and Pos. F1 achieved by the variant are 80.38 and 65.24. This demonstrates that, instead of being simply added in contextual messages, knowledge prefers to be exclusively modelled by gated units in our knowledge enhanced DAG networks.

1 The code and Appendix are available at https://github.com/ LeqsNaN/KEC

5.2 Ablation Study

To study the effect of different modules equipped in our model, we remove contextual knowledge unit GRUk, selfloop knowledge unit GRUs, neutral knowledge added when source utterances are neutral in line 14 of Algorithm 1, and emotion embedding respectively. The results are in Tab. 3.

The decreased performance by removing GRUk shows the necessity of controlling knowledge from the context. As for removing GRUs affects the performance, it indicates that self-knowledge enriching for targeted utterances is critical because 37% causal utterances are the targeted utterances themselves in RECCON. By taking off neutral knowledge, the performance shows decline. This manifests that neutral knowledge can help the model to better detect causes as it can provide some reasoning information.

When removing emotion embedding, the performance is still competitive to DAG-ERC who is equipped with emotion embedding. This can be attributed to that our knowledge selecting strategy can pick up sentiment related knowledge for the model. As emotion embedding and knowledge are all removed, the model becomes DAG-ERC without emotion information. The performance drops dramatically, which verify the claim that emotion information are important to C2E2. As for "Entail" without emotion, the performance is presented in Appendix C. In the next subsection, we will further show what causes are preferred by the emotion information.

5.3 Performance on Cause Pairs

There is a question about whether our model can promote the detection of causal utterances with emotions different from that of the targeted utterance. We therefore present the recalls of causal pairs whose utterances are with the same emotion (SE pairs) and different emotions (DE pairs) in Fig. 3 for every targeted emotion. From the figure, we can see that DAGERC with emotion embedding can dramatically boost the detection of SE pairs while limited improvement is achieved on DE pairs. Specifically, emotion information increases the recall on all SE pairs by about 23% while only about 3% on DE pairs. By introducing CSK, our model, compared with DAG-ERC with emotion embedding, further increases the recall by 3% on SE pairs and by 12% on DE pairs (especially boosting the detection of neutral causal utterances by 12%). This manifests that our model can promote the detection of DE pairs, especially on pairs whose targeted utterances are with negative emotions like Anger, Sadness in Fig. 3.

5.4 Effect of Window Size

To show the effect of the window size, we illustrate the trend of macro F1 scores with the window size increasing in Fig. 4. The trend line shows that our model achieves the best performance with a small window. As the window size keeps increasing, the performance will drop and fluctuate. The reason for this can be attributed to that over 80% causal utterances are located near the targeted utterance in no more than 2 steps. It can be future work to solve the problem that models bias to neighboring utterances in C2E2.

6 Conclusion

In this paper, we study Conversational Causal Emotion Entailment. We find that adding emotion information into graphs cannot effectively detect causal utterances with different emotions from the targeted utterance due to models' limited causal reasoning ability. To alleviate this, we introduce social commonsense knowledge and propose a graph-based structure to properly utilize knowledge. Specifically, we propose a Knowledge Enhanced Conversation graph (KEC) with a designed knowledge selecting strategy. To process KEC, we construct Knowledge Enhanced Directed Acyclic Graph networks. Our method outperforms baselines and detects more causes with different emotions from the targeted utterance.

Acknowledgments

This work was supported by National Natural Science Foundation of China (No. 61976207, No. 61906187).