The Interplay of Emotions and Norms in Multiagent Systems

1 Introduction

Agents interact with each other to make informative decisions. An agent's emotions can be influenced by whether its goals are achieved and whether norms are satisfied by other agents. Consider two agents, Alice and Bob. Suppose Alice has a goal to complete a task that she cannot complete by herself. She requests Bob to complete the task. Bob agrees, meaning he creates a commitment norm toward Alice to perform that task. Now consider two possibilities.

Example 1 Bob completes the task and, thus, satisfies his commitment toward Alice. The satisfaction of the commitment leads Alice to achieve her goals. Alice is now happy, her trust for Bob increases, and she may display positive emotions toward Bob. Bob, on receiving Alice's positive feedback, may feel encouraged to interact again with her. And, Alice may as well-possibly commit to Bob for something else.

Example 2 Bob fails to complete the task he committed to perform, thereby violating his commitment to Alice. Alice might be unhappy since she fails to achieve her goals. If she blames Bob, she may lose trust in Bob. And, each of them may be disinclined to commit to the other in the future.

Examples 1 and 2 suggest that Alice's decision to commit to Bob depends on the outcomes of her prior interactions with him, including how her goals turned out, her appraisal of the situation (e.g., blame Bob?), and her resulting emotions.

Works on norm recommendation are geared to norm emergence [Brooks et al., 2011; Mahmoud et al., 2016; Ajmeri et al., 2018] based on prior outcomes of norms, including sanctions such as rewards or punishments. But, reallife sanctions are more subtle, including change of trust or emotions that might influence an agent's actions [Nardin et al., 2016]. Thus, it is important to consider norm outcomes with respect to emotions, trust, and goals. Existing works capture relationships among these variables: emotions and trust [Dunn and Schweitzer, 2005; Antos et al., 2011; Paradeda et al., 2017], trust and commitments [Kalia et al., 2014], emotions and goals [Guiraud et al., 2011; Lall´e et al., 2018], and goals and commitments [Telang et al., 2019]. However, they lack a holistic view of the relationships.

Developing a suitable holistic model is nontrivial. To make the problem tractable, one, we limit our scope to commitments as a type of norm. Two, we simplify our treatment of emotions to reduce the complexity of our study design. Specifically, we adopt a simple form of appraisal theory [Arnold, 1960; Lazarus, 1966] assuming that agents process their emotions discretely (not continuously) by appraising the state changes of their goals and norms. Three, we adopt a simple form of Dimensional Theory [Russell, 1980] and of the Ortony, Clore, & Collins (OCC) model of emotions [Ortony et al., 1988], in which we consider valence for stating emotions but not arousal or intensity.

Contributions. We propose a novel empirical analysis of emotions and norms based on an extension of the well-known Colored Trails [Gal et al., 2010]. We map aspects of this game to variables such as norms, emotions, trust, and goals. We use the data we collect from the game play to generate Dynamic Bayesian models in a step-wise process to capture relationships between these variables. We evaluate these relationships via log-likelihood scores with respect to the data to understand whether these relationships hold. Further, for each target variable, we identify the relationship that yields the highest prediction accuracy.

2 RelatedWork

We now describe the key related work. From virtual agents, De Melo et al. [2012] provide a model that considers the interpersonal effect of emotion in decision-making. Antos et al. [2011] and Paradeda et al. [2017] endorse the importance of emotions in trust. Hoegan et al. [2017] show how emotions influence social decisions. S´ebastien et al. [2018] provide evidence that a student's emotions are modulated by the student's achievement goals. Such contributions focus on decision-making and do not formalize their outcomes in terms of norm or goal satisfaction. The lack of formalization makes it difficult to enhance a model with additional dependent variables.

In terms of multiagent systems, Dastani and Lorini [2012] associate emotions with goals. Steunebrink et al. [2007] formalize emotions in terms of agents' beliefs, goals, abilities, plans, intentions, and commitments. Lorini and Schwarzentruber [2009] capture emotions as the difference between the outcomes of current choices made and choices that could have been made. Existing formal models represent only a portion of the relationships. Importantly, these approaches neither provide nor evaluate computational models for predicting norm outcomes. Kalia et al.'s [2014] is another multiagent approach. They provide some experimental evaluations, but their model is limited to trust and commitments.

In the area of norm emergence, Brooks et al. [2011] capture how agents learn from their own experiences to converge to a behavior that becomes a norm in their society. Mahmoud et al. [2016] describe how punishment by peers leads to norm emergence. Nardin et al. [2016] suggest self-directed sanctions (e.g., guilt and trust) and other-directed sanctions (e.g., gossip and praise). Ajmeri et al. [2018] develop personal agents that infer contextually relevant norms on observing norm deviations and understanding the social context related to the deviation and applied sanctions. In psychology, Dunn and Schweitzer [2005] describe the influence of emotional states on trust. Forgas [1995] proposes how a human's emotions influence his or her judgments.

Overall, the above models do not provide a holistic picture of relationships between norms, goals, emotions, and trust.

3 Conceptual Framework

We treat commitments, goals, trust, and emotions as four discrete random variables in our Bayesian model.

Commitment CA,B. A commitment CA,B means that a debtor A commits to a creditor B to bring about a consequent provided an antecedent holds. A commitment provides grounds for B to expect some actions from A [Singh, 1999]. The outcome of a commitment can be represented as: satisfied (sat) when the consequent holds regardless of whether the antecedent does; or violated (vio) when the antecedent holds but the consequent fails to hold.

Goal GA. A goal is a condition that an agent wants to achieve and may motivate the agent to act, but is not directly visible to others. The outcome of a goal GA has a binary value, achieved (ach) or failed (fai).

Trust TA,B. A's trust in B refers to A's expectation of B to perform a specific task [Castelfranchi and Falcone, 1998; Singh, 2011]. The variable TA,B has three values: low, medium, or high.

Emotion EA. We model an agent's emotion as its response to an external or internal event [Friedenberg and Silverman, 2012; Smith and Ellsworth, 1985]. An emotion as EA has three values: negative (neg), neutral (neu), or positive (pos).

4 Baseline Models

We define baseline models and relationships within these models based on existing contributions. Suppose agents Alice (A) and Bob (B) interact with each other. Nodes in the following models are defined for B (A is analogous).

Figure 1 shows models for commitment outcomes. M1 represents B's current expectations about A satisfying its commitments toward B (C t A,B ) and M2 represents B's current decisions to satisfy its commitments toward A (C t B,A ).

We describe the relationships expressed in these baseline models and how they are justified in the literature. We identify the antecedents and consequents (of commitments and trust) and the conditions (of goals and emotions) as propositional variable, p and q, for relationships where the logical form is directly relevant. We leave ontologies as a basis for abstracting concepts outside our scope.

• T t-1 B,A (p, q) → C t A,B (p, q) represents that B's trust in A in the previous instant influences B's current expectation of Ai.e., the logical form matters. Indeed, paraphrasing Mayer et al. [1995]: this relationship indicates B's willingness to be vulnerable to the actions of A based on the expectation that A will perform the consequent irrespective of B's ability to monitor or control A.
• C t-1 B,A → C t A,B represents that B's past decision on the outcomes of commitments toward A influences B's current expectation on the outcomes of commitments from A. That is, B expects something in return; the two commitments

are logically unrelated. This relationship illustrates reciprocity [Hazard and Singh, 2013]. Hence, C t-1 A,B → C t B,A , which represents B's past expectations of the outcomes of commitments from A influences B's current decision on the outcomes of its commitments toward A.

• T t-1 B,A → C t B,A represents that B's past trust in A influences B's current decision on the outcomes of its commitments toward A. This relationship does not rely upon the logical form. It follows from two relationships: (1) the past trust of B toward A influences B's current expectation on the outcomes of A's commitments and (2) B's past expectations on the outcomes of A's commitments influence B's current decision on the outcomes of B's commitments toward A.
• E t-1 B → T t B,A represents that B's past emotions influence B's current trust in A [Dunn and Schweitzer, 2005; Antos et al., 2011; Paradeda et al., 2017]. This relationship does not rely upon the logical form of the variables.
• E t-1 B → C t B,A represents that B's past emotions influence outcomes of B's current commitments toward A. We include this relationship in the baseline since B's emotions influence its trust in A [Dunn and Schweitzer, 2005; Antos et al., 2011; Paradeda et al., 2017] and B's trust in A influences B's decisions to satisfy its commitments (positive outcomes) toward A. This relationship does not rely upon the logical form of the variables.
• C t-1 A,B (p, q) → T t B,A (p, q) represents that outcomes of A's past commitments toward B influence B's current trust in A. We include this relationship in the baseline based on prior approaches [Kalia et al., 2014; Singh, 2011].
• G t-1 B (q) → E t B (q) represents that past outcomes of B's goals influence B's current emotions. We include this relationship since Guiraud et al. [2011] and Dastani and Lorini [2012] propose that when B achieves its goals, B's emotions become positive (e.g., joy) and vice versa.
• G t-1 B (q)→T t B,A (p, q) represents that past outcomes of B's goals influence B's current trust in A. We include this relationship in the baseline assuming that the past outcomes of B's goals influence B's current emotions and B's past emotions influence B's current trust in A.
• C t-1 A,B (p, q) → E t B (q) represents that the outcomes of A's past commitments toward B influence B's current emotions. We include this relationship since emotions can be responses to the change of state of commitments [Friedenberg and Silverman, 2012; Smith and Ellsworth, 1985].
• C t-1 A,B (p, q) → G t B (q) represents that the outcomes of A's commitments toward B influence outcomes of B's current goals. We include this relationship in the baseline based on scenarios where when A satisfies its commitments toward B, B achieves its goals [Telang et al., 2019].

We compute the joint distribution for model M1 as P (CA,B, CB,A, TB,A, EB, GB) = P (CA,B | TB,A, CB,A) P (TB,A | EB, CA,B, GB) P(GB | CA,B) P(EB | GB, CA,B) and for model M2 as P (CB,A, CA,B, TB,A, EB, GB) = P (CB,A | TB,A, CA,B, EB) P (TB,A | EB, CA,B, GB) P(GB | CA,B) P(EB | GB, CA,B).

From the joint distributions for M1 and M2, we infer the following conditional dependencies indicating above relationships for the evaluation: (1) P (C t A,B | T t-1 B,A , C t-1 B,A ), (2) P

Evaluation criteria. We evaluate the relationships based on three criteria (1) log-likelihood (LL), (2) Akaike Information Criterion (AIC), and (3) Bayesian Information Criterion (BIC) scores, which capture the goodness of fit of models to the data. Once we obtain a model, we evaluate its accuracy by computing the area under receiver operating characteristic (ROC) curve (AUC). The curve plots sensitivity versus specificity for a classifier. We obtain an AUC score for each model by averaging three AUC scores obtained by performing threefold cross-validation respectively on the data for each model. When the data is inadequate for three folds, we consider the AUC score obtained from fewer folds. When we cannot calculate the AUC score at all, we select models based on LL, AIC, and BIC scores. We perform the one-tailed t-test on the scores obtained at the 5% significance level.

5 Empirical Study Design

To empirically ground our work, we develop a variant (Figure 2) of [Gal et al., 2010]'s Colored Trails game. Our variant provides a chat interface through which subjects negotiate and exchange tiles and express emotions toward opponents. We associate our variables to the data collected from game play, chats, and surveys provided by subjects. The description below assumes two players, Alice (A) and Bob (B).

Game rules. (1) A subject plays three games with different opponents. (2) Each game consists of five rounds. (3) Each round has a common goal position, and different starting positions for each subject. (4) In each round, subjects are allocated the same number but a different set (randomly selected) of colored tiles. (5) Subjects can communicate with their opponents via a chat interface, in which they can negotiate to transfer tiles to each other. (6) At the beginning of a game and at the end of each round, each subject fills a survey. (7) Let n be the number of tiles left unused and u be the number of tiles left used. The score for a subject who (a) does not reach his or her goal is n+1.5×u and (b) reaches his or her goal is n+3.0×u.

Mapping concepts with model. We describe the mapping between the game and concepts for B. The mapping for A is the same. (1) Goal. B's goal is to reach the goal position. (2) Commitment. During the game, A can create a commitment by agreeing (through chat) to transfer the specified tiles to B, usually in exchange for some tiles. If A provides the tiles, A satisfies his or her commitment toward B, otherwise A violates it. (4) Emotion. B's emotion changes as the games progress and may spill over from one game to the next. We determine B's emotion from B's survey response. (5) Trust. We determine B's trust in A from B's survey response.

Subjects. We recruited 30 (25 male; 5 female) subjects, all students in Computer Science. We offered a payment of 1020 USD each, depending on success in the game. We obtained IRB approval, and informed consent from subjects.

Surveys. (1) At the start of each game and at the end of each round, we asked a subject to record his or her trust for an opponent and an emotion on a five-point scale (very negative, negative, neutral, positive, very positive). We mapped the responses to our trust (TA,B) and emotion (EA) variables, respectively. For analysis, we converted the above five-point scale into a three-point scale by merging very negative and negative responses and very positive and positive responses.

Threats to validity and mitigation. We discuss some important threats to validity to our empirical study. First, the subjects could know each other from before. Thus, we anonymize and separate them to prevent them from knowing who their opponents are or sending any visual or auditory signals. Second, a subject could be an expert in the domain. We select subjects who do not have prior experience in our topic. Third, a subject could produce a strong negative emotional response from the outcomes of a few rounds and might decide to leave the game. We limit their emotional response by setting the game with lower stakes for success or failure.

6 Dataset and Processing

We collected 450 rows of data (30 subjects × 30 games × 5 rounds per game), including their survey forms, and whatever chat messages they exchanged. From subjects' interactions, we manually analyzed messages exchanged by the subjects to identify commitments (CB,A) and (CA,B) and their outcomes. For each commitment, we add preceding emotions (EA), trust (TA,B), and goals (GA). Table 1 shows the data distribution for each variable. Note that the count of commitments satisfied or violated is much smaller than the number of rounds played, indicating that in several rounds, either the players could achieve their goals without cooperation or that their negotiations failed.

Random variables. We represent the models in terms of a Dynamic Bayesian Network (DBN) with two time slices, t-1 and t. To do so, we mapped each round in a game to a time slice. For example, we mapped time slice t - 1 to Round 1 and the next time slice t to Round 2. In each round, for each player, we observed five discrete random variables. As shown in Figure 3, first, B records its trust in A, TB,A and emotion, EB. Then, in the game, in each round, the following variables were observed for B: outcomes of A's commitments toward B (CA,B), outcomes of B's commitments toward A (CB,A), and outcomes of B's goals (GB).

Correlations. Before we created a DBN, we determined Pearson's correlation coefficients R between the random variables observed during the game. Table 2 shows the correlations between variables observed in time slice t whereas Table 3 shows the correlations between variables observed in time slice t -1 and the variables observed in time slice t. We create these tables to observe if random variables have static or dynamic (causal) relationships.

Observations. From the correlation coefficients, we obtain the overall relationships between random variables. Based on the prior work [Dunn and Schweitzer, 2005; Steunebrink et al., 2007; Guiraud et al., 2011], we only consider positive correlations between the variables. For example, Dunn and Schweitzer [2005] observe that negative emotions decrease trust and vice versa. Steunebrink et al. [2007] suggest goal achievement brings positive emotions such as joy. In Tables 2 and 3, positive correlations vary from 0.01 to 0.6. We consider correlations where R is greater than or equal to 0.1. An R of 0.1 may indicate a weak correlation. However, such correlations are common in psychology [Chiaburu et al., 2011]. Based on the observed correlations, we incrementally created Bayesian models for emotions, goals, trust, and commitments by adding one variable at a time as input to a target variable (T 2 B,A , E 2 B , C 2 A,B , C 2 B,A , G 2 B ). For each such incremental model, we calculated the LL, AIC, and BIC scores-and show them in Figure 4.

7 Producing RefinedModels

We evaluate the existing relationships in the baseline models (M1 and M2 shown in Figure 1) using the log-likelihood (LL, AIC, and BIC) scores. In this process, we discover new relationships that were not present in the baseline models. We include only the relationships that produce significant changes and derive the new models M ′1 and M ′2 as shown in Figure 5.

Rel1. P (C t A,B | T t-1 B,A , C t-1 B,A ) represents a player's expectations about its opponent satisfying a commitment toward the player. From the LL, AIC, and BIC scores, we found that when T t-1 B,A , C t-1 A,B , and C t-1 B,A were incorporated in the model, the changes in scores were significant with p-values below 0.01 in each case. This suggests that the existing relationship hold. In addition, we learned a new relationship C t-1 A,B→C t A,B from the data. We also computed the AUC scores for C t A,B | T t-1 B,A (0.69), C t A,B | C t-1 B,A (0.68), and C t A,B | C t-1 A,B (0.71) suggesting the past satisfaction of expectations has the highest influence on C t A,B .

Rel2. P (C t B,A | T t-1 B,A , C t-1 A,B , E t-1 B ) represents a player's decision to satisfy its commitments. From the LL, AIC, and BIC scores, we found that when T t-1 B,A , C t-1 A,B , C t-1 B,A , and T t B,A were incorporated into the model, the changes in scores, shown in Figure 4(d), were significant (with p-values of 0.00, 0.00, 0.01, and 0.00, respectively). This suggests the relationships T t-1 B,A→C t B,A and C t-1 A,B→C t B,A hold whereas E t-1 B →C t B,A does not hold. In addition, we learned two new relationships: C t-1 B,A→C t B,A and T t B,A→C t B,A . We compare the AUC scores for C t B,A | T t-1 B,A (0.56), C t B,A | C t-1 A,B (0.71), C t B,A | C t-1 B,A (0.68), and C t B,A | T t B,A (0.71). This comparison suggests C t-1 A,B and T t B,A have the highest influence on C t B,A .

Rel3. P(G t B | C t-1 A,B ) represents the goals of a player. From the LL, AIC, BIC scores for each such model, we found that when C t A,B was added to the model, the change in the scores for C t A,B was significant (with a p-value of 0.01). This suggests that the existing relationship does not hold indicating the outcomes of past commitments do no influence the current outcome of a goal. The new relationship we obtain is C t A,B→G t B and its AUC score for G t B | G t B is 0.56.

Rel4. P(E t B | G t-1 B , C t-1 A,B ) represents the emotions of a player. From the LL, AIC, and BIC scores, we found that when trust T t-1 B,A , emotions E t-1 B , commitments C t-1 A,B , and goals G t-1 B were incorporated in the model, the changes in the scores were significant with p-values of 0.00, 0.00, 0.03, and 0.02, respectively). This means that the baseline relationships G t-1 B →E t B and C t-1 A,B→E t B hold. In addition, we learned two new relationships E t-1 B →E t B and T t-1 B,A→E t B . We obtain the AUC scores for E t B | G t-1 B (0.69), E t B | C t-1 A,B (0.5), E t B | E t-1 B (0.56), and E t B | T 1 B,A (0.59) to find that G t-1 B has the highest influence on E t B followed by E t-1 B and T t-1 B,A .

Rel5. P (T t B,A | E t-1 B , C t-1 A,B , G t-1 B ) represents trust of a player for another player. From the LL, AIC, and BIC scores, we found that when T t B,A , E t-1 B , and C t-1 A,B were incorporated in the model, the changes in the scores were significant with p-values of 0.00, 0.00, and 0.03, respectively. This suggests that the relationships C t-1 A,B→T t B,A and E t-1 B →T t B,A hold and G t-1 B →T t B,A does not hold. In addition, we learned a new relationship T t-1 B,A→T t B,A . We obtain the AUC scores for T t B,A | T t-1 B,A (0.74), T t B,A | E t-1 B (0.63), and T t B,A | C t-1 A,B (0.47) to find that T t-1 B,A has the highest influence on T t B,A .

Table 4 compares the relationships in the baseline models and the new models. Figure 5 shows the new models, M′1 and M′2 , produced by combining the new relationships

-3:

-2:

-1:

-3:

-2:

-1:

for the respective target variables, and highlights the relationships that yield the highest AUC scores of any relationship with the same conclusion.

8 Discussion and Conclusion

We initially construct two baseline models, M1 and M2, based on prior work that capture the outcome of commitments based on emotions along with trust and goals. We evaluate these models using the LL, AIC, BIC, and AUC scores based on data from human subjects. Accordingly, we construct two new models, M′1 and M′2 , by adding and removing relationships based on their empirical backing. We identify the following relationships as among those most strongly supported.

(1) The outcomes of an agent's past commitment to a second agent influence the second agent's current expectations of the outcomes of a commitment from the first. (2) The outcomes of an agent's past commitment to another agent influence the second agent's current decision on the outcome of its commitments to the first. (3) An agent's past trust in a second agent influences the agent's current decision on the outcome of its commitments to the second. (4) The outcomes of an agent's past goals strongly influence its current emotions. (5) An agent's past trust in another agent strongly influences its current trust in the second agent.

Real-world applications. Empirically grounded Bayesian models promise to support real-world applications, such as the following. (1) We can compute trust between teammembers based on their commitments, trust, and emotions, to recommend optimal team configurations [Kalia et al., 2017; Liemhetcharat and Veloso, 2012]. (2) We can enhance Mayer et al.'s [1995] Trust Antecedent Framework to incorporate emotions to recommend the amount of risk team leaders can take while assigning important tasks to team members. (3) We can use the dependency of emotions on trust to filter out dishonest advisors more accurately than by considering trust alone [Irissappane and Zhang, 2017]. We can leverage this dependency to build an agent that discovers valuable messages via trustworthy peers in a social network [Sardana et al., 2017]. (4) We can build help-desk cognitive assistants [Telang et al., 2018] who seek to solve customers' problems. An assistant would benefit from interpreting customers' emotions and trust to engage effectively with customers, including by prioritizing between problems that a customer brings up.

Future work. We plan to evaluate our models on largescale communication data to gauge their effectiveness and generality. Doing so would involve sophisticated text analysis to extract norms, trust, goals, and emotions. We can enhance the models proposed in this paper to incorporate intensity of emotions and trust. We can include additional emotions such as hope, joy, and sorrow to bring forth more interesting insights into the existing models for predicting norm outcomes.

Acknowledgments

We thank the anonymous reviewers for their helpful comments. This research was partially supported by the US DoD through the Science of Security Lablet (SoSL) at NCSU, a US ARL ORISE Fellowship, and by IBM Research.