NEGOTIATOR: A Comprehensive Framework for Human-Agent Negotiation Integrating Preferences, Interaction, and Emotion

1 Introduction

Agent-based agreement technologies have become effective tools for resolving conflicts among diverse stakeholders [Fatima et al., 2014; Marsa-Maestre et al., 2014]. While offering a versatile environment for researchers to design and assess negotiation mechanisms such as protocols and strategies, it is crucial to design and implement the software architecture in a modular fashion for seamless integration of new components while still providing a user-friendly interface. In this aspect, GENIUS (General Environment for Negotiation with Intelligent multi-purpose Usage Simulation) [Hindriks et al., 2009] and NegMAS (Negotiation MultiAgent System) [Mohammad et al., 2020] are leading automated negotiation platforms utilized in the Automated Negotiating Agents Competition(ANAC) [Jonker et al., 2017] to streamline the research in agent-based negotiation. Another simulation environment called NegoSim [Ebrahimnezhad and Fujita, 2023] extends the traditional bidding, opponent model, and acceptance strategy components with preference elicitation to facilitate the development of customized agents and protocols. All these platforms mainly focus on automated negotiations where intelligent agents negotiate on their users' behavior.

In recent years, there has been a growing demand for AI systems capable of interacting with humans, particularly in human-agent negotiation systems. Ficici et al. introduced Colored Trails (CT), a situated multiagent game environment designed to accommodate players who can be humans, agents, or a combination of both [Ficici et al., 2008]. CT offers customizability and extensibility, making it a valuable tool for the research community investigating multiagent decision-making processes. Avi et al. proposed the "NegoChat" framework, enabling negotiation between a negotiating agent and a human negotiator through bid exchanges in English [Rosenfeld et al., 2014]. Interactive Arbitration Guide Online (IAGO) introduces a Web-based platform for human-agent negotiation, which also allows exchanging arguments in natural language (e.g., preference statements) and emotional states via emojis [Mell and Gratch, 2016]. The platform enables the agent developers to change the negotiation scenarios but mainly focuses on resource allocation problems. Recognizing the significance of communication mediums in human interactions, Divekar et al. introduced a framework where a virtual agent communicates with human negotiators through speech, further enhancing the negotiation experience [Divekar et al., 2019]. All of those platforms are designed for either particular problems or settings.

While human-agent negotiation, particularly in humanrobot contexts, presents intriguing research challenges, prospective researchers are deterred by the initial setup overhead. Existing platforms mentioned earlier can be valuable if their research questions align with those platforms' capabilities. However, pursuing diverse research directions and specific needs often necessitates developing a tailored human-agent platform, which can be time-consuming. Thus, there is an urgent need for a versatile human-agent negotiation platform that supports various research directions, such as emotional awareness [Keskin et al., 2021], natural language usage [Chawla et al., 2023; Lewis et al., 2017], embodiment/appearance impact [C¸ akan et al., 2023; Keskin et al., 2024], and non-verbal communication such as facial expressions and gestures [Aydo˘gan et al., 2022].

Some of these directions require functionalities like speech recognition, language processing, video processing, and emotion/gesture recognition. Addressing this gap, our framework offers a comprehensive and modular approach to developing general-purpose human-agent negotiation sessions. Researchers can leverage this tool to craft innovative negotiation/interaction strategies and evaluate them through user experiments. This platform aims to lower the barrier to entry for newcomers and streamline the development process for seasoned researchers in human-agent negotiation. This framework particularly explores how physical embodiment and behavior-based negotiation strategies, augmented by robotic gestures, can significantly impact human responses and perceptions during negotiation processes.

2 NEGOTIATOR: Proposed Framework

There is a growing trend for human-agent interaction research, where a user should be able to engage with a human user effectively. A human-agent interaction generally should be able to accommodate the following abilities: (i) receiving verbal/nonverbal input from the user, (ii) processing verbal/nonverbal messages, (iii) reasoning on the given input in the underlying state, (iv) generating a response, and (v) communicating with the user. Our framework aims to build the necessary components and establish their interaction smoothly so researchers do not waste time developing their framework from scratch.

2.1 Framework Components

Figure 1 illustrates the components of NEGOTIATOR 12 , where each box, except for the core and the configuration manager, can be theoretically unplugged or swapped easily as long as the related messages are provided to the NegoCore, which is responsible for the orchestration of the messages exchanges among components. Along with NegoCore, Config Manager is the core system component that decides the respective configuration of the negotiation (e.g., the domain information, the deadline duration) and which components should be running. We focus on bilateral negotiations where a human and an agent negotiate in a range of joint decision-making scenarios spanning from resource allocation (i.e., zero-sum games) to multi-issue negotiation scenarios (i.e., nonzero-sum games). The core first starts with health checks of each module to ensure they have started without issue, then proceeds to send the underlying negotiation information to the relevant modules. It coordinates the message exchanges between respective modules by following the interaction rules governed by the negotiation protocol [Aydo˘gan et al., 2017].

1 NEGOTIATOR GitHub Repo: github.com/HumanRobotNego

2 You can access the demonstration video here.

• Speech-to-Text: Through this module, the system utilizes speech recognition, transforming audible inputs received from human negotiators into textual inputs
• Text-to-Offer: By parsing the textual data and/or employing NLP techniques, this module extracts a structured offer content.
• Emotional Modelling: From facial expressions or given textual data, this component recognizes the human negotiator's emotional state and passes the recognized emotion to the Negotiation Core. Such facial emotion modeling is comprised of categorical [Li and Deng, 2019], dimensional [Barros et al., 2020], and continual models [Churamani and Gunes, 2020].
• Agent: The agent component consists of three subcomponents: Offering Strategy, Opponent Model, and Acceptance Strategy. The main agent component receives the offer, emotional feedback, and argument inputs from the Core and sends the relevant information to the subcomponents. Here, the offering strategy generates the coming offer, whereas the acceptance strategy determines whether or not to accept the human negotiator's offer. Opponent modeling is an optional component, which might maintain a model of the human negotiator's preferences or strategy. The agent transfers the agent's underlying emotional state, Mood, as well as the offer content. Note that it could also send an acceptance.
• Interaction Manager: It acquires the agent's current offer and mood to express. For instance, if the agent wishes to express the "anger" emotion, the corresponding mood of "anger" is sent to the interaction manager,

and the respective embodied response or textual argument is given to the human negotiator. This component currently consists of the following components: Gesture Generator in the case of an embodied agent, Argument Generator, and Text to Speech in the case of speech-based communication is desired.

• GUI: This is an optional component where the human negotiator interacts with the agent via a user interface. All inputs from the agent, such as offer sentences, remaining time, and so on, are passed to this component.

Our open-source framework provides components specified in Table 1. The researchers can use speech/text/video to communicate with the human negotiators. For instance, a humanoid robot with a camera and microphone can utilize video to receive human inputs during the negotiation. There are three negotiation strategies: emotion-aware, a mix of time and behavior strategy [Keskin et al., 2021], and behaviorbased strategy. They can make their agent negotiate over multiple issues or a resource allocation.

2.2 SystemDesign

We build the system as a set of modules running Docker containers and other parallelized architectures to ensure that interaction between these modules remains as independent processes, giving the agent designers full autonomy in system requirements (e.g., the user can program their agent programming language independently as long as the messages are sent accordingly to core requests). An independent message queue provider (RabbitMQ) and respective message queues are adopted for each module to communicate within the system. We define a message as M<queue, sender, context, body, status> where the body comprises module-related input and outputs according to the direction.

2.3 Use Cases and Applications

The NEGOTIATOR has been utilized in several research projects to date. For instance, to explore the impact of gestures, we conducted experiments where human negotiators interacted with the humanoid Nao robot, both with and without gestures [Aydo˘gan et al., 2022]. Subsequently, our research team devised an emotion-aware negotiation strategy, necessitating emotion recognition through live video processing. This strategy was integrated into the framework to investigate the role of emotions in negotiation dynamics [Keskin et al., 2021]. Additionally, we sought to examine the influence of physical embodiment in negotiation scenarios [C¸ akan et al., 2023]. We required a virtual agent with a negotiation setting identical to our Nao robot to achieve this. The interoperability feature of the NEGOTIATOR significantly shortened development time, enabling us to leverage common models across both settings. Furthermore, we collaborated with another university to study the effect of robots' appearance (e.g., NAO, QT, Pepper) in negotiation [Keskin et al., 2024]. Notably, the straightforward setup process of both experimental configurations was facilitated by the user-friendly interface of the NEGOTIATOR, as shown in Figure 2(b). Apart from the robot interface, the framework could be used for chatbased negotiation where the human interacts with the agent via the Web interface, as shown in Figure 2(a). Its versatile design allows adaptation to meet various research demands, thereby establishing The NEGOTIATOR as a valuable asset for human-agent negotiation research.

3 Conclusion and FutureWork

The insights from this research highlight the potential for developing a more sophisticated, socially aware agent capable of engaging in negotiations with humans more naturally and effectively. This work further expands the horizon of negotiation frameworks, illustrating the importance of incorporating social and psychological dimensions into collective decisionmaking, thereby enriching the dialogue on human-agent collaboration. Incorporating these insights, the NEGOTIATOR aims to synthesize advancements from diverse negotiation frameworks, including dynamic scenario modeling, negotiation strategies, emotional intelligence, non-verbal/verbal communication, and human-robot interaction dynamics. This holistic approach ensures that the NEGOTIATOR provides automated negotiations' technical and strategic demands and embraces the complexities of human-like interactions, setting a new standard for future negotiation frameworks.

Ethical Statement

All user experiments on this framework were conducted under the approval of ¨Ozye˘gin the University Ethics Committee.

Acknowledgments

This work is partially supported by the Horizon Europe project SmartEdge under grant agreement No. 101092908 and by the Chist-Era grant CHIST-ERA-19-XAI-005, and by the Scientific and Research Council of Turkey (T ¨UB˙ITAK, G.A. 120N680).