NegoLog: An Integrated Python-based Automated Negotiation Framework with Enhanced Assessment Components

1 Introduction

Agent-based negotiation aims to resolve conflicts of interest with intelligent agents negotiating on behalf of users in a range of group decision-making scenarios, spanning from commercial transactions to everyday life situations [Fatima et al., 2014; Marsa-Maestre et al., 2014]. Several simulation environments have been developed to facilitate the research in this field. Among them, the General Environment for Negotiation with Intelligent multi-purpose Usage Simulation, GENIUS framework [Hindriks et al., 2009; Lin et al., 2014], utilized in the Automated Negotiating Agents Competition(ANAC) [Jonker et al., 2017] plays a pivotal role in negotiating agent development and assessment. Researchers can design their negotiation strategies and protocols in Java and assess their performance by running tournaments using a rich negotiating agent repository over various scenarios. Recent developments in machine learning, including in negotiation, make Python a more suitable platform, given its rich set of libraries. Moreover, researchers typically need to provide their own logging mechanism if they need to store additional data for their approach (e.g., Reinforcement Learning, additional opponent modeling metrics), which is not straightforward. Members of the ANAC research community have developed GeniusWeb [Jonker et al., 2019] to resolve the programming language dependency and to enable real applications by distributing preferences, protocols, and agents over separate servers. This allows negotiators to rely fully on their servers and not on those of others; however, its distributed nature makes the agile development of agents difficult.

Another leading negotiation tool also utilized in ANAC is Negotiation Multi-Agent System (NegMAS), offering Pythonbased support for various protocols, strategies, and scenarios, especially in complex environments such as supply chain management [Mohammad et al., 2020]. Highlighting the interdependence of utility functions in negotiations, NegMAS distinguishes itself by offering a platform where utility functions evolve during the negotiations, addressing the static nature of negotiation setups in previous platforms. NegMAS automatically generates Python-based plots that illustrate the negotiation dynamics within the current outcome space, a concept referred to as the Negotiation Dance. While NegMAS offers comprehensive evaluation metrics, it lacks automated analysis of negotiation moves, such as reporting the efficacy of each negotiation move (DANCE steps), as discussed by [Hindriks et al., 2011]. Furthermore, NegMAS does not facilitate the evaluation of opponent modeling approaches independently from negotiation strategies (i.e., without needing to incorporate them into negotiation strategies). Consequently, it does not report metrics for opponent modeling (e.g., Root Squared Mean Error (RSME) and Spearman Correlations) to compare preference learning accuracy.

Recently, another framework called Negotiation Simulation Platform (NegoSim) has been introduced [Ebrahimnezhad and Fujita, 2023]. It extends the traditional bidding, opponent model, and acceptance strategy components with preference elicitation to facilitate agent development. While NegoSim offers user-friendly analysis tools, it does not generate extensive logs automatically. It provides a limited set of evaluation metrics that do not include measuring the performance of opponent modeling algorithms.

The main goal of the NegoLog framework is to equip researchers with comprehensive analytical capabilities for bilateral negotiations, allowing them to evaluate key negotiation components such as bidding strategy, opponent preferences, and strategy modeling. NegoLog empowers researchers to generate negotiation scenarios with varying sizes, degrees of opposition, and utility distribution. It caters to machine learning approaches, such as time-series analysis and reinforcement learning methods, by providing a set of logging mechanisms for the negotiation process (e.g., move, sensitivity, and cooperativeness analysis), outcomes (e.g., individual utilities, Nash distances, offer distributions), and opponent modeling (e.g., Spearman and Kendall-Tau rank correlations and RSME, move estimation). NegoLog's scenario generator and loggers are customizable, allowing the research community to extend and adapt NegoLog further.

2 NegoLog Framework

NegoLog 1 2 can evaluate bilateral negotiation tournaments among several agents across various domains. In each negotiation session of the tournament, the negotiation process is governed by Stacked Alternating Offers Protocol (SAOP) [Aydo˘gan et al., 2017]. Under this protocol, both parties negotiate without fully revealing their preferences, as they can either accept the opponent's bid or propose a counteroffer until a pre-defined deadline. If the negotiation ends without agreement, both parties receive their reservation value.

2.1 Architecture Design andModules

Similar to the other frameworks, NegoLog requires the implementation of an abstract class based on BOA components for agent development [Baarslag et al., 2014]. Consequently, developers can implement each component in a modular way. While running a tournament, the tournament class generates possible negotiation configurations (e.g., strategy and preference pairs) and runs each session accordingly, as shown in Figure 1. The tournament configurations can be saved as YAML data along with the chosen domains and reused/modified later as needed. Each of the selected opponent models can be fed with the current negotiation data and be evaluated in terms of prediction accuracy independently from the negotiation strategy. Furthermore, NegoLog contains a customizable analytics and visualization module to generate diverse statistical analyses and graphs. Researchers can also easily create custom analysis modules with built-in abstract logger classes. The following sections briefly explain the essential components.

Domain Generator

NegoLog offers a versatile Domain Generator tool, enabling users to create negotiation scenarios based on user-defined parameters such as the number of issues and values, opposition, and utility distributions (e.g., uniform, random). Users also have the flexibility to edit generated domains manually. Besides, a recent research paper [de Jonge, 2022] underscores that balanced domains can be exploitable with sophisticated agents like Micro and outperform opponents despite their simplicity; accordingly, NegoLog has a balance score parameter to allow users to create both balanced and unbalanced domains. Our balance score metric measures to what extent the utility distribution for the agents is balanced and calculated as the average utility differences over the possible outcomes (i.e., 1n ∑ni=1 (U A i -U B i )). Note that it takes zero if the utility distribution is balanced. Otherwise, on average, the utility of the outcomes is higher for one of the agents. Our domain generator applies a boosting factor to profile A or B to create unbalanced domains. Figure 2 denotes four domains with various balance scores and opposition values.

1 NegoLog GitHub Repository: github.com/aniltrue/NegoLog

2 The demonstration video can be accessed here.

Individual Opponent Model Evaluation

NegoLog introduces an abstract class for opponent model evaluation, which listens to negotiation sessions and builds several opponent models at once that estimate the opponent's preferences. This structure allows the independent development and evaluation of the opponent model and analysis of preference estimation performance [Baarslag et al., 2013a]. NegoLog can supply various estimators with received offers from the perspective of each agent throughout a negotiation session without directly utilizing an agent strategy. Thus, the performance evaluations can be conducted without relying on any specific bidding strategy. NegoLog keeps track of the performance of the opponent models by precise distance calculation via RMSE and estimated rank accuracy via Spearman and Kendall-Tau correlations [Baarslag et al., 2013b].

Web-Based User Interface

NegoLog is accessible through the console (i.e., command line) and a user-friendly Web-based interface. The user interface provides functionalities for configuring tournaments and monitoring the tournament process. Figure 3 illustrates the flow of tournament configuration generation. Users can define negotiation settings, select participating agents, specify loggers to be utilized and opponent preference estimators, and determine negotiation domains among generated scenarios during the configuration process. Additionally, the Webbased user interface includes the Domain Generator tool, enabling users to create, display, and edit negotiation domains conveniently via the Web-based interface.

Analytics and Visualization Modules

NegoLog offers a range of built-in loggers designed to provide detailed negotiation logs, advanced analysis for evaluating agent strategy and opponent models, and statistical graphs. Table 1 summarizes the built-in analyses and their corresponding graphs. The evaluations are performed from three aspects: negotiation process, negotiation outcome, and preference estimation. Negotiation process-related analyses shed light on agents' behavior during negotiation sessions (e.g., utility distributions, concession, and selfish move percentages). Our negotiation outcome-related analytics provide the overall performance of the agents in the entire tournament, that of domain-based performances (e.g., performance in each domain), and the agent's performance against each opponent over all domains. Figure 4 illustrates an example of an opponent-based individual utility analysis in a heatmap graph. Lastly, preference estimation-related analyses evaluate the accuracy of estimators in predicting opponent preferences, enabling independent evaluation of opponent models. We aim to provide all statistics without requiring additional log file processing and simplify the researchers' lives.

Figure 4: Opponent-Based (a) & Domain-Based (b) Analysis

2.2 Use Cases and Applications

Studies such as the conflict-based opponent model study [Keskin et al., 2023] underline the importance of opponent model-related analyses provided by NegoLog. Metrics like utility estimation performance (e.g., RMSE) and bid order estimation performance (e.g., Spearman and Kendall-Tau Correlations) are commonly employed in such studies. Our system can calculate them at the end of each round or only in the final round. Collecting massive datasets for training deep learning models necessitates a comprehensive logging mechanism [Yesevi et al., 2022]. These studies highlight the power of our framework in addressing these needs.

3 Conclusion and FutureWork

NegoLog has been developed to be a versatile and userfriendly framework for researching, agile development, and comparative evaluation of automated negotiation agents. From the development and research perspective of bilateral negotiations, it combines the best of the state-of-the-art in automated negotiation research frameworks. It has the rapid development and testing options of NegoMAS, combined with the repositories of agents, domains, and additive utility functions of Genius. It provides advanced customizable loggers, auto-generated statistics, visualization tools, and outcome analytics, e.g., Pareto optimality, distance to Nash, average utilities, and sensitivity to opponent's moves. Its novel features include the visualization of negotiation tournaments (heatmaps), the statistics of the dance moves, and tools for the customized generation of domains and profiles. The domain generator allows the developer to customize the characteristics of the domains and profiles generated, such as size, competitiveness, and unbalanced utility distribution. Unlike other tools, NegoLog enables modular, independent evaluation of opponent modeling approaches; furthermore, NegoLog supports the analysis of bidding and opponent modeling strategies in line with the BOA framework and the DANCE moves. These tools foster the independent development and analysis of sophisticated negotiating agents and preference estimation approaches. Incorporating an automated scenario generation tool completes the framework, offering a user-friendly and customizable environment for negotiation-related research.

Acknowledgments

This work has been partially supported by the Chist-Era grant CHIST-ERA-19-XAI-005, and by the Scientific and Research Council of Turkey (T ¨UB˙ITAK, G.A. 120N680). This publication is part of the Vidi project COMBINE (VI.Vidi.203.044) which is (partly) financed by the Dutch Research Council (NWO). We also thank Duru Balpınarlı for her valuable contributions and feedback.