A Formal Model for Multiagent Q-Learning Dynamics on Regular Graphs

A Formal Model for Multiagent Q-Learning Dynamics on Regular Graphs

Chen Chu, Yong Li, Jinzhuo Liu, Shuyue Hu, Xuelong Li, Zhen Wang

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Main Track. Pages 194-200. https://doi.org/10.24963/ijcai.2022/28

Modeling the dynamics of multi-agent learning has long been an important research topic. The focus of previous research has been either on 2-agent settings or well-mixed infinitely large agent populations. In this paper, we consider the scenario where n Q-learning agents locate on regular graphs, such that agents can only interact with their neighbors. We examine the local interactions between individuals and their neighbors, and derive a formal model to capture the Q-value dynamics of the entire population. Through comparisons with agent-based simulations on different types of regular graphs, we show that our model describes the agent learning dynamics in an exact manner.
Keywords:
Agent-based and Multi-agent Systems: Multi-agent Learning
Agent-based and Multi-agent Systems: Agent Societies
Agent-based and Multi-agent Systems: Agent-Based Simulation and Emergence