Efficient Multi-Agent Communication via Shapley Message Value
Efficient Multi-Agent Communication via Shapley Message Value
Di Xue, Lei Yuan, Zongzhang Zhang, Yang Yu
Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Main Track. Pages 578-584.
https://doi.org/10.24963/ijcai.2022/82
Utilizing messages from teammates is crucial in cooperative multi-agent tasks due to the partially observable nature of the environment. Naively asking messages from all teammates without pruning may confuse individual agents, hindering the learning process and impairing the whole system's performance. Most previous work either utilizes a gate or employs an attention mechanism to extract relatively important messages. However, they do not explicitly evaluate each message's value, failing to learn an efficient communication protocol in more complex scenarios. To tackle this issue, we model the teammates of an agent as a message coalition and calculate the Shapley Message Value (SMV) of each agent within it. SMV reflects the contribution of each message to an agent and redundant messages can be spotted in this way effectively. On top of that, we design a novel framework named Shapley Message Selector (SMS), which learns to predict the SMVs of teammates for an agent solely based on local information so that the agent can only query those teammates with positive SMVs. Empirically, we demonstrate that our method can prune redundant messages and achieve comparable or better performance in various multi-agent cooperative scenarios than full communication settings and existing strong baselines.
Keywords:
Agent-based and Multi-agent Systems: Multi-agent Learning
Agent-based and Multi-agent Systems: Agent Communication
Machine Learning: Deep Reinforcement Learning