CFR-MIX: Solving Imperfect Information Extensive-Form Games with Combinatorial Action Space

CFR-MIX: Solving Imperfect Information Extensive-Form Games with Combinatorial Action Space

Shuxin Li, Youzhi Zhang, Xinrun Wang, Wanqi Xue, Bo An

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
Main Track. Pages 3663-3669. https://doi.org/10.24963/ijcai.2021/504

In many real-world scenarios, a team of agents must coordinate with each other to compete against an opponent. The challenge of solving this type of game is that the team's joint action space grows exponentially with the number of agents, which results in the inefficiency of the existing algorithms, e.g., Counterfactual Regret Minimization (CFR). To address this problem, we propose a new framework of CFR: CFR-MIX. Firstly, we propose a new strategy representation that represents a joint action strategy using individual strategies of all agents and a consistency relationship to maintain the cooperation between agents. To compute the equilibrium with individual strategies under the CFR framework, we transform the consistency relationship between strategies to the consistency relationship between the cumulative regret values. Furthermore, we propose a novel decomposition method over cumulative regret values to guarantee the consistency relationship between the cumulative regret values. Finally, we introduce our new algorithm CFR-MIX which employs a mixing layer to estimate cumulative regret values of joint actions as a non-linear combination of cumulative regret values of individual actions. Experimental results show that CFR-MIX outperforms existing algorithms on various games significantly.
Keywords:
Multidisciplinary Topics and Applications: Security and Privacy
Agent-based and Multi-agent Systems: Noncooperative Games
Multidisciplinary Topics and Applications: Computational Sustainability