Towards Efficient Detection and Optimal Response against Sophisticated Opponents

Towards Efficient Detection and Optimal Response against Sophisticated Opponents

Tianpei Yang, Jianye Hao, Zhaopeng Meng, Chongjie Zhang, Yan Zheng, Ze Zheng

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
Main track. Pages 623-629. https://doi.org/10.24963/ijcai.2019/88

Multiagent algorithms often aim to accurately predict the behaviors of other agents and find a best response accordingly. Previous works usually assume an opponent uses a stationary strategy or randomly switches among several stationary ones. However, an opponent may exhibit more sophisticated behaviors by adopting more advanced reasoning strategies, e.g., using a Bayesian reasoning strategy. This paper proposes a novel approach called Bayes-ToMoP which can efficiently detect the strategy of opponents using either stationary or higher-level reasoning strategies. Bayes-ToMoP also supports the detection of previously unseen policies and learning a best-response policy accordingly. We provide a theoretical guarantee of the optimality on detecting the opponent's strategies. We also propose a deep version of Bayes-ToMoP by extending Bayes-ToMoP with DRL techniques. Experimental results show both Bayes-ToMoP and deep Bayes-ToMoP outperform the state-of-the-art approaches when faced with different types of opponents in two-agent competitive games.
Keywords:
Agent-based and Multi-agent Systems: Multi-agent Learning
Agent-based and Multi-agent Systems: Agent Theories and Models