BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning
BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning
Lun Wang, Zaynah Javed, Xian Wu, Wenbo Guo, Xinyu Xing, Dawn Song
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
Main Track. Pages 3699-3705.
https://doi.org/10.24963/ijcai.2021/509
Recent research has confirmed the feasibility of backdoor attacks in deep reinforcement learning (RL) systems. However, the existing attacks require the ability to arbitrarily modify an agent's observation, constraining the application scope to simple RL systems such as Atari games. In this paper, we migrate backdoor attacks to more complex RL systems involving multiple agents and explore the possibility of triggering the backdoor without directly manipulating the agent's observation. As a proof of concept, we demonstrate that an adversary agent can trigger the backdoor of the victim agent with its own action in two-player competitive RL systems. We prototype and evaluate BackdooRL in four competitive environments. The results show that when the backdoor is activated, the winning rate of the victim drops by 17% to 37% compared to when not activated. The videos are hosted at https://github.com/wanglun1996/multi_agent_rl_backdoor_videos.
Keywords:
Multidisciplinary Topics and Applications: Security and Privacy
Agent-based and Multi-agent Systems: Multi-agent Learning