Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets

Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets

Cong Fei, Bin Wang, Yuzheng Zhuang, Zongzhang Zhang, Jianye Hao, Hongbo Zhang, Xuewu Ji, Wulong Liu

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence
Main track. Pages 2929-2935. https://doi.org/10.24963/ijcai.2020/405

Generative adversarial imitation learning (GAIL) has shown promising results by taking advantage of generative adversarial nets, especially in the field of robot learning. However, the requirement of isolated single modal demonstrations limits the scalability of the approach to real world scenarios such as autonomous vehicles' demand for a proper understanding of human drivers' behavior. In this paper, we propose a novel multi-modal GAIL framework, named Triple-GAIL, that is able to learn skill selection and imitation jointly from both expert demonstrations and continuously generated experiences with data augmentation purpose by introducing an auxiliary selector. We provide theoretical guarantees on the convergence to optima for both of the generator and the selector respectively. Experiments on real driver trajectories and real-time strategy game datasets demonstrate that Triple-GAIL can better fit multi-modal behaviors close to the demonstrators and outperforms state-of-the-art methods.
Keywords:
Machine Learning: Adversarial Machine Learning
Machine Learning: Reinforcement Learning
Machine Learning Applications: Applications of Reinforcement Learning