Online Evasion Attacks on Recurrent Models:The Power of Hallucinating the Future

Online Evasion Attacks on Recurrent Models:The Power of Hallucinating the Future

Byunggill Joe, Insik Shin, Jihun Hamm

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Main Track. Pages 3121-3127. https://doi.org/10.24963/ijcai.2022/433

Recurrent models are frequently being used in online tasks such as autonomous driving, and a comprehensive study of their vulnerability is called for. Existing research is limited in generality only addressing application-specific vulnerability or making implausible assumptions such as the knowledge of future input. In this paper, we present a general attack framework for online tasks incorporating the unique constraints of the online setting different from offline tasks. Our framework is versatile in that it covers time-varying adversarial objectives and various optimization constraints, allowing for a comprehensive study of robustness. Using the framework, we also present a novel white-box attack called Predictive Attack that `hallucinates' the future. The attack achieves 98 percent of the performance of the ideal but infeasible clairvoyant attack on average. We validate the effectiveness of the proposed framework and attacks through various experiments.
Keywords:
Machine Learning: Adversarial Machine Learning
Computer Vision: Adversarial learning, adversarial attack and defense methods
Machine Learning: Recurrent Networks
Machine Learning: Robustness