Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning
Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning
Yuchen Cui, Pallavi Koppol, Henny Admoni, Scott Niekum, Reid Simmons, Aaron Steinfeld, Tesca Fitzgerald
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
Survey Track. Pages 4382-4391.
https://doi.org/10.24963/ijcai.2021/599
Human-in-the-loop Machine Learning (HIL-ML) is a widely adopted paradigm for instilling human knowledge in autonomous agents. Many design choices influence the efficiency and effectiveness of such interactive learning processes, particularly the interaction type through which the human teacher may provide feedback. While different interaction types (demonstrations, preferences, etc.) have been proposed and evaluated in the HIL-ML literature, there has been little discussion of how these compare or how they should be selected to best address a particular learning problem. In this survey, we propose an organizing principle for HIL-ML that provides a way to analyze the effects of interaction types on human performance and training data. We also identify open problems in understanding the effects of interaction types.
Keywords:
Humans and AI: General