A Comparative Survey: Benchmarking for Pool-based Active Learning
A Comparative Survey: Benchmarking for Pool-based Active Learning
Xueying Zhan, Huan Liu, Qing Li, Antoni B. Chan
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
Survey Track. Pages 4679-4686.
https://doi.org/10.24963/ijcai.2021/634
Active learning (AL) is a subfield of machine learning (ML) in which a learning algorithm aims to achieve good accuracy with fewer training samples by interactively querying the oracles to label new data points. Pool-based AL is well-motivated in many ML tasks, where unlabeled data is abundant, but their labels are hard or costly to obtain. Although many pool-based AL methods have been developed, some important questions remain unanswered such as how to: 1) determine the current state-of-the-art technique; 2) evaluate the relative benefit of new methods for various properties of the dataset; 3) understand what specific problems merit greater attention; and 4) measure the progress of the field over time. In this paper, we survey and compare various AL strategies used in both recently proposed and classic highly-cited methods. We propose to benchmark pool-based AL methods with a variety of datasets and quantitative metric, and draw insights from the comparative empirical results.
Keywords:
Machine learning: General