Two-stage Training for Learning from Label Proportions

Two-stage Training for Learning from Label Proportions

Jiabin Liu, Bo Wang, Xin Shen, Zhiquan Qi, Yingjie Tian

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
Main Track. Pages 2737-2743. https://doi.org/10.24963/ijcai.2021/377

Learning from label proportions (LLP) aims at learning an instance-level classifier with label proportions in grouped training data. Existing deep learning based LLP methods utilize end-to-end pipelines to obtain the proportional loss with Kullback-Leibler divergence between the bag-level prior and posterior class distributions. However, the unconstrained optimization on this objective can hardly reach a solution in accordance with the given proportions. Besides, concerning the probabilistic classifier, this strategy unavoidably results in high-entropy conditional class distributions at the instance level. These issues further degrade the performance of the instance-level classification. In this paper, we regard these problems as noisy pseudo labeling, and instead impose the strict proportion consistency on the classifier with a constrained optimization as a continuous training stage for existing LLP classifiers. In addition, we introduce the mixup strategy and symmetric cross-entropy to further reduce the label noise. Our framework is model-agnostic, and demonstrates compelling performance improvement in extensive experiments, when incorporated into other deep LLP models as a post-hoc phase.
Keywords:
Machine Learning: Classification
Machine Learning: Deep Learning
Machine Learning: Weakly Supervised Learning