Image-level to Pixel-wise Labeling: From Theory to Practice

Image-level to Pixel-wise Labeling: From Theory to Practice

Tiezhu Sun, Wei Zhang, Zhijie Wang, Lin Ma, Zequn Jie

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 928-934. https://doi.org/10.24963/ijcai.2018/129

Conventional convolutional neural networks (CNNs) have achieved great success in image semantic segmentation. Existing methods mainly focus on learning pixel-wise labels from an image directly. In this paper, we advocate tackling the pixel-wise segmentation problem by considering the image-level classification labels. Theoretically, we analyze and discuss the effects of image-level labels on pixel-wise segmentation from the perspective of information theory. In practice, an end-to-end segmentation model is built by fusing the image-level and pixel-wise labeling networks. A generative network is included to reconstruct the input image and further boost the segmentation model training with an auxiliary loss. Extensive experimental results on benchmark dataset demonstrate the effectiveness of the proposed method, where good image-level labels can significantly improve the pixel-wise segmentation accuracy.
Keywords:
Machine Learning: Neural Networks
Machine Learning: Deep Learning
Computer Vision: Perception
Computer Vision: Computer Vision