Beyond Universal Saliency: Personalized Saliency Prediction with Multi-task CNN

Beyond Universal Saliency: Personalized Saliency Prediction with Multi-task CNN

Yanyu Xu, Nianyi Li, Junru Wu, Jingyi Yu, Shenghua Gao

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
Main track. Pages 3887-3893. https://doi.org/10.24963/ijcai.2017/543

Saliency detection is a long standing problem in computer vision. Tremendous efforts have been focused on exploring a universal saliency model across users despite their differences in gender, race, age, etc. Yet recent psychology studies suggest that saliency is highly specific than universal: individuals exhibit heterogeneous gaze patterns when viewing an identical scene containing multiple salient objects. In this paper, we first show that such heterogeneity is common and critical for reliable saliency prediction. Our study also produces the first database of personalized saliency maps (PSMs). We model PSM based on universal saliency map (USM) shared by different participants and adopt a multi-task CNN framework to estimate the discrepancy between PSM and USM. Comprehensive experiments demonstrate that our new PSM model and prediction scheme are effective and reliable.
Keywords:
Multidisciplinary Topics and Applications: Personalization and User Modeling
Robotics and Vision: Vision and Perception