3D-Aided Deep Pose-Invariant Face Recognition

3D-Aided Deep Pose-Invariant Face Recognition

Jian Zhao, Lin Xiong, Yu Cheng, Yi Cheng, Jianshu Li, Li Zhou, Yan Xu, Jayashree Karlekar, Sugiri Pranata, Shengmei Shen, Junliang Xing, Shuicheng Yan, Jiashi Feng

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 1184-1190. https://doi.org/10.24963/ijcai.2018/165

Learning from synthetic faces, though perhaps appealing for high data efficiency, may not bring satisfactory performance due to the distribution discrepancy of the synthetic and real face images. To mitigate this gap, we propose a 3D-Aided Deep Pose-Invariant Face Recognition Model (3D-PIM), which automatically recovers realistic frontal faces from arbitrary poses through a 3D face model in a novel way. Specifically, 3D-PIM incorporates a simulator with the aid of a 3D Morphable Model (3D MM) to obtain shape and appearance prior for accelerating face normalization learning, requiring less training data. It further leverages a global-local Generative Adversarial Network (GAN) with multiple critical improvements as a refiner to enhance the realism of both global structures and local details of the face simulator’s output using unlabelled real data only, while preserving the identity information. Qualitative and quantitative experiments on both controlled and in-the-wild benchmarks clearly demonstrate superiority of the proposed model over state-of-the-arts.
Keywords:
Machine Learning: Neural Networks
Machine Learning: Transfer, Adaptation, Multi-task Learning
Machine Learning: Semi-Supervised Learning
Machine Learning: Deep Learning
Machine Learning: Learning Generative Models
Computer Vision: Biometrics, Face and Gesture Recognition
Computer Vision: Computer Vision