Image-to-Image Translation with Multi-Path Consistency Regularization

Image-to-Image Translation with Multi-Path Consistency Regularization

Jianxin Lin, Yingce Xia, Yijun Wang, Tao Qin, Zhibo Chen

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
Main track. Pages 2980-2986. https://doi.org/10.24963/ijcai.2019/413

Image translation across different domains has attracted much attention in both machine learning and computer vision communities. Taking the translation from a source domain to a target domain as an example, existing algorithms mainly rely on two kinds of loss for training: One is the discrimination loss, which is used to differentiate images generated by the models and natural images; the other is the reconstruction loss, which measures the difference between an original image and the reconstructed version. In this work, we introduce a new kind of loss, multi-path consistency loss, which evaluates the differences between direct translation from source domain to target domain and indirect translation from source domain to an auxiliary domain to target domain, to regularize training. For multi-domain translation (at least, three) which focuses on building translation models between any two domains, at each training iteration, we randomly select three domains, set them respectively as the source, auxiliary and target domains, build the multi-path consistency loss and optimize the network. For two-domain translation, we need to introduce an additional auxiliary domain and construct the multi-path consistency loss. We conduct various experiments to demonstrate the effectiveness of our proposed methods, including face-to-face translation, paint-to-photo translation, and de-raining/de-noising translation.
Keywords:
Machine Learning: Learning Generative Models
Computer Vision: Computer Vision