Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis

Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis

Yang He, Ning Yu, Margret Keuper, Mario Fritz

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
Main Track. Pages 2534-2541. https://doi.org/10.24963/ijcai.2021/349

The rapid advances in deep generative models over the past years have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes. These advances make assessing the authenticity of visual data increasingly difficult and pose a misinformation threat to the trustworthiness of visual content in general. Although recent work has shown strong detection accuracy of such deepfakes, the success largely relies on identifying frequency artifacts in the generated images, which will not yield a sustainable detection approach as generative models continue evolving and closing the gap to real images. In order to overcome this issue, we propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection. The re-synthesis procedure is flexible, allowing us to incorporate a series of visual tasks - we adopt super-resolution, denoising and colorization as the re-synthesis. We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios involving multiple generators over CelebA-HQ, FFHQ, and LSUN datasets. Source code is available at https://github.com/SSAW14/BeyondtheSpectrum.
Keywords:
Machine Learning: Classification
Machine Learning: Deep Learning
Multidisciplinary Topics and Applications: Security and Privacy