Anti-Forgery: Towards a Stealthy and Robust DeepFake Disruption Attack via Adversarial Perceptual-aware Perturbations

1 Introduction

In recent years, we have witnessed the remarkable development of GAN in image synthesis and fine-grained image manipulation. Attackers could leverage GAN to generate realistic and natural synthetic images, audios, and videos (also known as DeepFake), which poses potential security and privacy concerns to individuals [Juefei-Xu et al., 2021]. In this AI era, we are living in a world where we cannot believe our eyes and ears anymore. Thus, effective countermeasures should be developed for fighting against DeepFakes.

∗Run Wang's Email: wangrun@whu.edu.cn

To defend against DeepFakes, various countermeasures are developed in both passive and proactive manner. However, both of them are still in their early stage and not prepared for tackling this emerging severe threat. The passive DeepFake detection merely determines the real or fake which is an ex-post DeepFake defense manner [Hu et al., 2021]. More importantly, the existing DeepFake detection techniques can not well tackle the DeepFakes created by unknown synthetic techniques [Wang et al., 2021a]. According to a result of the DeepFake detection challenge (DFDC) held by Facebook, the winner team can give a detection accuracy less than 70%. Thus, some researchers are working on developing proactive DeepFake defending techniques by adding adversarial noises into the source images to disrupt the DeepFake creation [Yang et al., 2020; Segalis, 2020]. Specifically, they hope that the created DeepFakes with added perturbations exhibit visually noticeable artifacts and provide signals for detectors. The DeepFake disruption is the most promising countermeasures for fighting against DeepFakes in a proactive manner, which shows potential for tackling DeepFakes in the wild [Chen et al., 2021]. However, existing DeepFake disruption studies by adding adversarial noises suffer challenge in tackling input transformations, which limits their practicality applications.

Existing studies mostly borrow the idea of prior adversarial attack (e.g., FGSM, PGD) via gradient-based or optimization-based strategies to generate imperceptible adversarial noises. However, a series of studies reveal that such adversarial noises could be easily removed or destroyed. A recent study, MagDR [Chen et al., 2021], illustrated that a simple input reconstruction could destroy the added adversarial noises for disrupting DeepFakes.

The ultimate goal of proactive DeepFake defense is to create low-quality DeepFakes when applying various techniques for forgery purpose. The created low-quality DeepFakes will exhibit noticeable artifacts and could be easily spotted even with simple DeepFake detectors. Thus, a desired proactive DeepFake defense technique should satisfy the following three properties, robust to input transformations (e.g., reconstruction), visually natural perturbations to human eyes, working on the black-box settings without obtaining any knowledge of the forgery model.

In this paper, we investigate the vulnerability of GANs in image synthesis and seek a kind of robust adversarial perturbation which could survive drastic input transformation. Specifically, we propose a novel anti-forgery approach by adding perceptual-aware perturbations into the source data to enforce the created DeepFakes exhibiting noticeable artifacts and detectable with simple DeepFake detectors. Our perceptual-aware perturbations operate on the Lab color space in the de-correlated a and b channel in an incessant manner. Unlike the prior studies working on the RGB color space which introduces perceivable distortions incurred by unnatural colors even with small variations [Huang et al., 2021], our proposed method works on a naturalcolor range of semantic categories which generates perceptually natural facial images. More importantly, our generated perceptual-aware perturbation is robust against input transformation which shows potential to be deployed in the real world. Figure 1 illustrates the comparison with the existing common DeepFake disruption techniques by adding adversarial noises on the RGB color space.

Extensive experiments conducted on the three types of DeepFake demonstrate the effectiveness of our proposed method and robustness against the input transformation, like reconstruction, compression. Additionally, some ablation studies are conducted for illustrating that our added perturbations could provide a clear signal for DeepFake detection. Our main contributions are summarized as follows:

• We introduce a novel anti-forgery method by adding adversarial perceptual-aware perturbations against forgery in a proactive manner by transforming facial images in an incessant and natural way, as opposed to adding meaningless and sparse noises in prior studies which are not robust to drastic input transformations, like reconstruction.
• We employ a simple yet effective method to generate the perceptual-aware perturbations by operating on the Lab color space in the decorrelated a and b channel that generates a visually natural face which could be further leveraged disrupting the DeepFake creation.
• For the first time, we conduct experiments on three types of DeepFakes to demonstrate the effectiveness of our devised perceptual-aware perturbations in creating DeepFakes with noticeable artifacts and robustness in surviving the input reconstruction compared with prior studies.
• Our research findings hint a new research direction towards perceptual-aware adversarial attack by investigating natural and robust perturbations against various transformations. We hope that more studies working on exploiting the vulnerabilities of GAN in image synthesis to develop DeepFake proactive countermeasures which could fight against DeepFakes in the wild.

2 RelatedWork

2.1 DeepFake Creation

In the past years, GAN and its variants have achieved tremendous success in image synthesis and manipulation. DeepFake leverages the success of GANs to generate forgery facial images or videos, which pose potential threats to individual privacy and political security. We are living in a world where we cannot believe our eyes anymore. In general, entire synthesis, attribute editing, identity swap, and face reenactment are four common types of DeepFakes.

The entire synthesis produces non-existent images in the world with random vectors to the network, such as PGGAN, StyleGAN. The attribute editing is a fine-grained face manipulation by modifying the simple face attributes (e.g., hair color, bald) and complex face attributes (e.g., gender, age) with popular GANs, like STGAN [Liu et al., 2019], StarGAN [Choi et al., 2018]. The identity swap is known as the popular face swap by swapping the face between target and source facial image. FaceSwap 1 and DeepFaceLab [Perov et al., 2020] are two popular available tools for identity swap. Similar to identity swap, face reenactment also known as expression swap involves facial expression swap between target and source facial image by applying popular tools, such as Face2Face [Thies et al., 2016].

Among the four types of DeepFake, the entire synthesis produces non-existent facial images without involving any source image manipulation, thus it will not bring any privacy or security concerns to individuals. In this paper, we mainly focus on the other three types DeepFakes (e.g., attribute editing, identity swap, and face reenactment), which pose security and privacy concerns to individuals.

1 https://github.com/Oldpan/Faceswap-Deepfake-Pytorch

2.2 DeepFake Defense

DeepFake Detection. Determining if the facial image is real or fake by observing the subtle differences between real and fake images is a straightforward idea for defending DeepFakes. However, DeepFake detection is an ex-post forensic manner and suffers various known challenges, such as generalization to tackle unknown synthetic techniques, robust against quality degradations, evasion to adversarial attacks, which are obstacles for its practical usage to be deployed in the wild [Juefei-Xu et al., 2021].

Studies have shown that the subtle differences between real and fake could be revealed in the spatial [Wang et al., 2020; Wang et al., 2021b] and frequency domain [Qian et al., 2020], however, they are susceptible to adversarial attack via various image-level and frequency-level manipulations [Carlini and Farid, 2020; Huang et al., 2020]. Another line work investigate the biological signals which are hard to replicate [Hu et al., 2021]. Unfortunately, these passive DeepFake detection techniques cannot be applied for blocking the wide spreading of DeepFakes in advance before causing damage impacts.

DeepFake Disruption. To address the aforementioned limitations of DeepFake detection, DeepFake disruption aims at disrupting the images being DeepFaked proactively by adding adversarial perturbations into the source image to produce damaged images. Recently, massive studies have shown that Deep Neural Networks (DNN) are vulnerable to adversarial attack (e.g., FGSM, PGD) by adding imperceptible noises to craft so-called adversarial examples. Thus, some researchers explore whether the DeepFakes are vulnerable to adversarial examples as well.

Yang et al. leveraged differentiable random image transformations to yield adversarial faces with noticeable artifacts [Yang et al., 2020]. Ruiz et al. presented a spreadspectrum adversarial attacks against conditional image translation network in a grey-box scenario [Ruiz et al., 2020]. Yeh et al. proposed two types of adversarial attack against image translation GANs with designed adversarial loss function by gradient optimization to output blurred and distorted output [Yeh et al., 2020]. Huang et al. [Huang et al., 2021] proposed an initiative defense framework against facial manipulation by injecting venom into the target manipulation model in the black-box settings. This work adds invisible perturbations by training a surrogate model, however, they merely claim its effectiveness in attribute editing and face reenactment since these two types of manipulation model share a similar pipeline. Unfortunately, it will be hardly applied in defending identity swap which is more challenge to find an appropriate surrogate model to generate perturbations.

Unfortunately, the existing DeepFake disruption techniques employ a naive adversarial attack to generate invisible perturbations which could be easily detected and corrupted by input reconstruction. A recent study MagDR [Chen et al., 2021] also demonstrates this and reveals that the DeepFake disruption techniques by adopting C&W and PGD are all failed in producing damaged images. Specifically, MagDR

employs image restoration techniques to remove the added perturbations and obtains the desired form further.

In this paper, we challenge the effectiveness of MagDR in dealing with our perceptual-aware perturbations which are generated in an incessant and natural manner. We aim at developing stealthy and robust imperceptible adversarial perturbations which could effectively survive MagDR and provide strong protection to our shared faces in social media.

3 Problem Statement

In this paper, we describe the DeepFake defense from both passive DeepFake detection and proactive DeepFake defense in Figure 1. In the real world, a user uploads his/ her personal facial images to social media, like Twitter to share with friends or anyone on the Internet. Unfortunately, an attacker could easily pick the victim's photos for malicious manipulation with GAN to create fake pornography for individuals to raise privacy concerns or to generate a fake official statement for celebrities to cause panic. Our method adds adversarial perceptual-aware perturbations into personal images before uploading to social media without introducing any visual artifacts. The facial image with our applied perturbations could prevent the GAN-based manipulations well by generating damaged facial images with noticeable artifacts. The key idea here is that our perturbed facial images should be robust enough to survive input transformations, like reconstruction before the DeepFake process. Finally, the proactive DeepFake defense is achieved for creating damaged images and exhibiting clear fake signals for detectors.

Here, we elaborate the details regarding Figure 1. In the left panel, we present the pipeline of passive DeepFake detection. A user (Figure 1-a1) first uploads personal facial images to the social media platform (Figure 1-b1). Then, attackers pick the shared photos to create DeepFake (Figure 1-c1) (e.g., change the color of hair, the intensity of facial expressions) which causes the spread of misinformation in the platform (Figure 1-d1). Finally, the DeepFake detector (Figure 1-e1) determines whether the suspicious one is real or fake in an ex-post forensics manner to further block the misinformation spreading in social media platforms if fake is confirmed. In the right panel, we present the pipeline of proactive DeepFake defense by comparing our proposed method with the prior DeepFake disruption approach by employing popular adversarial noises.

In contrast to the passive DeepFake detection in the left panel, a user first (Figure 1-a2) injects perturbations into facial images before uploading them to social media platforms (Figure 1-c2, c3). Our proposed method injects perceptualaware perturbations by operating the Lab color space in an incessant manner (Figure 1-b2), in comparing with prior studies inject imperceptible adversarial noises by perturbing the RGB color space in a sparse manner (Figure 1-b3). However, in a real battleground scenario, an adversary (Figure 1d2, d3) tries to remove or corrupt the added perturbations by employing input transformations (e.g., reconstruction) before creating DeepFakes. In creating DeepFakes (Figure 1-e2, e3), our proposed method adding perceptual-aware perturbations could survive the adversarial attack with input transformations and exhibit noticeable artifacts in the manipulated facial images (Figure 1-f2) and blocked immediately without further spreading to cause any panic and privacy concerns for individuals. However, the prior studies are not robustly output realistic and natural DeepFakes which cause the spreading of misinformation (Figure 1-f3).

In summary, in such a real battleground scenario, both the passive DeepFake detection and prior proactive DeepFake disruption methods failed in blocking the misinformation spreading before resulting severe social impacts.

4 Methodology

4.1 Insight

DeepFake disruption is a promising countermeasure for defending DeepFakes proactively [Chen et al., 2021]. However, the existing studies suffer two issues, 1) they are not robust to the input transformation which is demonstrated in MagDR, 2) the effectiveness on all types of DeepFakes is unclear [Huang et al., 2021], which is an obstacle for its practical usage. Thus, the community is not prepared for tackling this emerging threat.

A straightforward idea for defending GAN-based DeepFake proactively is to explore the vulnerability of GAN in image synthesis and manipulation. To this end, adversarial examples are employed for attacking GAN by adding imperceptible perturbations into the input to introduce distortions on the output. Generally speaking, the created output with perturbations should satisfy the following three properties when defending DeepFakes in the wild.

• Visually natural to human eyes. The uploaded facial images to the social media platform are mainly used to share exciting moments to friends at the most of the time, thus the visually quality degradation is not allowed.
• Robust to input transformation (e.g., reconstruction). In the real world, adversaries will corrupt the added perturbations intentionally to break the defense mechanism.
• Generalizing well to a wide types of DeepFakes. A facial image spreading in the social media platform would suffer various forgery which all cause security and privacy concerns to individuals.

Inspired by the advantages of Lab color space that are perceptual uniform and more satisfy the humans' to the outside world, we operate the a and b channel of Lab color space to generate natural color perturbations. More importantly, our perturbations are generated in an incessant manner, unlike the perturbations generated on RGB color space which are sparse and introduce distortions even small variations existed. We illustrate the power of Lab color space in color representation in Section 4.3. In the following subsections, we introduce the framework of our proposed anti-forgery method to generate adversarial perceptual-aware perturbations to disrupt the DeepFake creation.

4.2 Adversarial Attack to GAN

The foundation of DeepFake creation is GAN which consists of two deep learning networks, the Generator G and Discriminator D. In the GAN training, the Generator G tries to generate samples indistinguishable to the real, while the Discriminator D learns to differentiate synthesized sample G(z) from noises z and real samples. For example, a typical imageto-image translation network CycleGAN could be employed for identity swap 2 . Specifically, CycleGAN learns to build a mapping G : x → y and an inverse mapping F : y → x between two image domain x and y.

Similar to adversarial examples, the imperceptible perturbations used to disrupt DeepFakes are produced by introducing minimal distortion to preserve the natural effects of human eye vision. Let x be the input source image, θ is the imperceptible perturbation, ˜x is the crafted input with adversarial perturbations.

A generator G receives the two input x and ˜x and outputs G(x) and G(˜x) further. Ideally, G(x) and G(˜x) are totally different where G(˜x) exhibits noticeable artifacts to human eyes. More specifically, the objective function can be formulated as follows by maximize the distortion between G(x) and G(˜x).max

where ϵ is the maximum magnitude of the perturbation, L is a distance function for measuring the similarity, r is a groundtruth, r = G(x).

4.3 Lab Color Space

The Lab color space is designed to conform with human's sense of color which is perceptual uniform. A light channel L and two color channels a and b consist the Lab color space, where the L channel ranges from black (0) to white (100) representing the light, a channel ranges from green (-128) to red (+127), b channel ranges from blue (-128) to yellow (+127). Unlike RGB color space, we can change the lightness by simply modifying the value of L channel without involving any changes to the color channel a and b.

Due to the power of Lab color space in representing color space that is perceptual uniform and the wide value range in representing color, in this paper, our adversarial perturbations are generated by operating the Lab color space to add perturbation into the channel a and b.

4.4 Proposed Anti-forgeryMethod

Specifically, Algorithm 1 describes the details of our proposed method. Our goal for the generated perturbations is natural to human eyes without introducing any quality degradation and resistant enough to potential input transformation attacks. First, we convert the input image from the RGB color space to the Lab color space for adding perceptual uniform perturbations into the channel a and b. Then, the perturbations is updated by attacking the surrogate model M via optimization-based strategy adopted in C&W.

To improve the transferability in tackling multiple facial attributes, we select different facial attribute label c in each iteration. The objective function can be formulated as follows.

2 https://github.com/shaoanlu/faceswap-GAN

Input : Input image x, Surrogate model M, Label c ∈ C, Iteration K, Objective o, Learning rate τ . Output: Adversarial sample xadv with perturbation. 1 initialization θa, θb 2 for i ∈ {1...K} do 3 l, a, b ← rgb2lab(x) 4 ▷ Add perturbations for both channel a and b. 5 a′ ← a + θa 6 b′ ← b + θb 7 ▷ Convert into RGB color space. 8 xadv ← lab2rgb(l, a′ , b′ ) 9 ▷ Update the perturbations of channel a and b. 10 θa ← θa - τ · ∇θa L(M(xadv, c), o) 11 θa ← clip(θa, -ϵ, ϵ) 12 θb ← θb - τ · ∇θb L(M(xadv, c), o) 13 θb ← clip(θb, -ϵ, ϵ) 14 return xadv

where L could be L1 or L2 norm, o could be 0, 1, or even Gaussian noises. When o is a regular translation image, the objective can be formulated as follows.

4.5 Comparison with Popular Adversarial Noises

A straightforward idea for attacking GAN to disrupt the DeepFake creation would be employing the gradient-based and optimization-based strategies to generate imperceptible perturbations. However, such restricted perturbations operating on the RGB color space suffer sparse issues which could be easily detected and corrupted by input transformations [Meng and Chen, 2017]. Specifically, the three channels in RGB color space have strong correlations, indicating that natural perturbations require the modification of these three channels simultaneously. Thus, the perturbations generated by operating the RGB color space are not robust against real world input transformations.

In summary, our perceptual-aware adversarial perturbations by operating the Lab color space have the following main strengths. 1) Robust to input reconstruction. Our perturbations operated on Lab color space which is perceptual uniform, as opposite to the prior studies operating on the RGB space which is sparse. The perceptual uniform perturbations operated on the Lab color space is more likely to share the same distribution as the unperturbed input. 2) Generic to a wide types of fogery. There is a wide range of color values for operating on the Lab color space. We will always explore the best perturbations for disruption effectively.

5 Experiments

The experimental setup and more experimental results on the performance across diverse GANs, exploring other color spaces, ablation studies, visualizations, and the description of broader impact etc. , see the Appendix here 3 .

3 http://wangrun.github.io/supp/Anti forgery IJCAI2022.pdf

5.1 Effectiveness Evaluation

We report the experimental results of effectiveness evaluation on all three types of DeepFakes involving the manipulation of source images. Since a large portion of GANs in image synthesis are fine-grained attribute editing, we employ three different GANs (e.g., StarGAN, AttGAN, and Fader Network) which could represent the SOTA performance of GANs in attribute editing. Experimental results show that our proposed method by operating the Lab color space could disrupt the SOTA GANs for DeepFake creation with competitive performance. The visualization of our proposed method in disrupting DeepFaked images refer to the supplemental material.

Table 1 summarizes the effectiveness evaluation and comparison with two baselines. For the three attributes editing GANs, our method achieved competitive results with the two baselines measured by four different metrics and significantly outperforms them in AttGAN. In the metrics of ASR by giving a distortion restriction, our method outperforms the two baselines in the three GANs for attributes editing.

To the best of our knowledge, this is the first work to present a comprehensive evaluation on the three types of DeepFakes, including attribute editing, identity swap, and face reenactment. Experimental results in Table 1 demonstrated the effectiveness of our proposed method and illustrated that our proposed method achieved competitive results in comparing with the two baselines.

5.2 Robustness Analysis

In this section, we report the evaluation results of our method against common input transformations, including JPEG compression, Gaussian Blur, reconstruction, etc. . Specifically, in the real world scenarios, the input with added perturbations would be spread in the social media platform and suffers various real degradations (e.g., compression, blur, etc. ). A recent study MagDR revealed that the prior DeepFake disruption studies failed in tackling input reconstruction where the added imperceptibly could be easily removed or corrupted by simple reconstruction. Thus, we aim to answer one question that whether our proposed method is robust against the common input transformations, especially the input reconstruction studied in MagDR.

Performance on common input transformations. Table 2 summarizes the performance of our method in tackling various input transformations. Experimental results show that our method shows promising performance in the adversary settings and demonstrated its robustness in surviving various input transformations. Specifically, our method significantly outperforms the two competitive baselines in all four adversary settings. In Table 2, the second row Defense means a clean setting without any input transformations; the third row JPEG compression indicates that the input is compressed before leveraged for creating DeepFakes; the fourth row JPEG Compression (ϵ = 0.1) denotes that the added perturbations is restricted to a certain range larger than the default ϵ = 0.05; the fifth row Gaussian Blur represents that Gaussian blur is employed to input; the last row Blur (Data augmentation) explores whether the resistant against blur can be improved via data augmentation. Experimental results show that our method outperforms the two baselines measured by four different metrics on four input transformations.

Performance on MagDR [Chen et al., 2021]. Beyond the aforementioned popular input transformations, a recent study MagDR revealed that existing DeepFake disruption techniques are not robust to input reconstruction. Specifically, the added imperceptible perturbations could be corrupted or even removed by employing simple reconstruction and further output a high-quality DeepFake without noticeable artifacts. In our experiments, we also explore whether our method could resist the threat of input reconstruction.

Specifically, Table 3 summarizes the experimental results and compares them with one baseline. We reproduce MagDR and ensure the implementation details are correct by checking with the authors of MagDR. For the input with added perturbations, our method exhibits fewer perturbations than the baseline but leaves more perturbations in the input after employing MagDR which is our desired result as it would be lead to corrupted DeepFakes. For the output by employing MagDR, the value of the three metrics indicates the effectiveness in creating DeepFakes with noticeable artifacts. We can find that the MagDR failed in corrupting our added perceptual-aware perturbations over the three metrics by operating the Lab color space in comparison with the baseline.

In summary, a comprehensive robustness evaluation against both the common input transformations and input reconstruction demonstrated that our proposed method shows strong capabilities in resisting common input transformations and outperforms the two baselines measured by four different metrics in tackling the three common input transformations. Our proposed method shows its potential to be deployed in the real-world scenarios.

5.3 Discussion

Here, we discuss the limitations of our proposed method by operating on the Lab color space to generate perceptualaware perturbations for DeepFake disruption and suggest provisional countermeasures against them. Our generated perturbations are added into the whole input which suffers the potential threat that the adversary crops the background to destroy the perturbations directly. In our future work, we explore more robust technique to evade such attack by setting masks to enforce the perturbations are added into the internal region of face.

6 Conclusion and Future Research Direction

We propose the first adversarial perceptual-aware perturbations operating on the Lab color space for anti-fogery and performed an extensive evaluation of our method on three types of DeepFakes. Experimental results demonstrate its effectiveness in disrupting DeepFakes by exposing noticeable artifacts to human eyes while preserving the visually naturalness of the source image. More importantly, our method exhibits its robustness against input reconstruction which significantly outperforms prior studies. In addition, the created DeepFakes by adding our perceptual-aware perturbations provide strong signals for DeepFake detection. Our findings of perceptualaware adversarial perturbations present a new insight for defending DeepFakes proactively by investigating more natural and robust adversarial perturbations like facial background [Xiao et al., 2020], etc. . Thus, in our future work, we will explore more physical scenes to serve as guards for protecting images being DeepFaked.

Acknowledgments

This research was supported in part by the National Key Research and Development Program of China under No.2021YFB3100700, the Fellowship of China National Postdoctoral Program for Innovative Talents under No.BX2021229, the Fundamental Research Funds for the Central Universities under No.2042021kf1030, the Natural Science Foundation of Hubei Province under No.2021CFB089, the Open Foundation of Henan Key Laboratory of Cyberspace Situation Awareness under No. HNTS2022004, the National Natural Science Foundation of China (NSFC) under No.61876134, No.U1836202, No.62076187, No.62172303.