Are Watermarks Bugs for Deepfake Detectors? Rethinking Proactive Forensics

1 Introduction

Large generative models like ChatGPT and Stable Diffusion are transforming the way we communicate, illustrate, and create [Hacker et al., 2023]. Despite their positive potential in applications such as entertainment, education, art, etc., we cannot neglect the negative impacts. Particularly, the photorealistic Deepfake [Hu et al., 2022], interpreted as "fake face" generated through "deep learning", has taken false messages and misleading content to a new level, thereby damaging social trust and public interest [Juefei-Xu et al., 2022]. In response to this dilemma, emerging standards such as the Coalition for Content Provenance and Authenticity [C2PA, 2021]

∗Corresponding author.

call for embedding provenance information within the released images online [Bui et al., 2023]. To build Responsible AI, major tech companies, including Google and Microsoft, are also developing tools to trace the origin and identify synthetically altered or created images [Zhao et al., 2023a]. It has never been more important to understand the provenance and authenticity of the images we see, helping us make informed decisions and protect ourselves from deception.

Robust watermarking is the prevalent measure for the task of provenance tracking [Zhu et al., 2018; Tancik et al., 2020]. In such a watermarking prototype, subtle watermark signals that identify the provenance are imperceptibly embedded into the host image by a watermark encoder, rendering them almost invisible to the human naked eyes. After that, even if the watermarked image has been subjected to severe distortions, the watermark can still be extracted correctly by a watermark decoder in a blind manner, i.e., without knowledge of the original host image. It is also acknowledged that the watermark embedding can be accomplished during the content generation [Xiong et al., 2023], similar to the encoder implanted into a specific generative model. It should be noted that in this paper, we focus on more general watermarking methods, which modify the host images and are applicable to arbitrary generative models as well as non-generated images.

In previous studies, proactive watermark injection and passive Deepfake detection were completely independent, without any consideration of their correlation. However, we reveal that in practice, the presence of watermarks leads to more false-negative results when using most passive detectors (see Table 1), which is indeed contrary to the belief in AI responsibility and does a disservice. This can be explained by the fact that current watermarking models were originally devised for genuine images, where the watermarks are prone to overlap with the forgery signals used for detection. One underlying assumption for the watermark effects is that well-trained detectors are asymmetrically tuned to detect patterns that make an image fake [Ojha et al., 2023]. Consequently, anything else without these patterns would be classified as genuine.

To gain more insights into this intriguing phenomenon, we remark that invisible watermarks are analogous to natural adversarial perturbations [Hou et al., 2023]. Both of them are crafted by subtle signals that are not easily perceptible by human eyes; adversarial perturbations are intended to cause misjudgments of the detectors, while watermarks are designed to enable provenance tracking but unintentionally reduce the detection performance. More fundamentally, from the perspective "adversarial examples are features", they can be attributed to non-robust features that are highly predictive for the detectors [Ilyas et al., 2019]. Therefore, in another vein, it is possible for the embedded watermarks to be both recoverable and adversarial at the same time.

To this end, we propose AdvMark, which exploits the adversarial vulnerability of passive detectors for good, making the watermarks adversarial to positively improve the performance of Deepfake detectors. The distinctions between the proposed AdvMark and current watermarking models are observable in Figure 1. Unlike previous harmful adversarial examples and watermarking, our proposed helpful adversarial watermarking deceives the detector into correctly classifying the input image following its ground-truth label. Specifically, in contrast to the original images that were incorrectly predicted, the watermarked images force the detector to report a correct prediction. Meanwhile, the watermarks should not change the detection results of the original correctly predicted images. It is also worth noting that the meaningful watermarks are capable of tracking the image's provenance once extracted, while the meaningless perturbations are not.

Our contributions can be summarized into three-fold:

• We present a harmless proactive forensics solution, AdvMark, which leverages robust watermarking to fool Deepfake detectors for the first time, where the embedded watermarks behave both recoverable and adversarial, thereby achieving the purposes of provenance tracking and detectability enhancement simultaneously.
• To the best of our knowledge, this is the first attempt to formulate the concept of helpful adversarial watermarking, which amends the watermarked images to improve the accuracy of downstream Deepfake detection without tuning the in-the-wild detectors.
• Extensive experiments conducted on well-trained detec-

tors demonstrate the effectiveness of the proposed AdvMark under white- and black-box attacks, on various types of Deepfake, such as face swapping, expression reenactment, attribute editing, and entire synthesis.

2 RelatedWork

2.1 Deepfake Forensics

Up to now, Deepfake has covered massive types and advanced rapidly, leading to more realistic and convincing face forgeries. To control the dissemination of these, countermeasures are continually developed and can be roughly classified into three categories: passive forensics, proactive defense, and proactive forensics [Wu et al., 2023]. Among them, proactive defense performs adversarial attacks to destroy the creation of Deepfake [Huang et al., 2022]; however, it leaves few forensic clues, as the visually perceptible artifacts are easily perceived and can alert the adversary [Chen et al., 2021].

Passive Forensics. Deep learning-based detectors exploit powerful backbone networks, such as Xception [Rossler et al., 2019] and EfficientNet [Li et al., 2021], and their automatic feature extraction empirically outperforms hand-craft features. CNND [Wang et al., 2020] incorporates appropriate data augmentations during the network training. FFD [Dang et al., 2020] utilizes informative attention maps that indicate suspicious regions to detect the forgery. PatchForensics [Chai et al., 2020] develops the patch-based detector with limited receptive fields to exaggerate local artifacts in patch regions. MultiAtt [Zhao et al., 2021] captures local discriminative features from multiple attentive regions to identify subtle and local differences between genuine and forged images. RFM [Wang and Deng, 2021] erases several sensitive facial regions to guide the detector to allocate more attention to the representative regions. RECCE [Cao et al., 2022] mines common compact representations of genuine faces from the reconstruction perspective. SBI [Shiohara and Yamasaki, 2022] follows the self-supervised paradigm where the more general forgeries can be produced on-the-fly.

Proactive Forensics. To combat Deepfake proactively, FakeTagger [Wang et al., 2021] is derived from robust watermarking, where the embedded watermarks can survive both before and after Deepfake, enabling the tracking of the provenance of watermarked images. FaceSigns [Neekhara et al., 2022] introduces a semi-fragile watermarking framework, and its proactive detection is realized through verifying the extracted watermarks. SepMark [Wu et al., 2023] enables the embedded watermark to be extracted at different levels of robustness, achieving the purposes of provenance tracking and proactive detection simultaneously. Likewise, BiFPro [Liu et al., 2023] extends watermarking into diverse forensic scenarios where the watermark manifests either fragility or robustness. The embedded watermarks can be identityrelated messages [Zhao et al., 2023b; Wang et al., 2023; Guan et al., 2023]. Other than deep watermarking, SourceID-Tracker [Lin et al., 2022] employs steganography to hide the original images within their forged versions. PIMD [Asnani et al., 2022] and MaLP [Asnani et al., 2023] add the learned templates to conduct proactive detection and location, respectively. Moreover, AFP [Yu et al., 2021] and Faketracer

[Sun et al., 2022a] poison the training data, resulting in watermarked generative models.

2.2 Adversarial Attacks

Seminal work [Szegedy et al., 2014] discloses the vulnerable nature of deep neural networks to adversarial examples. Transferability is an intriguing property of adversarial examples; deliberately perturbed inputs generated for the surrogate model can mislead other victim models' predictions. Adversarial attacks can be categorized as white-box and black-box, depending on whether the target model can be accessed (e.g., architecture and parameters). Moreover, untargeted attacks aim to cause the model to misclassify inputs regardless of the specific category. Accordingly, targeted attacks aim to cause the model to misclassify inputs into another specific category. Notably, the generative attacks [Poursaeed et al., 2018; Xiao et al., 2018] leverage a trainable generator to directly generate desired perturbations, which are ingeniously similar to current watermarking models and require only a single forward pass in the inference stage, as opposed to instancespecific iterative optimization attacks. For more details on this avenue, the review [Zhao et al., 2023c] is recommended.

3 Motivation & Insight

Motivation. The previous proactive defense method [Wang et al., 2022b] takes the downstream task of passive forensics into consideration, where the attacked unrealistic Deepfake should be effortlessly recognized by the detectors. It is noteworthy that the proactive defense methods [Zhu et al., 2023; Zhang et al., 2024b] also consider proactive forensics, providing the adversarial perturbations with the forensic clue that is capable of provenance tracking. However, the research regarding the correlation between proactive forensics and passive forensics remains substantially unfilled. To study how the watermarks impact Deepfake detectors, we report the detection results of the images watermarked by MBRS [Jia et al., 2021], FaceSigns, and SepMark, which represent robust, semi-fragile, and multipurpose watermarking, respectively. Table 1 shows that the detectors have a clear tendency to predict the watermarked forged images as genuine. To rectify this, a harmless proactive forensics solution is urgently needed, which should concurrently take the downstream Deepfake detectors into consideration, where the watermarked images should be more easily distinguished by the Deepfake detectors, relative to non-watermarked images.

Insight. Although the term "adversarial" leaves an indelible impression of its malicious functionality, adversarial training for benign purposes can effectively boost model robustness. What's more, along with the belief in "adversarial for good" [Al-Maliki et al., 2024], if harnessed in the right manner, the adversarial nature can be utilized to improve image recognition models [Xie et al., 2020], amend the accuracy level of neural models [Yu et al., 2023], boost object detection performance [Sun et al., 2022b], protect watermarks against removal [Liu et al., 2022], prevent automatic painting imitation [Liang et al., 2023] and Deepfake generation [Wang et al., 2022a]. In a similar vein, we delve into robust watermarking to exploit the adversarial vulnerability of Deepfake detectors for good. Note that this paper is not focusing on developing a new robust watermarking model or a new Deepfake detector. Instead, we propose a plug-and-play targeted attack procedure for fine-tuning any robust watermarking to help improve the accuracy of downstream Deepfake detection without tuning the detectors (see Figure 2). Concretely, during our adversarial fine-tuning procedure, only the parameters of watermarking models are updated, and Deepfake detectors keep unchanged. Fine-tuning robust watermarking into adversarial watermarking involves several challenges and key factors. First, it's crucial that the detector attack exhibits efficacy in both white- and black-box scenarios, with adversarial transferability occupying a pivotal position. Secondly, the watermark extraction should still be conducted successfully to enable provenance tracking. Lastly, the visual quality of the watermarked images should remain pleasing.

4 Methodology

4.1 ProblemFormulation

Adversarial Perturbations Harm Detectors. Let x denote the clean host image, and y denote its ground-truth label. We use ˆx to represent the adversarial image, i.e., ˆx = x + η, where η is the imperceptible perturbation with the Lp-norm constraint ‖η‖p ≤ ϵ, p ∈ {0, 2,∞}. Suppose there is a perturbation generator P(·) generating the perturbations directly, i.e., ˆx = x + P(x), and a Deepfake detector D(·) predicting the label of the input. We recall the vanilla adversarial attack, where the objective is to mislead the detector into making an incorrect prediction by

where F denotes the commonly used cross-entropy loss or binary cross-entropy loss for the task of Deepfake detection. It should be noted that the perturbations generated by P(·) are typically constrained by a small L∞-norm budget, to maintain the lower bound of visual quality but may not align well with human perception, with respect to perturbed images.

Adversarial Watermarks Help Detectors. Given a watermark encoder En(·) with the input of the host image x and watermark w, we can obtain the watermarked image xw, i.e., xw = En(x, w). A corresponding watermark decoder De(·) should be able to extract the watermark w exactly from the image xw. Nevertheless, considering that the watermarked image is sometimes delivered and distributed through the dirty channel, it is proposed to introduce a noise layer N(·) to distort the image xw with diverse data augmentations, where the distorted image ˜ xw = N(xw). In this way, the robust decoder can extract an approximate watermark from the image ˜ xw, i.e., ˜ w = De(˜ xw), meaning ˜ w ≈ w. Formally, the objective of our adversarial watermarking, which helps the detector classify watermarked images, can be formulated as

where G denotes the loss function used for image similarity measures (e.g., mean-squared error loss and LPIPS loss [Zhang et al., 2018]), and H represents the mean-squared error loss or binary cross-entropy loss, which usually characterizes the watermark extraction as either a regression or a classification problem, respectively.

4.2 AdvMark Framework

As illustrated in Figure 3, the pipeline of our proposed AdvMark consists of five components: Watermark Encoder, Noise Layer, Watermark Decoder, Adversary Discriminator and downstream Deepfake Detector. Specifically, we showcase the functionalities of each component as follows:

1. The encoder En receives a batch of the host image x ∈ RB×3×H×W (which belongs to either genuine or forged, labeled by y ∈ {0, 1}B×1 ), and random watermark bits w ∈ {0, 1}B×L as input. It produces the watermarked image xw, which has intriguing adversarial properties for helping the downstream Deepfake detectors.
2. The noise layer N augments the watermarked image xw with various simulated or real distortions, resulting in the distorted image ˜ xw. It is parameterless but differentiable, which connects the encoder En and decoder De to facilitate end-to-end robust training.

Figure 3: Overview of our proposed AdvMark. The training process consists of two stages. In Stage I, the encoder and decoder are jointly trained end-to-end, obtaining the pre-trained encoder and decoder that serve as robust watermarking. In Stage II, the encoder and decoder are fine-tuned end-to-end by fooling a surrogate Deepfake detector, aiming at transforming robust watermarking into adversarial watermarking. During inference, only the final watermark encoder and decoder will be adopted.

1. The decoder De extracts the watermark ˜ w from the distorted image ˜ xw, where ˜ w ≈ w, to reliably track the provenance of watermarked image xw. Note that it also participates in updating the encoder En to generate the image that is more conducive to watermark extraction.
2. The adversary discriminator Ad attempts to distinguish between the host image x and the watermarked image xw. Conversely, the encoder En should mislead the discriminator so that Ad cannot distinguish xw from x, to acquire better visual quality of the watermarked image.
3. The surrogate detector D classifies the watermarked image xw as genuine or forged. In the case of D(x) ̸= y, the watermarked image generated by the encoder En makes sense when D(xw) = y, reflecting the enhanced forensic detectability. This watermarked image is even easier to be detected by other downstream detectors.

Below we will provide a detailed description of the pretraining stage of robust watermarking, the fine-tuning stage of adversarial watermarking, and the inference stage of final watermark encoder and decoder in turn.

Encoder-Decoder Robust Training. For the watermark encoder En, its watermarked image xw should be visually similar to the original host image x:

where θ denotes the trainable parameters of the encoder.

For the watermark decoder De, its extracted watermark ˜ w should be identical to the original embedded watermark w, from the distorted image ˜ xw augmented by noise layer N:

where

and φ represents the trainable parameters of the decoder.

For the adversary discriminator Ad, there are several loss variants for stabilizing the GAN training [JolicoeurMartineau, 2019], and for ease of presentation, here we describe the standard GAN loss [Goodfellow et al., 2014]:

where γ indicates the trainable parameters of the discriminator. Note that the discriminator is trained alternately with the encoder-decoder, where the encoder En is also updated by

To sum up, the total loss for Stage I can be formulated by

where λ1, λ2, λ3 are the weights for respective loss terms. After training, we get the pre-trained encoder EnI and decoder DeI. We can also freely adopt the released models 1,2 as the pre-trained encoder and decoder for this stage. More implementation details can be found in Section 5.2.

Adversarial Fine-tuning Procedure. The advent of highly realistic generated content can easily deceive human eyes. Manually examining a large volume of images through observation is extremely expensive, even when the defects are obvious. Therefore, it is preferable to develop automatic Deepfake detectors on the machine end. However, with the proliferation of generated images, merely verifying their authenticity seems insufficient without understanding their provenance, since it's difficult to differentiate between images created for malicious intents and those created for neutral or positive purposes. Building upon the watermarking at Stage I, a responsible individual or social network can embed the provenance evidence into the published images using his/her watermark encoder EnI. Unfortunately, the proactive injection will harm the passive detectors, which are more prevalent and widely deployed in the wild than the solely watermark decoder DeI. This is indeed contrary to the belief in AI responsibility and does a disservice. In light of the above, we propose a plug-and-play targeted attack procedure that fine-tunes robust watermarking into adversarial watermarking, aiming to help improve the accuracy of downstream Deepfake detection by deliberately fooling the passive detectors (i.e., crossing the decision boundary; see Figure 2).

To achieve the fooling objective, we freeze a surrogate detector D and force the detector to predict the correct label y with respect to the watermarked image xw by updating EnI :

where F denotes the loss function (e.g., binary cross-entropy loss) used to train the original detector D.

To summarize, the total loss for Stage II is formulated by

where λ4 is the weight for the fooling loss term. After finetuning, we get the final encoder EnII and decoder DeII . Detecting the watermarked images amended by EnII will help improve detection accuracy without tuning the detectors.

1 https://github.com/jzyustc/MBRS

2 https://github.com/sh1newu/SepMark

Model Inference. In the inference process, only the well-trained watermark encoder EnII and decoder DeII are adopted. Consequently, depending on our adversarial watermarks, we can categorize the harmless proactive forensics into the following cases.

Case 1: If ˜ w ̸≈ w, the suspicious image is either not watermarked by the encoder EnII at all or has been extremely altered by significant distortions, reflecting that it is impossible to track its provenance by the decoder DeII .

Case 2: If ˜ w ≈ w, the image has been watermarked by the encoder EnII, and we can extract the watermark ˜ w to serve as evidence of the image's provenance, and effortlessly determine whether it is genuine or forged by the detector D, which we refer to as enhanced forensic detectability.

Case 3: Even in a more challenging setting where the detector D′ was unseen during the training process, the watermarked image xw is also more easily distinguished by D′ , which is known as adversarial transferability.

5 Experiments

5.1 Dataset Preparation

We collect the real faces sourced from CelebA-HQ [Karras et al., 2018] and resize them to the resolution of 256 × 256. These real faces are then manipulated by recent Deepfake generative models: SimSwap [Chen et al., 2020] for face swapping, FOMM [Siarohin et al., 2019] for expression reenactment, and StarGAN [Choi et al., 2018] for attribute editing. We further utilize the entire synthesized faces provided by StyleGAN [Karras et al., 2019], which are also resized to 256 × 256. These four fake categories are evenly sampled to balance the numbers of real and fake faces. More specifically, there are equal numbers of faces in real and fake subsets, and we divide them into training, validation, and testing, respectively, referencing the official split 24183/2993/2824.

5.2 Implementation Details.

As the proposed work is not focusing on developing a new robust watermarking model, we directly adopt MBRS and the robust branch of SepMark as the pre-trained encoder and decoder in Stage I, and also as our baselines. To show that the performance gain is due to our adversarial fine-tuning procedure rather than training for more iterations, we establish additional baselines where the two watermarking models undergo a vanilla fine-tuning procedure (without fooling loss) on the dataset. Moreover, we utilize nine well-trained Deepfake detectors, as described in Section 2.1, to validate the capability of forensic detectability and adversarial transferability. For the hyper-parameter settings of MBRS and SepMark, we strictly follow their original implementations. For example, besides the difference in the length of the watermark bits (256-bit for MBRS and 128-bit for SepMark), the noise layers they use are also inconsistent. To be specific, MBRS includes Identity, JPEG, and simulated JPEG, while SepMark contains all the noises listed in Table 3. Therefore, we will compare our AdvMark with the respective baselines for each to mitigate the model bias introduced by different backbones. Lastly, the whole fine-tuning lasted for 10 epochs with a batch size of 8, and we set the weight of the fooling loss λ4 to 0.1.

5.3 Detector Attack

The benign functionality of adversarial watermarking is confirmed in both white- and black-box scenarios. The evaluation metric here is the accuracy (ACC) of the detector D:

where

and y ∈ {0, 1}B×1 means the ground-truth label, | · | denotes the absolute value. For a comprehensive evaluation, we report the Real/Fake ACC on the separate real and fake subsets. The higher the value, the better the detection performance.

White-box Attack. In the white-box scenario, we evaluate the "training accuracy" with the detector seen during training. When the watermark encoder-decoder and Deepfake detector are in the same camp, i.e., the detector is already known by the watermarking developer, the surrogate detector used can be consistent with the downstream detector. By observing Table 2, we note that the white-box detector without any tuning can get significant performance gain in detecting the images watermarked by AdvMark. The baselines MBRS and SepMark are similar to the image processing operations, e.g., JPEG compression and Gaussian noise, which are harmful to the detector, where they distort the host image in different fashions. The fine-tuned baselines also have inferior detection performance compared to the clean host images without watermarking. By contrast, our helpful AdvMark turns the watermark from an enemy into a friend, ensuring nearly 100% accuracy for the detector in the white-box setting. This is also the case when several detectors, e.g., Xception, MultiAtt, and RFM, are included for the adversarial fine-tuning.

Black-box Attack. We further evaluate the "testing accuracy" with the detector unseen during training, in the blackbox scenario. Table 2 shows that compared to the clean host images, the watermarked images using AdvMark are more easily distinguished by other unseen detectors, thanks to the transferability of adversarial watermarks. For example, using MultiAtt or SBI as the surrogate detector, the resulting detection performance with AdvMark is higher than that without watermarking, on average. We notice that AdvMark is inferior to the baselines on a few unseen detectors, which may be due to the different decision logic shared by the detectors. E.g., most detectors mainly identify distinct fake patterns in the images, while the exception RECCE seeks to discriminate what is real. Since AdvMark performs quite well in the whitebox setting, we speculate that it has the potential to overfit to the seen detector, but on some unseen detectors it may have ruined the detection. Suggested by ensemble attacks [Poursaeed et al., 2018], we can attack an ensemble of detectors, which is more conducive in the real world. By simultaneously attacking Xception, MultiAtt, and RFM, in the black-box setting we obtain an overall improved detection performance, which is indeed better than attacking only one detector.

5.4 Watermark Extraction &Visual Quality

Since our AdvMark focuses on improving the forensic detectability of watermarked images without compromising provenance tracking, we expect as few performance degradations in Bit Error Rate (BER) as possible. Suppose that the embedded watermark w ∈ {0, 1}B×L and the extracted watermark ˜ w ∈ {0, 1}B×L, formally,

A lower BER indicates stronger robustness in watermark extraction. From Table 3, we can see that the BER of AdvMark is comparable to the baselines, and the performance gaps are almost negligible, regardless of the surrogate detector used. Interestingly, the adversarially fine-tuned MBRS enhances the robustness against unseen distortions, e.g., salt pepper and Gaussian noise, which is likely due to the redun- Without training the watermarking model from scratch, AdvMark can serve as a plug-and-play procedure, seamlessly integrating with existing watermarking. By fine-tuning robust watermarking into adversarial watermarking, we do not harm the performance of passive detectors while fulfilling proactive forensics, achieving harmless provenance tracking and concurrently enhancing forensic detectability. We also investigate the transferability of adversarial watermarks, allowing AdvMark to generalize the watermarking to unseen Deepfake detectors, further validating its effectiveness in the wild. In our future work, we plan to conduct more analyses of the watermark effects on other downstream tasks, such as tamper localization [Zhang et al., 2024a].

Acknowledgments

This work is supported by National Natural Science Foundation of China (Grant Nos. U22A2030, U20A20174), National Key R&D Program of China (Grant No. 2022YFB3103500), Hunan Provincial Funds for Distinguished Young Scholars (Grant No. 2024JJ2025). Supplementary material, including more details and experiments of our work, can be found in the supplement at https://github.com/sh1newu/AdvMark.