DcDsDiff: Dual-Conditional and Dual-Stream Diffusion Model for Generative Image Tampering Localization

1 Introduction

Diffusion-based image generation technology is widely noted for its superior quality in image creation. For example, Stable Diffusion's [Rombach et al., 2022] inpainting function allows users to re-create specific parts of an image, enabling a rich variety of visual expressions and innovative effects. However, this technological advancement also presents challenges in information security. As image generation becomes more convincing, it's easier than ever to create deceptive image tampering. As shown in Figure 1, malicious actors could use these technologies to produce fake images, spread misinformation, and destabilize public opinion and social stability. Thus, detecting and localizing Generative Image Tampering (GIT) is becoming increasingly critical.

∗Corresponding Author

Current mainstream Image Tampering Localization (ITL) methods, while effective in ITL tasks of Splicing, CopyMove, or Removal, struggle with handling GIT. GIT is not limited to a single tampering form. Its diversity in purpose (e.g., removal, filling, replacing [Ju et al., 2024]), generated object types, and mechanisms (e.g., example-guided [Yang et al., 2023], context-aware [Zhuang et al., 2025], text-guided [Ju et al., 2024]) demands strong generalization capabilities of forensic models. Most ITL methods are built on conventional encoder-decoder architectures, following a linear information processing mechanism, lacking feedback loops or intermediate adjustments to optimize predictions. This fixed process lacks flexibility and cannot adapt to diverse tampering forms and complex scenes. Additionally, the generated objects in GIT are highly integrated with the background, making the edges of the tampered areas and nearby pixels highly uncertain. Conventional ITL methods often misidentify these edges due to over-confidence in linear processing, which hinders the generation of accurate and complete localization results.

In response to the aforementioned issues, we propose a Dual-Conditional and Dual-Stream Diffusion Model (DcDsDiff), as shown in Figure 2. Specifically, we introduce a diffusion model strategy for Generative Image Tampering Localization (GITL) task, redefining GITL as a diffusion model-based bi-generation task, achieving joint generation of mask image (tampered area) and detail image (edge and surrounding pixel). Firstly, the model includes a parallel Dual-Stream Denoising Network (DSDN) that refines predictions through iterative denoising. This iterative process enhances DcDsDiff's understanding of data distributions, allowing it to better generalize to diverse tampering forms. It also enables the model to fine-tune predictions based on image conditions, thereby helping the model to make more cautious predictions in edge and detail areas, thus avoiding overconfidence. To achieve mutual promotion between the two streams, we introduced a Multi-Stream Interaction Enhancement (MSIE) module. This module uses detail features to guide the prediction of the mask image, improving the mask image stream's perception of detail areas. Simultaneously, it utilizes mask features to provide overall positional information for the detail image, thereby improving detail image's completeness. Secondly, DcDsDiff also includes a Dual- View Conditional Network (DVCN) that provides clues to the DSDN. The RGB conditional network processes spatial information, while the high-frequency conditional network captures tampering traces in the high-frequency view, which are invisible in the RGB view. This high-frequency view is generated by the High-Frequency View Generator (HFVG) to reflect local anomalies caused by GIT. Additionally, we designed a Multi-Modal and Multi-Scale Feature Fusion (MMMSFF) module. This module aligns features of both modalities spatially, re-calibrates features at the channel level, integrates features of the same level based on their importance, combines features from different levels to provide the DSDN with the necessary fused conditional feature.

To sum up, our main contributions are as follows:

• A diffusion-based DcDsDiff model was proposed, introduced a DSDN to simultaneously generate mask image and detail image, and designed a MSIE to enhance information complementarity between the two streams.
• Designed a DVCN to extract tampering features from multiple views, introduced an HFVG to capture highfrequency anomalies, and designed a MM-MSFF to integrate multi-modal and multi-scale features.

• Extensive experiments demonstrate that DcDsDiff has significant advantages in terms of localization performance, generalization, extensibility, and robustness.

2 RelatedWork

2.1 ITL

We categorize mainstream ITL methods into four major categories: traditional methods [Ren et al., 2023; Liu et al., 2022], noise view-assisted methods [Niloy et al., 2023; Dong et al., 2022; Lin et al., 2023; Wu et al., 2019; Ji et al., 2023; Wu and Zhou, 2021; Frick and Steinebach, 2024], frequency domain view-assisted methods [Frick and Steinebach, 2024; Wang et al., 2022a; Liu et al., 2023], and edgeassisted multi-task methods[Shi et al., 2023; Ren et al., 2024; Hao et al., 2024c; Hao et al., 2024a; Dong et al., 2022; Wang et al., 2025]. Traditional methods primarily rely on the attribute differences between tampered and untampered regions in spatial domain features. Noise view/frequency domain-assisted methods aim to detect tampering traces in noise view/frequency domain that are invisible in the RGB view. However, poorly designed noise view/frequency domain generators often introduce redundant information, leading to adverse interference in localization results. Edgeassisted methods aim to capture edge inconsistencies caused by tampering. However, GIT lacks distinct boundaries. An overly confident edge prior can often mislead the model into producing erroneous localization results.

2.2 DiffusionModels for Image Segmentation

Currently, the application of diffusion models in the field of image segmentation is relatively limited. BerDiff [Chen et al., 2023] is a conditional Bernoulli diffusion model for medical image segmentation, generating accurate and diverse segmentation masks through Bernoulli noise and multiple samplings. CamoDiffusion [Chen et al., 2024] utilizes the denoising mechanism to gradually reduce the differences between initial noise and the true mask, effectively improving the detection performance of camouflaged objects. Additionally, BiDiCOS [Jiang et al., 2024] combines bilateral diffusion models with depth estimation techniques, aiming to optimize the segmentation process by fusing depth information, enhancing the accuracy and robustness of camouflaged object detection.

3 Proposed DcDsDiff

3.1 Background

In this paper, we propose a dual-stream diffusion model based on denoising diffusion probabilistic models, which generates mask image and detail image simultaneously. As shown in Figure 3, the diffusion model is mainly divided into two stages: the forward process and the backward process. In the forward process, we start with noise-free original mask image M0 and original detail image E0, iteratively adding Gaussian noise until the diffusion step T is large enough that the original mask image and detail image completely degrade into noise. This process can be described by the following Markov process:

where, t runs from 1 to T , and variance is controlled by noise schedule βt ∈ (0, 1). I is an identity matrix. Mt and Et represent the mask image and detail image at time step t, respectively, and can be obtained by the following equations:

where, αt = ∏Ti=1 αt, αt = 1 - βt. The backward process is the reverse of the forward process. The model starts from two noisy images, gradually denoises them, and eventually restores the clean original images.

where, θ refers to the model parameters. µθ(Mt, t) and µθ(Et, t) refer to the means predicted by the model at a given time step t. ∑θ (Mt, t) and ∑θ (Et, t) refer to the predicted covariances. ∑θ (Mt, t) and ∑θ (Et, t) are set to σ2 t = 1-at-1 1-at βt. µθ(Mt, t) and µθ(Et, t) can be represented as:

where, ˆM0 and ˆE0 refer to the model's predicted mask image and detail image.

3.2 Overview

The architecture of DcDsDiff is shown in Figure 2. In training, the input are an RGB image I with the size of W×H×3, the GT of the mask image M0, and the GT of the detail image E0. The output are the predicted mask image ˆM0 and detail image ˆE0. Specifically, firstly, we use the HFVG to generate the high-frequency view of I, and then input I and high-frequency view A into the DVCN. In DVCN, we employ two pre-trained PVTv2 [Wang et al., 2022b] models as the backbones. These two PVTv2 models can respectively generate spatial feature set r1, r2, r3, r4 and high-frequency feature set a1, a2, a3, a4 of different levels. We utilize the MMMSFF to fuse the multi-modal and multi-scale features r1-4 and a1-4 to obtain the conditional feature c. Secondly, the mask image M0 and the detail image E0 undergo the noise addition process to generate noisy images Mt and Et at time step t. Subsequently, we use the DSDN to denoise the noisy images Mt and Et under the guidance of c, and leverage the MSIE to achieve information complementarity between the two streams. Finally, the losses of the predicted mask image ˆM0 and detail image ˆE0 are calculated. During the inference stage, the input is the GIT image, and the output is the localization results of the tampered areas. Specifically, firstly, we input the GIT image into the trained DcDsDiff model and repeat the operations of the DVCN from the training phase. Secondly, we initialize two Gaussian noise images MT and ET as inputs to the DSDN. Through iterative denoising by the DSDN, we gradually recover the detail image ˆE0 and the mask image ˆM0 from the noise images. Finally, we average the multiple predicted images generated during the iteration process to obtain the final predicted mask image ˆM f 0 and detail image ˆEf 0 .

3.3 HFVG

As shown in Figure 4, GIT can cause local anomalies visible in the high-frequency view, which are difficult to capture in the RGB view. Therefore, we designed the HFVG to provide supplementary clues for RGB conditional network. The workflow of the HFVG is shown in Figure 2. Specifically, first, we perform a 2D Fast Fourier Transform (2D FFT) on the RGB image I along the channels. Then, we obtain the high-frequency components of the three channels through a high-pass filter. Next, we convert these high-frequency components back to the RGB domain by performing an inverse 2D Fast Fourier Transform (2D IFFT) and merging them. Finally, to further highlight the local anomalies in the highfrequency view, we enhance the values of this high-frequency view, resulting in the final high-frequency view A.

where, ⊗ refers to the multiplication operation. FFT() and IFFT() refer to the 2D FFT operation and the 2D IFFT operation, respectively. HPF() denotes the high pass filter and ς is the manually designed threshold which controls the low frequency component to be filtered out.

3.4 MM-MSFF

The MM-MSFF consists of two parts: Multi-Modal Feature Fusion (MMFF) and Multi-Scale Feature Fusion (MSFF). MMFF aims to enhance and fuse spatial features and highfrequency features at the same level. Specifically, firstly, the positions of the tampered areas in the features of both modalities should be the same. Therefore, we use Spatial Attention (SA) to obtain their shared spatial attention map and align the features of both modalities using this spatial attention map.

where, SA() refers to SA, which is an operation sequence composed of a Global Max Pooling operation (GMP) along the channel, a convolution operation with the kernel size of 3×3 (3×3 Conv), and a Sigmoid function [Liu et al., 2021]. Secondly, high-frequency features mainly contain rich tampering features, while spatial features primarily contain rich texture and appearance information, with different focuses. Therefore, we use Channel Attention (CA) to further highlight the important information in both modalities.

where, CA() refers to CA, which is an operation sequence composed of a GMP, a 1 × 1 Conv, a ReLU function, and a Sigmoid function. Lastly, the importance of features from the two modalities varies when processing different images. Ignoring their different contributions and directly fusing them can lead to a decrease in model performance. We consider dynamically allocating the weights of the two features when fusing them.

where, Sig() refers to Sigmoid function, C3×3 refers to 3×3 Standard convolution (3 × 3 CBR), ⊕ refers to the addition operation. High-level features contain richer semantic information, which is beneficial for the model to determine the approximate location of the tampered area. Low-level features contain more detailed information, which is beneficial for the model to generate more refined localization results. Therefore, in MSFF, we use a concatenation operation to fuse features from different levels to obtain c = Cat(f1, f2, f3, f4), where, Cat refers to the concatenation operation. c will be input into the DSDN.

3.5 DSDN

In ITL tasks, accurately locating the edges of tampered areas is crucial, as they often reveal the overall contour and local details of the tampered area. However, GIT makes the tampered area blend seamlessly with the background, leaving minimal traces. Edge pixels are few and highly similar to surrounding pixels, making direct edge prediction challenging. In contrast, identifying detail areas of the tampered region is easier. These areas include not only edge pixels but also nearby pixels. The values of surrounding pixels, determined by distance transformation [Wei et al., 2020b], are inversely proportional to the distance from the nearest edge, with higher values for pixels closer to the edge and lower values for those farther away. Compared to methods relying solely on edge pixels, detail maps include more pixels, facilitating a more balanced pixel distribution.

Traditionally, Denoising network aims to decode the denoisedmask predictions ˆM0 and ˆMt-1 based on the diffusion paradigm. However, the DSDN aims to achieve joint prediction of the mask image and detail image through two parallel UNets (MIB and DIB). As shown in Figure 2, specifically, first, we use multiple convolution layers to encode the two input noisy images into two feature maps of the same size as the conditional feature c. Then, we fuse these two feature maps with c through a simple concatenation operation. Finally, we use multiple convolutions and up-sampling operations to gradually restore the size of the two feature maps and obtain the mask feature dM and detail feature dE . We adopted adaptive group normalization [Chen et al., 2024] in both UNets, incorporating the time step information t into the convolutions, making our DSDN sensitive to changes in the time step.

MIB and DIB are not independent of each other; their interaction is of significant importance. The mask image stream provides semantic information about the tampered area, while the detail image stream provides contour and detail information of the tampered area. Effective interaction between the two can enable the model to generate localization results with rich detail information and high integrity. Inspired by [Qian et al., 2020], we designed the MSIE module to implement a cross-attention mechanism, as shown in Figure 2. Specifically, we first extract the Query features and Key features of the mask features and detail features.

where, ∗ denotes the transpose operation. Then, we use the Query features and Key features to compute the weights.

where, SoftMax() refers to SoftMax function. Finally, we use the cross-attention weights to enhance the features of both streams.

where, γ and υ are learnable parameters used to adjust the intensity of attention. pM and pE will be used respectively to predict the mask image ˆM0 and the detail image ˆE0.

3.6 Loss Function

In terms of loss functions, we utilize a combination of Weighted Binary Cross-Entropy (WBCE) and Weighted Intersection over Union (WIoU) loss [Wei et al., 2020a] for the predicted mask image ˆM0. For the detail image ˆE0, the average of L1 loss and L2 loss is applied.

where, LWBCE+WIOU refers to the combination of WIoU loss and WBCE loss, LL1 refers to the L1 loss, and LL2 refers to the L2 loss.

4 Experiments

4.1 Experiment Settings

In this paper, we implemented DcDsDiff using the PyTorch framework. All experiments were conducted on a single NVIDIA GeForce RTX 4090 GPU. Input images were resized to 352 × 352 and augmented via random horizontal flipping. During the training phase, we used the AdamW optimizer with the learning rate of 0.001 and the batch size of 6. ς was set to 0.5. We employed a Signal-to-Noise Ratio (SNR)-based variance schedule [Hoogeboom et al., 2023] to adjust the SNR of the diffusion process. The model was trained for 100 epochs. During the inference phase, the model undergoes ten iterative steps. For each step of sampling, we used a nonlinear method to select the value of t [Jiang et al., 2024]. In terms of datasets, firstly, we constructed a GIT10K dataset containing 10,000 images using four common diffusion-based local inpainting methods: Brush Net (BN) [Ju et al., 2024], Paint by Example (PE) [Yang et al., 2023], Inpaint Anything (IA) [Yu et al., 2023], and Power Paint (PP) [Zhuang et al., 2025], with each method contributing 2,500 images. Secondly, to comprehensively evaluate DcDsDiff, we tested its performance on datasets containing other tampering types, including RLS [Hao et al., 2024c], IMD [Novozamsky et al., 2020], Nist16 [Guan et al., 2016], DEFACTO Splicing (DEF) [Mahfoudi et al., 2019], and AutoSplice (AUTO) [Jia et al., 2023]. Finally, we divided the train and test sets of the aforementioned datasets in a 9:1 ratio. For evaluation metrics, we chose the F1-Score (F1) and Intersection over Union (IoU) to assess the performance of DcDsDiff.

4.2 Ablation Study

In this experiment, we conducted a detailed ablation study on DcDsDiff using the GIT10K dataset, which includes four sub-test sets: BN, PE, IA, and PP, each containing 250

tampered images. We designed four experimental schemes, with their configurations and results detailed in Table 1. Firstly, the introduction of the DIB and MSIE significantly enhanced DcDsDiff's performance, particularly for the PE sub-test set. Secondly, incorporating HFVG further improved performance based on Scheme 2, especially for the PE, IA, and PP sub-test sets, highlighting the importance of capturing tampering traces in the high-frequency view. Lastly, the introduction of MM-MSFF effectively fused the features of high-frequency view and RGB view. Compared to Scheme 3, Scheme 4 demonstrated notable improvements in F1 and IoU on BN, PE, and IA sub-test sets.

4.3 Comparison with State-of-the-Art Methods

In this experiment, we compared DcDsDiff with five ITL methods (MVSS-Net [Dong et al., 2022], MFI-Net [Ren et al., 2023], TA-Net [Shi et al., 2023], CFL-Net [Niloy et al., 2023], EMF-Net [Ren et al., 2024]), UGEE-Net [Hao et al., 2024b] and EC-Net [Hao et al., 2024c] using the GIT10K dataset, with results shown in Table 2. The results indicate that DcDsDiff outperformed these ITL methods across all test sets. Notably, on the PE and IA test sets, DcDsDiff significantly exceeded the performance of the second-ranked EMFNet, underscoring its superior localization capabilities. Figure 5 illustrates the localization results of these eight methods. Compared to the other seven methods, DcDsDiff provides more complete and detailed localization results. This advantage is primarily due to our detail image generation stream. Figure 6 illustrates the detail images generated by DcDsDiff and the edge images produced by five edge-assisted methods. As shown in Figure 6, when processing tampered areas that blend into the background, MVSS-Net and TA-Net struggle to generate complete edge images, while EMF-Net, EC-Net and UGEE-Net incorrectly identify non-edge pixels as edge pixels due to overconfidence. In contrast, DcDsDiff successfully avoids these issues, further validating the effectiveness of the detail image generation stream. More visualizations can be found in the supplementary material.

4.4 Cross-Generator Generalization Capability

In this experiment, we adopted a cross-dataset testing strategy to evaluate the generalization capabilities of DcDsDiff and five other methods. The models were trained on the subtrain sets generated by specific GIT methods and tested on the sub-test sets generated by other GIT methods. The results are shown in Table 3. As seen in Table 3, DcDsDiff's performance is generally much better than the other five methods. The iterative denoising strategy of DcDsDiff enhances the model's understanding of data distributions, reduces overreliance on labeled data, and improves the model's generalization capability.

4.5 Extensibility Evaluation

In this experiment, we evaluated the extensibility of DcDsDiff using the RLS, IMD, Nist16, DEF, and AUTO datasets.

IMD consists of real-life manipulated images. RLS and Nist16 include three types of tampering: Splicing, CopyMove, and Removal. DEF contains synthetic splicing images. And AUTO is an AIGC dataset generated by the DALL-E2 model. The results are presented in Table 4. All models are separately trained and tested on each dataset without crossdatasetmixing. Although DcDsDiff performs slightly worse than CFL-Net on the DEF test images, it significantly outperforms the other five methods on the RLS, IMD, Nist16, and AUTO test sets. This experiment demonstrated that DcDsDiff is a versatile ITL model.

4.6 Robustness Evaluation

In this experiment, we individually applied Gaussian noise and gamma correction attacks to the mixed test set of BN, PE, IA, and PP sub-test sets to assess the stability of DcDsDiff. We then compared its performance with five mainstream methods. These models were trained on the mixed train set of GIT10K. As illustrated in Figure 7, Gaussian noise does indeed interfere with GITL. As the intensity of the standard deviation increases, the performance of all models shows a significant decline. Compared to the other five models, DcDsDiff consistently demonstrates the best performance. This proves that DcDsDiff has a certain degree of robustness against Gaussian noise. Gamma correction also affects the model's localization performance. When the gamma value is below or above 1.0, all models show varying degrees of performance degradation. However, DcDsDiff still exhibits the best performance.

5 Conclusion

DcDsDiff demonstrates superiority in several aspects: Firstly, it introduces the strategy of DSDN to synchronously generate mask images and detail images, enhancing the model's generalization capability. Secondly, through MSIE, it achieves information complementarity, generating localization results with rich details. Thirdly, DVCN and HFVG effectively capture tampering features, while the MM-MSFF module strengthens feature fusion. Extensive experiments have proven the superiority of DcDsDiff compared to traditional ITL methods.

Acknowledgments

This work is supported by National Key Research and Development Program of China [grant number 2024YFF0907404].