Latents-Inv:Robust Semantic Watermark via Dual-Path Mutual Information Redundancy for Diffusion Models

1 Introduction

Digital watermark improves the security of content generated by powerful cross-modal vision models[Xian et al., 2024; Song et al., 2024; Lei et al., 2024; Zhang et al., 2024b]. As users increasingly train personalized models to produce content[Rombach et al., 2022; Ho et al., 2020; Xu et al.,

∗Corresponding author

2024], open customization raises trust concerns, including model theft, copyright disputes, and fake generation[Zhang et al., 2024c]. Watermark addresses these issues by protecting model copyright and embedding traceable identifiers into outputs[Bao et al., 2024; Pan et al., 2024; Min et al., 2024; Feng et al., 2024; Chen et al., 2024], enabling source verification and detection of unauthorized content.

For generative models of visual content, watermark information can be embedded by marking the model's outputs with a secret message[Chen et al., 2024; Pan et al., 2024; Trias et al., 2024]. Currently, as shown in Figure 1, most current watermarking methods for diffusion models embed verifiable information during the generation process, at specific stages of image creation. These methods include fine-tuning parts of the model[Rezaei et al., 2024; Hu et al., 2024a] and watermarking the initial noise before denoising. The former method typically operates by fine-tuning specific components of the model architecture. It has poor generalization that watermarking methods for UNet-based models often do not transfer well to MMDiT-based models. The later semantic watermarking methods rely on accurate inversion from image to latent noise. However, DiT-based rectified flow models employ a first-order Euler-like solver that lacks stable and precise inversion, limiting the cross-architecture transferability of these watermarking approaches. Moreover, existing watermarking methods show poor robustness against adversarial attacks [Yang et al., 2024a], where watermarks can be easily damaged by difficult image transformations, such as rotation or erased by proxy-model-based deep perturbation[M¨uller et al., 2025]. When a watermarked image undergoes severe distortions such as large-scale cropping, inevitable information loss occurs, causing the extracted watermark to fall below the verification confidence threshold and thereby invalidating authentication. Additionally, attackers can exploit proxy models to perform optimizationbased learning on watermarks. By iteratively refining a single watermarked image, they may uncover the fixed structural patterns of deterministic schemes like Tree-Ring[Wen et al., 2023] or the stochastic sampling paradigms of probabilistic methods like Gaussian Shading[Yang et al., 2024b], enabling effective watermark removal or forgery and ultimately undermining verification reliability.

Therefore, as shown in Figure 3, we propose Latents-Inv, an in-processing watermarking method that embeds information directly into latent representations. This design is theoretically well-suited for plug-and-play compatibility across diverse diffusion models. Our approach employs a flowbased model with a bidirectional structure to enable reversible watermark embedding and accurate extraction. The dual-path architecture splits the watermark payload: the primary path conveys watermarked initial latents, and the auxiliary path embeds the copyright owner's key with redundant watermark information. During training, we adopt a joint-training strategy that leverages negative-sample pairs under both accuracy and fidelity constraints. This ensures that watermark verification does not overly rely on mutual information from a single path. Additionally, we train a coarse-grained decoder with the same architecture to support key-free watermark extraction, thereby fully exploiting the carrier image's frequencydomain capacity during encoding. To address precision limitations in DiT-based rectified flow models, our method predicts backward-step noise from forward-step predictions and uses this estimate to refine the forward trajectory, as detailed in Section 3.2. Experimental results show that Latents-Inv maintains strong ownership identification even under attack: by leveraging the owner's key to recover corrupted watermark components, it enables fine-grained mutual information verification with sufficient accuracy, ensuring robust extraction under various disturbances.

In summary, our work advances robust semantic watermarking for diverse diffusion models by: (i) enabling reliable watermark recovery under distortions through mutual information redundancy between the watermarked latents and the owner's key; (ii) achieving visually imperceptible watermark embedding via a joint training strategy that incorporates negative sample constraints to preserve image fidelity while enhancing verification reliability; and (iii) solving the backward Euler inversion in DiT-based rectified flow models via fixedpoint iteration.

2 RelatedWork

2.1 SemanticWatermarkingMethod

Embedding watermark information into data as traceable metadata is highly beneficial for source verification and copyright protection [Saberi et al., 2023; Liang et al., 2024; Sun et al., 2025]. With the development of deep neural networks, watermarking in diffusion models is increasingly embedded during the image generation process to ensure imperceptibility. In-processing approaches vary across different generative models, but the main strategies fall into two categories: one fine-tunes part of the model structure to embed watermark information during image generation [Feng et al., 2024]; the other embeds the watermark at the initial noise sampling stage of diffusion models, which reduces computational overhead. For example, Tree-Ring [Wen et al., 2023] and Ring-ID [Ci et al., 2024] embed watermark patterns into the Fourier space of latents, while Gaussian Shading [Yang et al., 2024b] and GaussMarker [Li et al., 2025] redefine the sampling space. PRC [Gunn et al., 2024] further introduces error-correcting codes for encrypted protection. WIND [Arabi et al., 2024] and SEAL [Arabi et al., 2025], among others, utilize multi-stage encoding to embed watermark information. However, as shown in Figure 2, these existing methods perform poorly when transferred to new generative models [Umrajkar and Singh, 2025] and are susceptible to black-box optimization attacks targeting individual images [M¨uller et al., 2025]. In this paper, we introduce Latents-Inv to ensure watermark robustness and verification reliability, and we achieve strong semantic watermarking performance in DiT-based models through our designed inversion scheduler.

2.2 Anti-watermarkMethod

Robustness against attacks remains a key challenge for modern semantic watermarks. Early watermark removal methods relied on simple image transformations such as cropping, noise addition, and rotation, which physically alter the image to disrupt invisible watermarks embedded in the spatial or transform domain [Liang et al., 2024]. However, with advances in neural networks and deep learning, such post-processed perturbative watermarks can be easily detected and removed by deep models [Jiang et al., 2023]. Meanwhile, existing in-processing watermarking methods embed information during generation, but they often rely on fixed mathematical patterns in the initial noisy latent, leading to consistent image distributions. This makes the watermark vulnerable to statistical analysis [Zhang et al., 2024a; Yang et al., 2024a] or proxy models [Hu et al., 2024b; M¨uller et al., 2025] that can learn the embedding pattern, enabling watermark removal or forgery. These attacks pose challenges to conventional semantic watermarks. However, our design preserves the watermark distribution and imperceptibly protects copyright information while maintaining robustness and verification reliability under attacks.

3 Preliminaries

3.1 Mutual Information Redundancy

For a complete watermark verification system, traceability assessment is based on observing the verified watermark ̂ W and the original watermark W for judgment. From an information-theoretic viewpoint, it is measured by the amount of mutual information of W provided by ̂ W. Formally, it is the reduction in entropy

whereH(·) denotes Shannon entropy andH(· | ·) conditional entropy.

The mutual-information content quantifies how many bits of uncertainty about W are resolved by observing ̂ W. Thus, the verification condition I(̂ W;W) ≥ τ equivalently requires that the extracted observation must supply at least τ bits of information about the embedded watermark, thereby guaranteeing a minimum mutual-information budget for reliable detection.

In a conventional watermarking system, the watermark information is embeded in a watermarked signal X, which makes I(X;W) = I(̂ W;W) as well as H(X) = H(̂ W ). And X is usually corrupted by an attack channel, yielding X′ . This process can be modeled as the Markov chain

Due to the data processing inequality, any attack reduces the information available for verification, i.e.,

In contrast, we consider a watermark encryption scheme that outputs both the watermarked signal X and an auxiliary variable K. K is not related to model generation, so it will not be released to the public and will be preserved by the copyright owner. :

At the verification stage, the decoder jointly observes (X′ ,K). X′ is verified signal after propagation and secret owner K remains invariant under attacks.

By the chain rule of mutual information, we have

where the inequality follows from the non-negativity of conditional mutual information.

This auxiliary information redundancy improves robustness by enlarging the mutual-information budget available for watermark verification under attacks.

3.2 Inversion in Rectified-FlowDiffusionModel

Besides security concerns about robustness, the lack of crossarchitecture generalizability is another limitation of existing watermarking methods. Here, we demonstrate how to achieve accurate inversion in the MMDit model to map images back to their latent noise. More preliminaries are in Appendix.

For Rectified-Flow scheduler, Zti-1 denotes the latents at timestep ti, and Zti is the latents from the previous step. The term ti-1 - ti represents the time step size, and vθ(Zti , ti) is the velocity network that predicts the flow direction at ti. When given the velocity function v0, query latents Z0, and time steps t = {t0, . . . , tN} where t0 = 0 and tN = 1, the inversion algorithm updates are defined as follows:

Initial conditions:

For i = 1 to N:

Intuitively, because the distributions of initial latents ZN and final output ̂ ZN are similar, the parallel noise distribution errors on the same time step will be smaller. Theoretically, this inversion corresponds to an iterative generation method that implements the backward Euler ODE solver, where each step is solved via fixed-point iteration. The local truncation error of this scheme is O(∆t2i ), with ∆ti = ti - ti-1.

4 Method

4.1 Overview

As shown in Figure 3, our method employed a flow-based codec to embed watermark information into two outputs: the carrier latents and a owner's key. The carrier latents are used for image generation, and after social dissemination, image are reversed by the copyright holders through the corresponding inversion pipeline. In the extraction phase, fine-grained and high-precision watermark recovery is achieved with the mutual information of the key, while a coarse-grained decoder could verify the watermark with carrier latents independently. Next, we present a detailed description of the codec operational mechanism and training strategy.

4.2 Latents Processing and Flow-based Codec

The flow-based model is inherently reversible: for every forward encoding function fθ, there exists a corresponding backward function f -1 θ . This one-to-one invertibility naturally aligns with the watermarking pipeline, where embedding and extraction are symmetric processes[Ma et al., 2022; Fang et al., 2023].

Before any forward or backward pass through the flowbased model (Figure 3), the input latents is pre-processed by Latents-Inv using a Haar transform to map it into the frequency domain. Embedding the watermark multiple times in this transformed domain better exploits channel capacity and enhances robustness without degrading generation quality [Wen et al., 2023; Kassis and Hengartner, 2025].

In the embedding stage, Latents-Inv accepts the watermark information w0 and the carrier information x0. And in each invertible block, the model performs additive affine transformations to gradually blend the watermark with the carrier image. Finally, the codec outputs (xn,wn) = fθ(w0, x0) and confirms kn = wn as the key of copyright owner. Here is the invertible block working method:

where Ui denotes the upsampling network composed of transposed convolutional layers, D1 i ,D2 i are downsampling network composed of strided convolutional layers, and ⊗ indicates element-wise multiplication. In the ideal extraction stage, we input the key kn and the query carrier yn = xn through backward input, and extract the watermark information by performing the same parameter convolution and reverse coupling of the carrier and key feature information precisely. The corresponding backward propagation of the extraction process is formulated as:

Simultaneously, we also use a trained independent coarsegrained watermark decoder with the same structure, inputting an empty matrix zn and the query carrier yn, and directly extracting the watermark information from the carrier. This decoder ensures I(x;w) remains above a prescribed mutual information threshold, reducing reliance on redundant residual watermark information in the key during fine-grained extraction.

Post-processing aligns dimensions and preserves image quality: the key is passed through a fully connected layer to match the original watermark size, while carrier latents are formed by concatenating clean and watermarked frequencydomain signals across all four Haar bands along the channel dimension, followed by a 1 × 1 convolution.

4.3 Watermark Protection Strategy

First, unlike prior semantic watermarking methods, LatentsInv minimally perturbs the discretized latent noise to preserve its distribution, ensuring content consistency between watermarked and original generations. We avoid KL divergence as the distribution loss, as it inadequately constrains the ℓ2 distance between initial latents and may cause perceptible distortions. Our joint training strategy optimizes both image generation quality and codec accuracy as follows:

where θ denotes the trainable parameters of the codec fθ, including convolutional layers dedicated to watermark protection and fully connected layers designed for feature fusion. And w0 denotes the watermark bits, x0 the carrier latents as in Figure 3. πx and πw denote the projection maps defined by πx(x,w) = x and πw(x,w) = w. As described above, the secret key kn = wn, which is protected by the copyright holder. Moreover, since watermark embedding is fully decoupled from the diffusion model, we have yn = xn in training stage, and this equality holds throughout the section.

To enhance robustness, our method embeds the watermark via an invertible codec split between the public carrier latents and a private key. As detailed in 3.1, since the watermark information associated with the private key remains intact, our method is inherently robust against attacks that attempt to corrupt the watermark in the carrier. However, to reduce false positive rates, we prevent over-reliance on the key's mutual information and ensure sufficient discriminative capability over the distribution of paired carrier latents. We made efforts respectively in two aspects: increasing the information content of the carrier with watermark and reducing the information content of the key with unpaired carrier.

We introduce a coarse-grained decoder and takes a zero matrix with the carrier noise as input to reconstruct the initial watermark. The corresponding loss for promoting watermark embedding in the latent channel is:

where θ′ denotes the trainable parameters of the coarsedecoder fθ′ and zn denotes the empty matrix.

For decreasing the information content of the key with unpaired carrier during decoding, we introduce negative sample pairs (kin , xjn ) with i ̸= j, where kin is not the key jointly output with xjn . we employ the non-negative binary crossentropy loss as a metric to measure the wrong classification rate on negative sample pairs, and introduce a penalty term to prevent overfitting:

Our goal is to ensure that negative sample pairs achieve a bit accuracy approximately equivalent to that of a random guess on a balanced binary watermark sequence. To this end, we set the penalty upper threshold to log 2, which corresponds to the BCE loss of random guessing. This avoids false detection when arbitrary latents is paired with the key and enhances robustness against watermark forgery attack.

The overall training loss with the corresponding weight coefficients λ1, λ2 and λ3 is:

5 Experiments

5.1 Experiment Settings

Baselines. We compare with semantic watermarking methods that embed information in the initial latent noise, focusing on their robustness and transferability-particularly across architectures. We select Tring-Ring (TR)[Wen et al., 2023], Ring-ID(RI)[Ci et al., 2024], WIND[Arabi et al., 2024], SEAL[Arabi et al., 2025], Gaussian Shaping (GS)[Yang et al., 2024b], GaussMarker(GM)[Li et al., 2025], and PRC[Gunn et al., 2024] as baselines to evaluate the performance across UNet to MMDIT models. To assess robustness, we consider two attack paradigms: (i) white-box distortions, including Rotation(Rot.), Crop & Scale(CrSc), GaussianNoise(GauN), GaussianBlurring(GauB), and (ii) a black-box attack by surrogate model aims to single-image latent inversion attacks that perturb initial latents[M¨uller et al., 2025].The former uses toolkit MarkDiffusion[Pan et al., 2025] to evaluate the traditional robustness and the latter can perform threatening watermark removal and forgery attacks facing the most advanced modern attack.

Diffusion Models. We evaluate across four diffusion models: SD1.5 and SD2.1-base (UNet-based), and SD3.medium and FLUX1.dev (MMDiT with rectified flow [Liu et al., 2022]). For black-box forgery and removal attacks [M¨uller et al., 2025], we use SD2.1-base as the surrogate model. We test all eight methods against white-box attacks on SD2.1, SD3.0, and FLUX1.dev, and further compare our approach with representative pattern-semantic (TR) and sampling-semantic (GS) watermarks under black-box attacks on SD1.5.

Datasets & Evaluation metrics. All evaluation images are generated using prompts sampled from the Stable- Diffusion-Prompts dataset and the MSCOCO val2017 validation set [Lin et al., 2014]. To comprehensively assess the impact of watermark on image quality, we compute four metrics, including PSNR, IS, FID, and CLIP Score, across 100 distinct user. For a unified comparison of extraction accuracy across different watermarking methods, performance is quantified by the True Positive Rate (TPR) @ 10-2 FPR.

Implementation Details. Since we retain the latentsinversion error intact, the lightweight flow-based codec is trained without any diffusion model, completing in two hours on a single RTX3090 GPU. During the evaluation stage, the experiments are conducted on four NVIDIA A40 GPUs. We release our code at https://github.com/ mushangexia/Latents-Inv.

5.2 Experiment Results

Comparison of Robustness Performance. As shown in Table 1, our method (LI) significantly outperforms existing approaches under diverse white-box attacks. While baselines such as TR, RI, and WIND completely fail under geometric perturbations-yielding low TPR on SD3.0 for both Rotation and Crop & Scale-LI maintains strong detection rates: 0.8726 (Rotation) and 0.2967 (Crop & Scale) on SD3.0, improving further on FLUX.1 (0.8633 and 0.3019). LI also excels under Gaussian noise and blurring, matching or surpassing strong baselines like GS and GM, indicating that LI embeds watermarks in semantically stable latent regions resilient to both structural distortions and pixel-level corruptions.

Under black-box removal attacks (Fig. 5), adversaries perform gradient-guided optimization to erase the watermark. LI consistently maintains the highest verification credibility, retaining a strong signal even after 100 steps. In contrast, TR degrades rapidly, while GS eventually collapses to near-random levels (credibility ≈ 0.5). We further evaluate LI against watermark forgery attacks(Fig. 6), where adversaries attempt to implant fake watermarks into clean images. Thanks to LI's design-which preserves data distribution consistency before and after embedding-forging a valid watermark becomes extremely difficult, endowing the method with strong anti-forgery capability and reliable verification credibility.

These results confirm that LI leverages mutual information redundancy and inversion fidelity to achieve robustness across diverse attacks and advanced diffusion models.

Comparison of Image Quality Performance. As shown in Table 1, our method (LI) achieves an excellent balance between visual fidelity and semantic coherence. On SD3.0, LI attains the lowest FID (84.93) among all methods, indicating that watermarked images closely match natural image distributions. This is further supported by competitive FID scores on other models such as SD2.1 and FLUX.1. In terms of perceptual quality metrics, LI maintains high IS and CLIP-s scores-6.8454 and 0.8575 on SD3.0-confirming preserved high-level semantics. Notably, despite its robustness under various attacks, LI does not suffer from severe perceptual degradation, as evidenced by its superior PSNR values across all models: 23.52 on SD3.0 and 26.58 on FLUX.1, demonstrating minimal pixel-level distortion. As Fig. 4 illustrates, watermarked images generated by LI are visually indistinguishable from originals. Difference maps reveal that the watermark is uniformly distributed across fine-grained textures such as edges, ripples, and shadows, rather than being localized. This global, texture-level embedding ensures that watermark information is widely dispersed, avoiding concentration in attack-prone regions. Consequently, this design enables robust recovery through multi-region signal aggregation, even when parts of the image are attacked or distorted.

Computational Efficiency. Figure 7 shows that LatentsInv achieves consistently low latency across SD2.1, SD3.0, and FLUX.1, thanks to its lightweight codec design that avoids redundant computation while preserving watermark fidelity-making it well-suited for real-time or large-scale deployment.

Ablation Studies. As shown in Table 2, both singleand dual-path decoders achieve near-perfect bit accuracy (≥0.984) on clean images. However, under adaptation and removal attacks, the single-path variant drops sharply to 0.804-0.831 and 0.622-0.724, respectively, while the dual-path design maintains significantly higher accuracy-confirming that the auxiliary path provides redundancy to preserve mutual information and enhance robustness.

Table 3 further shows that removing the training penalty causes AUC to collapse to 0.50 under attack (despite perfect clean-image TPR), indicating loss of discriminative power against non-watermarked or adversarial samples. Thus, the penalty is essential for balancing mutual information between paths and enabling reliable verification under distortion.

6 Conclusion

In this work, we propose Latents-Inv, a semantic watermarking method for diffusion models that embeds invertible watermarks into the initial latent space. By employing a joint-training strategy that leverages negative-sample pairs under both accuracy and fidelity constraint for a flow-based codec. Our method achieves high visual fidelity and strong robustness across diverse architectures, including UNet and MMDiT. Extensive experiments show that Latents-Inv outperforms existing methods under both traditional distortions and black-box removal/forgery attacks, demonstrating superior transferability and resilience.

Acknowledgments

This work is supported by the the National Nature Science Foundation of China (62425114, 62121002, U23B2028, 62472395) .