OMS: One More Step Noise Searching to Enhance Membership Inference Attacks for Diffusion Models

1 Introduction

Recently, diffusion models [Ho et al., 2020; Song et al., 2020b] have been widely recognized for their unparalleled capability to generate images of exceptional quality, which are increasingly becoming indistinguishable from their realworld counterparts. Due to the high quality of images generated by diffusion models, an increasing number of AI companies are developing generative tools predicated on diffusion models for commercial art design.

∗Corresponding author

Nonetheless, these advancements are accompanied by inherent challenges [Brittain, 2023; Liu et al., 2021]. The dataintensive nature of diffusion models has amplified the risk of privacy infringements and copyright disputes. Trained on extensive unauthorized data scraped from the Internet, these methods overlook the copyrights and privacy of the original owners. A case in point is the recent lawsuit filed by Getty Images against Stability AI, alleging unauthorized use of 12 million of Getty's images for model training. Thus, it is imperative to develop tools to detect diffusion models' privacy infringements.

To audit these privacy risks, Membership Inference Attacks (MIA) [Shokri et al., 2017] have emerged as a potential solution. The objective of MIA is to ascertain whether a data sample has been utilized in the training process of a machine learning model. Existing MIA methods [Sablayrolles et al., 2019; Salem et al., 2019; Song and Mittal, 2021] typically operate under the assumption that member records tend to exhibit lower prediction losses compared to non-member records. Consequently, these methodologies compute the prediction losses and utilize this metric to differentiate between member and non-member records.

Although the utilization of prediction loss to differentiate between member and non-member records has been empirically validated for numerous deterministic models, such as classification models and Generative Adversarial Networks (GANs) [Chen et al., 2020; Hayes et al., 2019; Hilprecht et al., 2019; Choquette-Choo et al., 2021; Hanzlik et al., 2021], its efficacy is diminished when applied to diffusion models due to the intractability of the training loss. More precisely, during the training process of the diffusion model, a random noise is sampled, serving not only as a component of the model's input but also as the target in the training loss. However, during the execution of the membership inference, it is virtually impossible to replicate the exact noise sampled during the training phase. The discrepancy between the noise used during training and membership inference contributes to the inaccuracy of the loss estimation.

Instead of the loss assumption, we propose an alternative hypothesis: it is more feasible for members to find the noise counterpart used in the training process. This assumption aligns more closely with the inherent stochastic characteristics of diffusion model training. Based on this assumption, we introduce a novel MIA framework for diffusion models, leveraging a noise searching mechanism. We formalize the noise searching process as an optimization problem with the training loss as the optimization objective.

Moreover, we propose to utilize the fixed-point iteration to solve the optimization problem. By iteratively applying a function to the initial guess, we strive to facilitate the convergence to the noise encountered by the members during the training stage. We begin with an empirical analysis focuses on the convergence properties of the fixed-point iteration. We discern a distinct attribute where member samples exhibit faster convergence rate compared to non-member samples. This observation implies that the convergence rate can be employed as a discriminative feature to differentiate between member and non-member samples.

This attribute also provides further insights into current MIAs for diffusion models. Specifically, from the perspective of the fixed-point iteration, we reinterpret current MIA methods as assessments of convergence rate, primarily through the first residual. To refine this measurement and capture the nuances of convergence dynamics more comprehensively, we introduce an augmentation to the iteration process, incorporating an additional step that considers the second residual which is termed as the "One More Step" (OMS) approach.

We conduct experiments across various diffusion models, spanning CNN-based and Transformer-based architectures, along with various datasets and MIA methods. Notably, to the best of our knowledge, we are the first to evaluate MIA performance on Transformer-based diffusion models. The experimental results demonstrate the effectiveness of the proposed OMS approach and the noise searching MIA framework. In summary, our paper makes the following contributions:

• We reveal the noise inconsistency issues in current MIA methods for diffusion models. To address this, we devise a novel framework in the perspective of noise searching. Formally, we conceptualize the noise searching process as an optimization problem.
• We propose the fixed-point iteration to solve the noise searching optimization problem. Moreover, we investigate its convergence properties in practice and find that members exhibit faster convergence rate compared to non-members.
• We analyze existing MIA methods through the proposed framework, revealing that the efficacy of existing methods is linked to the convergence rate, particularly as characterized by the first residual. Motivated by this, we introduce a refinement strategy by iterating One More Step (OMS) to include the second residual.
• We conduct experiments on various diffusion models, encompassing CNN-based and Transformer-based architectures, using various datasets. The results not only

confirm the validity of our MIA framework but also underscore the efficacy of the OMS refinements.

2 RelatedWork

Membership InferenceAttack (MIA). The goal ofMIA is to predict the presence of a specific data record in the training set of a given model. The effectiveness of MIA fundamentally relies on the hypothesis that machine learning models exhibit differential responses to member records versus unfamiliar non-member records. Given the manner to exploit model's reactions, existing methods can be divided into two categories, model-based methods and metric-based methods. In the realm of model-based methods [Shokri et al., 2017; Salem et al., 2019; Long et al., 2020; Chen et al., 2020; Truex et al., 2019], a shadow model is trained to mimic the responses of the target machine learning model. Subsequently, attack algorithms are formulated, predicated on the reactions of the shadow model, with the ultimate objective of achieving generalization to the target model. Despite the significant advancements in the field, model-based methods are characterized by their computational intensity and exhibit susceptibility to model's architecture. Methods grounded in metrics [Sablayrolles et al., 2019; Yeom et al., 2018; Salem et al., 2020; Bentley et al., 2020] primarily employ a metric (typically the loss value) as a representative measure of the model's response to each sample. The membership of a specific sample is subsequently determined based on the numerical values of the selected metric.

MIA for diffusion models. Given the computational intensity of training a shadow model with comparable parameters, model-based methods are deemed unsuitable for diffusion models. As a result, most current MIA tailored for diffusion models [Duan et al., 2023; Matsumoto et al., 2023; Kong et al., 2023] are metric-based methods. These methods assume that the loss value for members is smaller than that for non-members. Despite achieving substantial performance, these methods still suffer from the inaccuracy of loss estimation. During the training phase of the diffusion model, the loss value is dictated by the training target (a random Gaussian noise). However, when executing the MIA, replicating the exact noise sampled during the training process is virtually unattainable. Instead, we propose a novel MIA framework predicated on noise searching. This innovative approach promises to enhance the overall performance of MIA and provide insight into the principles of MIA methods for diffusion models.

3 Method

Given a data record x0, the goal of MIA is to identify whether x0 is in the training set of the target diffusion model ϵθ. Existing MIAs [Duan et al., 2023; Matsumoto et al., 2023; Kong et al., 2023] tailored for diffusion models predominantly assume the members' loss values are lower than those for non-members. However, these loss-based approaches are susceptible to inaccuracies in loss estimation, which are caused by noise inconsistency between the training and inference stages (Section 3.2). In contrast, we propose a novel MIA framework that employs noise searching, an approach we believe aligns more closely with the stochastic nature of the model's training process. We formulate the noise searching process as an optimization problem and use the fixedpoint iteration to solve this problem (Section 3.3). We posit that members can retrieve the training noise with less effort compared to non-members. Subsequently, we analyze the convergence properties of the fixed-point iteration and further validate that it is more feasible for members to search the noise than non-members (Section 3.4). Motivated by this, we reinterpret the underlying mechanisms contributing to the efficacy of existing MIA methods and propose an enhancement by incorporating One More iteration Step (OMS) (Section 3.5).

3.1 Background and Notations

We begin with a brief introduction of the background and notations of the diffusion models. Denoising Diffusion Probabilistic Models (DDPM) [Ho et al., 2020; Song et al., 2020a] consist of a forward and a reverse process. The forward process, also named as the diffusion process, gradually adds Gaussian noise to the input image x0 in T time steps according to a predefined variance schedule β1, ..., βT :

Let αt = 1 - βt and ¯αt = ∏ts=1 αs, this process can be simplified to:

When t is large enough, the ¯α is approaching 0, making xt an isotropic Gaussian noise. The reverse process aims to recover the data distribution from the Gaussian noise. The reverse process in one step can be represented as:

where Σt is a constant depending on the variance schedule βt and µθ(xt, t) is determined by a neural network:

By recursively leveraging the reverse step, Gaussian noise can be recovered to the original image. To train the DDPM, an image x0, a timestep t and a random noise ϵ ∼ N(0, I) are first sampled. A noisy image xt is then obtained by using the forward process (Equation 2). We then input both the noisy image xt and the timestep t into a U-Net [Ronneberger et al., 2015] ϵθ to predict the noise within xt. The optimization objective for the denoising U-Net can be written as:

3.2 Noise Inconsistency Between Training and Inference

The diffusion model's training procedure can be described by Equation 5. To elaborate, given the input image x0 and a specific timestep t, a random noise ϵtrain is sampled from the standard normal distribution. This noise is then utilized to perturb x0 into a corrupted version xt, following the schedule predefined in Equation 2. Subsequently, the diffusion model, parameterized by θ generates a prediction of the noise within xt (denoted as ϵθ(x0, t, ϵtrain)). The training loss for the diffusion model is computed as the distance between the predicted noise ϵθ(x0, t, ϵtrain) and the actual sampled noise ϵtrain. During the inference phase, due to the infeasibility of the training noise ϵtrain, an alternate noise ϵinf is sampled to estimate the loss value. However, it is important to note that there exists no guarantee that ϵinf is identical or approximately similar to the noise ϵtrain. This inconsistency in noise significantly impacts the accuracy of the loss values, consequently affecting the effectiveness of existing MIA targeting diffusion models.

3.3 MIA by Noise Searching

In contrast to existing MIA that rely on the randomly sampled inference noise as a surrogate for the training noise ϵtrain to approximate the loss, thereby encountering the noise inconsistency issue, we introduce a novel MIA framework for diffusion models, leveraging a noise search strategy. Our approach aims to reconstruct, for a given record x0, the corresponding training noise ϵtrain that minimizes the training loss as defined in Equation 5. We assume that it is more feasible for members to obtain the training noise ϵtrain. We formulate the process of noise searching as an optimization problem:

This optimization framework directly addresses the noise inconsistency issue by focusing on identifying the true training noise ϵtrain. Compared with loss-based methods, the proposed approach emphasizes the identification of ϵtrain. We argue this approach aligns more coherently with the inherent stochastic nature of the diffusion model's training process.

Fixed-point iteration. We address the aforementioned optimization problem by the fixed-point iteration. For a given record x0 and timestep t, the predicted noise ϵθ(x0, t, ϵ) is solely dependent on ϵ. This dependency can be represented as an implicit function ϵ = f(ϵ). The optimal noise, which pairs with the record x0 during the training process, is also identified as the solution to the implicit function f. To address this, we employ the fixed-point iteration [Smart, 1980]. The iterative process can be represented as follows:

We assume that the fixed-point iteration process essentially satisfies the constraints embedded within the optimization problem, given the fact that the model ϵθ is trained to generate noises adhering to the distribution. Consequently, we hypothesize that the model's outputs also conform to the standard normal distribution. Note that we do not use more advanced methods for solving implicit functions such as NewtonRaphson or Conjugate Gradient [Nocedal and Wright, 1999]. This is because the Newton-Raphson method needs to compute gradients while the Conjugate Gradients need to search high dimension gradients, both of which are computationally intensive and potentially intractable.

3.4 Convergence of the Fixed-point Iteration

The primary concern about the fixed-point iteration lies in its practical convergence properties. We present an empirical analysis to address this concern. Given the initial ϵ0 , our objective is to demonstrate that the sequence {ϵn }, n → ∞ generated by Equation 7 converges. To achieve this, we aim to prove that the residual δn = ϵn - ϵn-1 converges, which would imply that {ϵn } is the Cauchy sequence. The residual can be expressed as follows:

Through Taylor Expansion, we obtain:

In a sufficiently confined domain, the term ||O(δ2 )|| can be considered negligible, and the convergence dynamics are primarily governed by the Jacobian norm || ∂f(ϵ) ∂ϵ ||. If the Jacobian norm is below 1, it indicates that the implicit function f is contractive, leading to an exponential decay in the residuals, thereby affirming the convergence of the fixed-point iteration. We visualize this Jacobian norm (the top row) along with the residuals (the bottom row) across various iterations and timesteps in Figure 1. Notably, the Jacobian norm consistently remains below the threshold of 1, thereby empirically validating the convergence of the fixed-point iteration. In the residual plots, the theoretical distance (i.e., the distance between two random Gaussian noise) is depicted in green, whereas the residuals for member and non-member sets are depicted in blue and red, respectively. Moreover, it is observed that the residuals for both member and non-member sets exhibit rapid convergence, with the member set residuals smaller than those of non-member set, further indicating that it is more feasible for members to obtain the training noise.

It is also important to highlight that the first residual δ1 , representing the divergence between ϵ0 and ϵ1 , is frequently employed as an estimation of the training loss in current MIA methods. We also provide validation about the convergence property and convergence speed from the lens of contraction mapping theorem [Berinde and Takens, 2007].

3.5 ExistingMIAs and One More Step (OMS)

We re-assess the efficacy of prevailing MIA methods through the lens of the fixed-point iteration. Current MIA methods approximate the training loss of diffusion models by measuring the divergence between the initial noise ϵ0 and the noise after the first iteration ϵ1 . In the context of the fixed-point iteration, this divergence is equivalently characterized as the first residual δ1 . As illustrated in the bottom row of Figure 1, this residual effectively discriminates between members and non-members. However, it is also discernible that the second residual δ2 for members is smaller than non-members, indicating its substantial potential for enhancing membership discrimination. Motivated by this observation, we take One More Step (OMS) beyond ϵ1 to obtain ϵ2 and utilize the distance between ϵ0 and ϵ2 as the discriminative metric. This metric can be interpreted as an ensemble of the first and second residuals (δ1 and δ2 ). This approach not only preserves the discriminative capability inherent in the traditional lossbased approach (i.e., δ1 ), but also incorporating the extra information δ2 to augment the performance. The relationship is mathematically expressed as:

Note that we do not leverage further residuals such as δ3 and δ4 , though they also seem potential metric to distinguish the member and non-member records. This is because the marginal gain is decreased. We also provide experiments utilizing different residuals in Section 4.5.

4 Experiment

4.1 Experimental Setup

Diffusion Models and Datasets. We evaluate our proposed method across diverse diffusion models, specifically DDPM [Ho et al., 2020], Stable Diffusion [Rombach et al., 2022] and U-ViT [Bao et al., 2023]. DDPM represents a foundational approach in the realm of diffusion models, which employs convolutional neural networks as the backbone. We train DDPM on four datasets: Cifar10, Cifar100 [Krizhevsky et al., 2009], LFW [Huang et al., 2008] and Lsun-Cat [Yu et al., 2015]. The Stable Diffusion models, which are prominently recognized for their text-to-image synthesis capabilities, have undergone numerous iterations. We selectively adopt SD1.5 and SD2.1 due to their widespread usage and recognition within the research community. U-ViT, a recently introduced diffusion model, incorporates transformers as its core architecture. Our investigation leverages the open-source implementation of U-ViT which has been trained on the Cifar10 datasets.

Evaluation Metrics. To evaluate the performance of our proposed method, we adopte established metrics in previous works [Carlini et al., 2022; Carlini et al., 2023; Duan et al., 2023; Kong et al., 2023] including Attack Success Rate (ASR), AUC and the True Positive Rate (TPR) at extremely low False Positive Rate (FPR). Specifically, TPR@1%FPR and TPR@0.1%FPR refer to the True Positive Rate (TPR) when the False Positive Rate (FPR) is constrained to 1% and 0.1%, respectively.

Implementation Details. To evaluate the effectiveness of the proposed OMS, we conduct a series of experiments, aligning our benchmarks with state-of-the-art MIA methods designed for diffusion models, which include the Naive Attack (NA) [Matsumoto et al., 2023], SecMI [Duan et al., 2023], PIA [Kong et al., 2023], GSA [Pang et al., 2023] and Quantile [Bertran et al., 2024; Tang et al., 2023]. We strictly follow the prescribed settings of these methods, and exclusively introduce a further fix-point iteration. Notably, our approach not only seamlessly integrates with these established MIA methods but also augments their performance.

4.2 Evaluation Results

Performance on DDPM. The comparative results on DDPM, with and without OMS, are presented in Table 1. It can be observed that the OMS confers substantial improvements in performance, with increases of 8.20%, 9.65% and 3.28% in the Average AUC across the four datasets, compared to those baselines (NA, SecMI, PIA) without OMS. The improvements demonstrate the advantage of executing multiple fixed-point iterations over the conventional singleiteration approaches. We also observe that our method is particularly effective for weak attackers: an AUC increase from 64.90 to 85.67 for SecMI. Besides, we also note our method can further boost strong attackers with an average 3.72% AUC improvement for PIA. There results demonstrate the broad applicability of our proposed OMS. Furthermore, we provide the results of TPR at extremely low FPR in Table 2. These results demonstrate that the OMS notably enhances the prediction confidence, thereby amplifying the practical applicability in scenarios requiring high prediction certainty.

Performance on text-to-image diffusionmodels. Distinct from unconditional diffusion models, text-to-image diffusion models require dual inputs: the image itself and an accompanying text. However, in real-world scenario, images are seldom annotated by texts. It is a common case that users do not have access to the text employed during the training phase. To replicate this real-world scenario, we leverage BLIP [Li et al., 2022] to generate text captions for the input images. The results on text-to-image diffusion models are detailed in Table 3. These evaluation further corroborate the substantial performance improvements that can be achieved by incorporating the OMS into current MIA methods.

Performance on Transformer-based diffusion models. The traditional diffusion models predominantly leverage CNNs as their backbone. However, recent advancements have seen an increasing trend towards the adoption of Transformers as the foundational architecture [Bao et al., 2023; Chen et al., 2024; Peebles and Xie, 2023; Esser et al., 2024]. To assess the efficacy of existing MIA methods on Transformer-based diffusion models and substantiate the effectiveness of our approach, we conduct experiments utilizing the U-ViT model, a continuous time diffusion model based on Transformers. Notably, the majority of existing MIA methods are specifically designed for discrete time diffusion models, posing a challenge for direct application to the U-ViT model. To address this, we implement a simple mapping strategy, converting the discrete timestep within the range [0, 1000] to the continuous range [0, 1]. Additionally, we observe that the performance of PIA approximates random guessing. To mitigate this issue, we leverage a regularization technique [Kong et al., 2023], hereby referred to as PIAN. The results, presented in Table 4, reveal that the incorporation of an additional fixed-point iteration, as proposed in our method, led to performance improvements in existing methods, suggesting the robustness and efficacy of OMS approach across diffusion models with diverse architectures.

4.3 Integration with Quantile Regression

Quantile Regression [Tang et al., 2023] incorporates the terror metric (proposed by SecMI [Duan et al., 2023]) to learn a quantile regression model that predicts the α-quantile of the t-error for each individual sample. This approach enables the estimation of a sample-specific α-quantile as a refined persample threshold for identifying membership status. While t-error serves as a fundamental confidence metric for quantile regression, we have shown that the t-error can be augmented through OMS (Table 1- 3). Similarly, we refine quantile regression by incorporating an additional fixed-point iteration to current confidence metric (t-error). This refinement leads to improved performance, as evidenced in Table 5.

4.4 Integration with Gradient-basedMethod

GSA [Pang et al., 2023] constitutes a gradient-based MIA method which posits that the gradients inherently convey a

(a) The results of OMS for differ-

ent training epochs.

(b) The results of OMS for differ-

ent timesteps.

Figure 2: The results of AUC and TPR@1%FPR metrics of OMS for different training epochs and different timesteps on Cifar10.

more direct indication of how the target model responds to member and non-member samples. As a white-box attacker, GSA demonstrates significant efficacy against diffusion models compared to other attackers. We concentrate on the backpropagation GSA, which harnesses the backward pass of gradient computation during loss optimization to execute MIA. We refine this back-propagation GSA by backwarding the loss after OMS (Equation 7). Notably, as evidenced in Table 6, the OMS is also capable of enhancing the efficacy of gradient-based methods.

4.5 Ablation Study

The Training Steps. Previous researches [Leino and Fredrikson, 2020; Salem et al., 2019] have highlighted the tendency of machine learning models to memorize training data as the training procedure progresses. Based on these insights, we conduct an evaluation of our method throughout the training process. The results are presented in Figure 2(a). We observe that all these examined MIA methods exhibit enhanced performance as the training epochs increase, which corroborates the phenomenon of model memorization. Another notable observation is the consistent efficiency of our method throughout the entire training process. Furthermore, we identify that PIA begins to saturate in terms of AUC after 1500 training epochs. Our method further boosts PIA's performance on TPR@1%FPR, providing compelling evidence for the superiority and robustness of our approach.

The Timesteps. The timestep serves as a crucial parameter for the level of noise incorporated into the input of the denoising U-Net, significantly impacting the performance of the diffusion models. Consequently, we execute MIA across a range of timesteps, specifically from 50 to 250. The performance enhancements attributed to OMS are presented in Figure 2(b).

Our observations indicate that the OMS exhibits robust performance across timesteps but the improvements diminishes as the timestep increases. This observed decline can be attributed to the increasing prominence of noise in the model's input. Specifically, as the timestep increases, the noise component becomes the dominant factor, potentially disrupting the stability of the fixed-point iteration process.

The number of iteration. We incorporate an additional fixed-point iteration for computational efficiency, which is also validated by previous experimental results. In this experiment, we explore varying fixed-point iterations and harness the distance to execute MIA. The results are depicted in Figure 3. Specifically, ||ϵ1 - ϵ0 || represents the NA approach, whereas ||ϵ2 - ϵ0 || represents NA with OMS in previous experiments. It is evident that increasing the number of iterations leads to improved performance, with ||ϵ3 - ϵ0 || demonstrating the optimal results. While residuals (||ϵ2 - ϵ1 || and ||ϵ3 - ϵ2 ||) exhibit some level of effectiveness, their performance diminishes as the iteration count increases. It is also noteworthy that while additional fixed-point iterations hold the potential for superior performance, the marginal gains diminish progressively.

5 Conclusion

In this paper, we explore the MIA for diffusion models in a novel perspective, i.e., the noise searching. We first analyze the noise inconsistency issue between the training and membership inference stage. To address this issue, we introduce a noise searching framework that formulates the search for optimal training noise as an optimization problem. Utilizing the fixed-point iteration, we solve the optimization problem and conduct a thorough examination of its convergence properties, revealing distinct convergence rates between member and non-member data. Based on this observation, we rethink the effectiveness of current MIA methods and propose an enhancement through one more iteration step, resulting in a substantial performance boost for existingMIAmethods. In conclusion, the proposed noise searching framework provides a unique and unified perspective for comprehending the fundamental principles of MIA tasks for diffusion models. We anticipate that our contributions will foster further research into the privacy risks associated with diffusion models and contribute to the ongoing research in this field.

Ethical Statement

The primary objective of our research is to devise a method capable of discerning whether a particular sample was included in the training dataset. The proposed method offers a multitude of beneficial applications, encompassing the detection of privacy violations and the assessment of model privacy. While acknowledging the potential for malevolent entities to misuse our method for privacy attacks, we underscore the capacity of privacy protection techniques, such as differential privacy, to counteract such threats. It is crucial to note that the development of these techniques is not intended to facilitate malicious activities, but rather to advance the field of privacy protection. We trust that our contributions will be used responsibly to enhance privacy protection measures and promote ethical practices in machine learning research.

Acknowledgments

This work was supported in part by Science and Technology Innovation (STI) 2030-Major Projects under Grant 2022ZD0208700, and National Natural Science Foundation of China under Grant 62376264.