Membership Inference via Backdooring

1 Introduction

Machine learning (ML) has achieved tremendous results for various learning tasks, such as image recognition [He et al.,

∗Corresponding Author.

2016] and natural language processing [Devlin et al., 2018]. Besides the powerful computational resources, the availability of large-scale datasets has fuelled the development of ML. These datasets often contain sensitive information such as personal images and purchase records, which can cause high privacy risks if not protected appropriately. For example, an AI company Clearview collected millions of photographs without owners' consent from Twitter and Facebook for suspect identification, causing severe privacy breaches and regulation violation [Smith and Miller, 2022]. Recently, many data privacy regulations and legislations such as GDPR (General Data Protection Regulation) [Mantelero, 2013] and CCPA (California Consumer Privacy Act) [de la Torre, 2018] have been issued to protect individuals' data and privacy. Especially, such regulations grant individuals the right to be forgotten. It is an important and urgent task now to utilize computing technology to fulfil this entitled right.

In the context of ML, the right can be fulfilled by the machine unlearning techniques which can let a model forget about a training sample (a.k.a. member) if requested by the data owner [Bourtoule et al., 2021]. Many recent studies [Song et al., 2017; Carlini et al., 2019] have shown that deep learning models can easily memorize training data, and existing machine unlearning works [Guo et al., 2020; Bourtoule et al., 2021] mainly focus on how to eliminate the contribution of a training sample to the model. Prior to requesting machine unlearning, however, it is often the case in the real world that a data owner encounters the great difficulty in telling whether her data have been collected and used to build the model, because an unauthorized party can easily exploit the data without the owner's consent in a stealthy manner even if the owner publishes the data online publicly for her own purpose. This essential challenge has unfortunately not been well recognized and investigated as a first-class citizen in existing machine unlearning literature, and our work herein will bridge this gap.

Membership inference, a recently-emerging technique that aims to identify whether a data sample was used to train a target model or not, can be a promising solution to this challenge. However, existing membership inference techniques are mainly developed in the setting of membership privacy attacks [Shokri et al., 2017; Yeom et al., 2018; Salem et al., 2019]. A typical example is that an attacker can construct an attack model to identify whether a clinical record has been used to train a model associated with a certain disease, breaching the record owner's privacy. Being mainly explored from the attacker's perspective, most existing membership inference methods assume that the attacker has rich information for inference, e.g., the knowledge of training data distribution and the architectures of target models. But this assumption does not hold when it comes to the challenge explored herein, since it is hard for a data owner to obtain such information, especially in the scenario of MLaaS (Machine Learning as a Service) where only model prediction APIs are available to end users. Moreover, existing membership inference attack models fail to achieve sufficiently high attack accuracy when the target modes are well-generalized, and the attack model training is often computation-intensive [Hu et al., 2021]. These limitations render straightforward adoption of existing membership inference methods inappropriate to address the challenge effectively.

In this paper, we propose a novel membership inference approach called Membership Inference via Backdooring (MIB), inspired by the backdoor technology in ML [Gu et al., 2019; Chen et al., 2017; Li et al., 2020]. The intuition of MIB is that a data owner proactively adds markers to her data samples when releasing them online, so that at a later stage she can conduct membership inference to determine whether a model in question (i.e., target model) has exploited her released data for model training. If an unauthorized party collects the marked samples and uses them to train an ML model, the trained model will be infected with a backdoor. Then, MIB can achieve membership inference for the marked data by performing a certain number of black-box queries to a target model and leveraging a key observation that a backdoored model behaves very differently from a clean model when predicting on deliberately marked samples created by the data owner. To provide theoretical guarantees for the inference results, we innovatively adopt statistical hypothesis testing [Montgomery and Runger, 2010] to prove whether a target model has been backdoored. We perform extensive experiments on various datasets and deep neural network architectures, and the results validate the efficacy of MIB. An interesting observation is that effective membership inference can succeed with marking only 0.1% of training data. The source code is available at: https://github.com/HongshengHu/ membership-inference-via-backdooring.

Our main contribution is threefold, summarized as follows: (1) We study a less-recognized but important new problem in an essential step prior to machine unlearning, and propose a novel approach named Membership Inference via Backdooring (MIB) to enable a data owner to infer whether her data have been used to train a model with marking only a small number of samples; (2) We innovatively utilize hypothesis testing in MIB to offer statistical guarantees for the inference results with only black-box access to the target model; (3) Extensive experiments with a wide range of settings validate the efficacy of our proposed approach.

2 RelatedWork

Membership Inference. Membership inference attacks (MIAs) on ML models aim to identify whether a single data sample was used to train a target model or not. There are mainly two types of techniques to implement MIAs, i.e., shadow training [Shokri et al., 2017] and the metricbased technique [Yeom et al., 2018; Song and Mittal, 2021; Salem et al., 2019]. Both of them require the prior knowledge of training data distribution or architectures of target models [Hu et al., 2021], while we consider such information is unavailable in our problem setting, which is more challenging but more practical in real-world applications.

Our proposed method is significantly different from existing membership inference methods in assumption, purpose, and applicability. Adopting the shadow training technique from MIAs, [Song and Shmatikov, 2019] designed a membership inference method that allows a data owner to detect whether her text was used to train a text-generation model. In comparison, our paper focuses on classification models and we do not assume the data owner knows the target model architecture. [Sablayrolles et al., 2020] focused on datasetlevel membership inference and proposed an approach to detect whether a particular dataset was used to train a model or not. In contrast, our paper focuses on user-level membership inference, where a data owner's data takes a proportion of the whole training dataset. Last, [Zou et al., 2021] proposed an inference approach by embedding a signature into a data owner's personal images. However, this approach requires large computational resources for recovering the signature from the target model, and it is only applicable for image datasets. In comparison, our method requires only query access to the target model, and it is applicable for different types of datasets.

Backdoor Technology. Backdoor attacks on ML models aim to embed hidden backdoor into the models during the training process such that the infected models perform well on benign samples, whereas their prediction will be maliciously changed if the hidden backdoor is activated by the attacker-defined trigger during the inference process [Li et al., 2020]. To achieve backdoor attacks, training data poisoning [Gu et al., 2019; Liu et al., 2020; Schwarzschild et al., 2021] is the most common technique via injecting a portion of poisoned samples into the training dataset. Backdoor technology has been adopted to protect the intellectual property of ML models [Adi et al., 2018; Jia et al., 2021] and datasets [Li et al., 2020]. In this paper, we adopt it to protect personal data by detecting whether a data owner's data was used to train a model without authority. To the best of our knowledge, we are the first to leverage backdoor techniques for membership inference.

3 Membership Inference via Backdooring

3.1 ProblemFormulation

In this paper, we focus on classification problems and userlevel membership inference. Let u be a data owner who has multiple data samples (x1, y1), · · · , (xn, yn), where each sample has its feature x ∈ X and label y ∈ Y . An unauthorized party collects these data samples from the owner u and includes them in a dataset Dtrain to train a classification model f(·). After training, the unauthorized party releases the trained model to end users for commercial purposes, e.g., Machine Learning as a Service. The data owner is curious about whether her data was used by the party to train f(·) because such training usage is unauthorized and can cause severe privacy risks to her.

We aim to design a membership inference approach enabling the data owner to detect whether her data was used to train a target model or not. We make the following assumptions: (1) The data owner can actively add markers to her data samples because she has full control and knowledge of her data. (2) The data owner has only black-box access to the target model, i.e., she can query the target model and get the prediction output, which is the most challenging setting for membership inference.

3.2 MIB: Membership Inference via Backdooring

Fig. 1 shows the process of the proposed membership inference approach, which consists of three phases: a) The data owner generates marked data; b) The unauthorized party collects the data from the data owner and includes it to the training dataset to train a deep neural network (DNN) model; c) The data owner queries the target model to detect whether her data was used to train the target model or not. We describe the three phases in more detail.

a) Generating Marked Data. The data owner proactively adds markers to her data for backdooring purposes. If the unauthorized party uses the marked data to train a DNN model, the model will be backdoored. Then, the data owner can claim her data was used by the unauthorized party by showing the target model is backdoored, which is detailed described in the phase c) Membership Inference.

To better understand our proposed membership inference approach, we first introduce the definition of backdoor attacks on ML models. A backdoor attacker A is associated with a target label yt ∈ Y , a backdoor trigger p ∈ P, and a backdoor-sample-generation function g(·, ·). The backdoor trigger p belongs to the trigger space P , which is a subspace of X. For a data sample z = (x, y) with its feature x and label y, the backdoor-sample-generation function g(z, p) generates a backdoor sample z′ = (x′ , yt). A model f(·) trained on backdoor samples will be infected with a hidden backdoor, which can be activated by the trigger p. The goal of a backdoor attacker is to make the backdoor attack success probability Pr (f(x′ ) = yt) to be high, where f(x′ ) is the prediction results of a backdoor testing sample. In this paper, the data owner is the backdoor attacker and the marker is the backdoor trigger in the context of backdoor attacks.

We adopt the backdoor technique from BadNets [Gu et al., 2019], which is the first proposed backdoor technique in deep learning. Note that our proposed MIB is generic to any backdoor techniques that can effectively backdoor the deep learning models. The adoption of more advanced backdoor techniques than BadNets [Gu et al., 2019] is orthogonal to the goals of this paper. In Badnets [Gu et al., 2019], the backdoor-sample-generation function g(·, ·) is defined as:

where ⊗ is the element-wise product, and v is a mapping parameter that has the same form as x with each element ranges in [0, 1]. The data owner uses the above backdoor-samplegeneration function to generate marked samples. If an unauthorized party includes these marked samples to the training dataset to train a DNN model, the model will finally learn the correlation between the trigger and the target label, i.e., the model will be backdoored.

Although the data owner can arbitrarily modify her data in principle, following most studies of backdoor attacks [Chen et al., 2017; Saha et al., 2020; Liu et al., 2020], we adopt one best practice for the data owner when generating the marked samples. Let:

where x is an original sample and x′ the corresponding marked sample. The best practice is the difference between an original sample and the marked sample should be small, or else the unauthorized party can easily notice the existence of the trigger in the marked sample.

b) Training. The unauthorized party collects the data from the data owner. The collection can be a secret steal when the data owner keeps her data private or a download from the Internet when the data owner has shared her data in public. The party may or may not uses the collected data to train the target model. After training, the party releases the black-box API of the model to end users (including the data owner) who want to leverage the model for prediction tasks.

c) Membership Inference. Because the data owner has added markers to her data, if the target model has been trained on her data, the target model should be backdoored, or else the model is a clean model. Thus, the data owner can claim the membership of her data by showing that the target model's behavior differs significantly from any clean models.

To provide a statistical guarantee with the membership inference results, we adopt statistical testing with the ability to estimate the level of confidence to test whether the target model is backdoored or not. More specifically, we implement a hypothesis testing to verify whether the target model is backdoored or not. We define the null hypothesis H0 and the alternative hypothesis H1 as follows:

where Pr (f(x′ ) = yt) represents the backdoor attack success probability of the target model, and β represents the backdoor attack success probability of a clean model. In this paper, we set β = 1K (i.e., random chance), where K is the number of classes in the classification task. In the experiments, we will show that the backdoor attack success probability of a clean model is actually far less than 1K .

The null hypothesis H0 states that the backdoor attack success probability is smaller than or equal to random chance, i.e., there is no significant differences between the behavior of the target model and a clean model. On the contrary, H1 states that the backdoor attack success probability is larger than random chance, i.e., the target model behaves significantly different from the clean model. If the data owner can reject the null hypothesis H0 with statistical guarantees, she can claim that her data was used to train the target model.

Because the data owner is given black-box access to the target model, she can query the model with m backdoor testing samples and obtain their prediction results R1 · · · Rm, which are used to calculate the backdoor attack success rate (ASR), denoted as α. ASR is defined as follows:

The value of ASR can be considered as an estimation of the backdoor attack success probability. In the next section, we will present how large ASR needs to be to reject the null hypothesis H0.

4 Theoretical Analysis

In this paper, we consider the data owner can use a ttest [Montgomery and Runger, 2010] to test the hypothesis. Below, we formally stated under what conditions the data owner can reject the null hypothesis H0 at the significance level 1 - τ (i.e., with τ confidence) with a limited number of queries to the target model.

Theorem 1. Given a target model f(·) and the number of classes K in the classification task, with the number of queries to f(·) at m, if the backdoor attack success rate (ASR) α of f(·) satisfies the following formula:

the data owner can reject the null hypothesis H0 at the significance level 1 - τ, where β = 1K and tτ is the τ quantile of the t distribution with m- 1 degrees of freedom.

The proof of the theorem can be found in Appendix 1 . Theorem 1 implies that if the ASR of the target model is higher than a threshold, the data owner is able to reject the null hypothesis H0 at the significance level 1 - τ with m queries to the target model. In other words, the data owner can claim the membership of her data with τ confidence via limited queries to the target model when the value of ASR is large enough.

5 Experimental Evaluation

We evaluate MIB on benchmark datasets, i.e., CIFAR10 [Krizhevsky et al., 2009], Location-30 [Shokri et al., 2017], and Purchase-100 [Shokri et al., 2017], which are widely used in existing research on MIAs [Hu et al., 2021]. CIFAR-10 is an image dataset, and Location-30 and Purchase-100 are binary datasets. We use Resnet-18 [He et al., 2016] as the target model for CIFAR-10 and fullyconnected (FC) neural network with two hidden layers (256 units and 128 units) for Location-30 and Purchase-100. The data description is summarized in Table 1. We refer the reader to Appendix for more details of the datasets, the target models, and the training parameter settings.

5.1 Threshold of ASR for Membership Inference

The data owner is given black-box access to the target model for membership inference. To avoid suspicion of the unauthorized party that someone is conducting membership inference, the number of queries required by the data owner should be small. Thus, we set the number of queries to its minimum value, i.e., 30, to ensure the Central Limit Theorem (CLT) [Montgomery and Runger, 2010] is valid for the hypothesis test. We set the significance level at 0.05, which is the common practice in statistical hypothesis testing. [Craparo, 2007]. According to Theorem 1, the threshold of ASR to claim the membership with 95% confidence is calculated as 23.3%, 14.4%, and 10.7% for CIFAR-10, Location-30, and Purchase-100, respectively.

1 Please refer to the version of this paper with Appendix in arXiv.

5.2 Experimental Results

To demonstrate the effectiveness of MIB, we first show the results of the single data owner case: There is only one data owner who marks her data and wishes to implement membership inference. Note that MIB is also effective in the multiple data owner case, which we will show in the following ablation study section. For CIFAR-10, we consider the data owner uses a 3×3 white square (i.e., ϵ ≤ 9) as the trigger pattern and stamps it on the bottom right corner of the selected samples. Since images can be visualized, we adopt the blended strategy from [Chen et al., 2017] by setting the elements of the mapping parameter v to be 0.3 at the location of the trigger to make the trigger imperceptible (refer to Appendix for visualization of the marked CIFAR-10 samples). For Location-30 and Purchase-100, the data owner uses a 20-length binary array (i.e., ϵ ≤ 20) as the trigger pattern and replaces the last 20 features of the selected samples to the trigger. We set the target backdoor label as 1 for all dataset.

Table 4 shows the experimental results of the one data owner case. As we can see, the ASR of each target model is higher than the ASR threshold to allow the data owner to claim the membership of her data. Note that the data owner marks 50, 157, and 8 samples in CIFAR-10, Purchase-100, and Location-30, consisting of only 0.1% of the training samples of CIFAR-10 and Purchase-100 and 0.2% of the training samples of Location-30, respectively. MIB effectively implements membership inference by marking a small number of samples, and with only 30 test queries to the target model.

5.3 Ablation Study

We conduct detailed ablation studies to investigate how different factors affect the performance of MIB. The experimental setting is the same as Section 5.2 except for the factor we evaluated in the ablation study.

Evaluation on trigger pattern and target label. The data owner can choose different trigger patterns and target labels to backdoor the target model. Table 2 shows the ASR of 5 different randomly generated trigger patterns and 5 different randomly selected target labels. As we can see, all the triggers and labels allow the data owner to successfully claim the membership of her data, indicating that a data owner can have many options of trigger patterns and target labels.

Evaluation on trigger location and trigger size. The data owner can place the trigger pattern to different locations. We evaluate 4 different trigger locations for CIFAR-10, i.e., stamping the trigger at different corners of the image. We evaluate 3 different trigger locations for Location-30 and Purchase-100, i.e., placing the trigger to the beginning, the center, or the end of the data sample. The trigger size is directly related to ϵ, which should be small to make the trigger imperceptible. Table 3 shows the ASR of different trigger locations and sizes. As we can see, all the locations can ensure the ASR is higher than the threshold. We notice that when the trigger size is too small, e.g., the trigger is a 2×2 white square for CIFAR-10 and a 1-length binary array for Location-30 and Purchase-100, it seems that the target model cannot learn the association between the trigger and the target label. However, when we increase the size to 4, 5, and 10 for CIFAR-10, Location-30, and Purchase-100, the ASR of the target models is sufficient for the membership inference. Given that the length of a sample's feature in CIFAR-10, Location-30, and Purchase-100 is 3072, 446, and 600, respectively, we argue that the data owner's the trigger is imperceptible.

Evaluation on marking ratio. The marking ratio is the fraction of the training samples that are marked by the data

• (a) ASR of different marking ratio (b) Ten data owners case

Figure 2: ASR in different settings. (a) ASR of different marking ratio. (b) ASR of ten different data owners.

owner. Fig. 2a shows the ASR of different marking ratio, and the dotted lines represent the ASR threshold. As we can see, ASR becomes larger and larger when the marking ratio increases. When the marking ratio is larger than 0.1%, the ASR of the the target models is larger than the threshold, indicating that a data owner can claim the membership of her data by marking a very small proportion of the training dataset.

Evaluation on # of data owners. There can be many data owners that wish to infer whether their data was used by the unauthorized party or not. Fig. 2b show the ASR of a ten data owner case, where each data owner uses different trigger patterns with different target labels. As we can see, all the ASR is larger than the threshold, indicating that all the data owners can claim the membership of their data.

There is a limit to the number of data owners for membership inference. Assuming a data owner should mark at least p% of the training data to backdoor the model (e.g., in our settings, p ≥ 0.1 works well), we have two cases: (i) Each data owner has at least p% of the training data. The maximum number of owners is ⌊ 1 p% ⌋. (ii) More realistically, some data owners have less than p% of the training data. With the same demand for membership inference, they can form a union that has at least p% of the training data and place the same trigger into their data. This union can be regarded as a 'virtual' data owner which is equivalent to that in Case (i). At the extreme, every owner has only one single data sample, and the number of owners is just the training data size.

Trigger options. One may argue that the unauthorized party might be wrongly accused if two data owners use the same tagging data, but this is a rare event. Let A denote the event. The probability Pr(A) is very low for two reasons: (i) There are many options for trigger design, e.g., using a 20-bit array for the triggers results in 220 different candidates, and Pr(A) = 1 220 ≈ 10-6 . (ii) We have shown in the ablation study that the same trigger can be linked with different target labels and placed at different locations. Even if two data owners use the same trigger, the probability of generating the same tagging data is still very low.

Comparisonwith the baseline. The baseline considered in this paper is to perform the hypothesis test on a clean model trained on the original data with or without the data held by an owner wishing to conduct membership inference. As we can see in Table 5, there is no obvious difference between the ASR of the target model when trained on the original data with or without the data held by an owner in the baseline. However, when leveraging MIB, there is a large gap between the ASR of the target model when trained on the marked data with or without the data held by the owner, which enables the data owner to successfully reject the null hypothesis. Our method has a significant improvement against the baseline.

6 Discussion

An adaptive unauthorized party may adopt different approaches to prevent a data owner's membership inference. The unauthorized party can leverage backdoor defense such as Neural Cleanse [Wang et al., 2019] to mitigate backdoor attacks and prevent a data owner from implementing MIB. However, we argue that the data owner can adopt more powerful backdoor techniques than the technique we adopted from BadNets [Gu et al., 2019], as backdoor attacks are developed rapidly [Li et al., 2020]. Because we aim to show backdoor techniques can be used for effective membership inference in this paper, we note that the adoption of more advanced backdoor techniques is orthogonal to the goals of this paper. The unauthorized party may consider another defense technique called differential privacy (DP) [Dwork et al., 2006] to defend against MIB, as DP has been widely used to mitigate MIAs [Hu et al., 2021]. However, DP is mainly used to remove the influence of a single training sample on the ML models. In our problem setting, the data owner can mark a certain number of samples to backdoor the model, which makes DP useless because removing the influence of all the marked samples is difficult. Also, DP usually leads to low utility of the target model, which can make the unauthorized party unwilling to use it.

7 Conclusion

In this paper, we propose a new membership inference approach called membership inference via backdooring (MIB), which allows a data owner to effectively infer whether her data was used to train an ML model or not by marking a small number of her samples. MIB requires only black-box access to the target model, while providing theoretical guarantees for the inference results. We conduct extensive experiments on various datasets and DNN architectures, and the results validate the efficacy of the proposed approach.

Acknowledgments

Dr Xuyun Zhang is the recipient of an ARC DECRA (project No. DE210101458) funded by the Australian Government.