Zero-Shot Machine Unlearning with Proxy Adversarial Data Generation

1 Introduction

The success of modern AI systems relies heavily on largescale datasets [Krizhevsky et al., 2012; Xue et al., 2023; Rajendran et al., 2024]. However, well-trained models are known to memorize their training data [Feldman, 2020], making them vulnerable to privacy attacks wherein adversaries can infer sensitive information by crafting sophisticated queries [Fredrikson et al., 2015; Shokri et al., 2017]. To address these risks, legislatures obligate the model owner to delete users' data upon receiving an unlearning request. Machine unlearning has emerged as a promising paradigm to remove specific data from a well-trained model, effectively erasing their influence as if they were never included in training [Xu et al., 2023].

∗work done while at University of Queensland

† corresponding author

One challenge of machine unlearning is overunlearning [Hu et al., 2024]. The model's performance on the remaining data significantly deteriorates after unlearning. Several approaches have been proposed to address this issue. For example, [Foster et al., 2024] suggest only modifying parameters associated with the unlearning samples to prevent affecting the remaining samples. [Wang et al., 2022; Tarun et al., 2023] fine-tune the unlearned model on the remaining data to recover the performance. However, existing methods require accessing the remaining samples, which renders them inapplicable in practice. We envision two critical requirements for a practical unlearning algorithm: R1 It should work without accessing the remaining data (i.e., zero-shot); R2 It should precisely remove only the targeted samples without leading to over-unlearning. This task is challenging because knowledge is embedded in the model's weights and is highly interconnected [Shwartz-Ziv and Tishby, 2017]. Adjustments to one part of the weights can inadvertently affect other parts. The zero-shot constraint further complicates the challenge.

This paper proposes ZS-PAG (Zero-Shot machine unlearning with Proxy Adversarial data Generation), a novel zeroshot machine unlearning method. In terms of R1, we perturb the unlearning samples over the decision boundaries and use the resulting samples to approximate the remaining samples. For R2, the unlearning should be more localized to the unlearning samples. Inspired by the phenomena that gradients lie in a subspace [Li et al., 2018], we use the generated adversarial samples to approximate the inaccessible remaining samples and identify a corresponding subspace. The unlearning process is projected into the complementary subspace to prevent over-unlearning. In addition, we propose to optimize the pseudo-labels assigned to the unlearning samples using the influence function [Koh and Liang, 2017]. This ensures that unlearning the pseudo-labeled samples will have a positive influence on the remaining samples. In summary, our main contributions include:

• Challenging problem: We consider the zero-shot machine unlearning problem to represent the most practical and challenging setting for machine unlearning.
• Novel method: We introduce a novel three-stage zero-

• shot machine unlearning framework, ZS-PAG, effectively preventing over-unlearning in a zero-shot context.
• Superior performance: Experiments on various benchmarks demonstrate the efficacy of ZS-PAG. For instance, ZS-PAG outperforms the best baseline method by 6.03% on the CIFAR-100 in a zero-shot setting.

2 RelatedWorks

Machine unlearning. Machine unlearning has received increasing attention over the past years [Xu et al., 2023]. Current research can be broadly categorized into three areas: Sample-level unlearning, feature-level unlearning, and classlevel unlearning. Sample-level unlearning focuses on the removal of specific training data points [Bourtoule et al., 2021; Cao and Yang, 2015; Izzo et al., 2021; Guo et al., 2020]. This is particularly relevant in scenarios where an individual requests the deletion of their data. Feature-level unlearning targets the forgetting of particular features extracted from the training data [Warnecke et al., 2021; Li et al., 2023b]. This is useful when sensitive attributes like race or gender inadvertently influence the model. By unlearning these features, the model can mitigate biases while retaining its overall utility. Class-level unlearning concentrates on making the model forget specific classes of data [Chen et al., 2023; Tarun et al., 2023; Golatkar et al., 2020; Fan et al., 2023; Chang et al., 2024]. This is vital when ethical or organizational priorities demand eliminating outdated or harmful classifications. The objective is to remove all knowledge related to that class while maintaining the model's capability to handle other classes effectively.

Mitigating over unlearning. A significant challenge in machine unlearning is over-unlearning [Hu et al., 2024; Shen et al., 2024], where the model's performance significantly degrades after unlearning. Various approaches have been proposed to address this issue. [Chundawat et al., 2023a] employs a teacher-student framework, where the unlearned model is distilled from the original model. During distillation, only knowledge about the remaining samples is retained, while the information related to the unlearning samples is omitted. [Bourtoule et al., 2021] proposes splitting the training data into multiple shards to isolate the influence of specific data points, ensuring their impact does not propagate to the remaining samples. [Foster et al., 2024; Fan et al., 2023] focus on selective model pruning to remove the targeted samples while preserving the model's utility. These methods identify critical parameters essential for general knowledge and selectively prune those linked to the targeted samples, ensuring the model's performance is preserved. [Kurmanji et al., 2024] proposes fine-tuning the original model on both the remaining and unlearning datasets, using gradient negation for the latter to ensure effective unlearning without sacrificing performance.

However, existing methods assume access to the remaining data, which may be invalid. This paper differentiates itself from existing methods by considering unlearning in a zeroshot manner. While some existing works [Chen et al., 2023; Chundawat et al., 2023b; Zhang et al., 2024] also explore zero-shot settings, they face the issue of over-unlearning, resulting in performance degradation on retained samples. In contrast, our method can accurately unlearn the targeted samples without causing over-unlearning. Our work shares a similar intent with [Hoang et al., 2024; Chen et al., 2024; Li et al., 2023a], but several key differences exist. Firstly, the research problems addressed are distinct. This paper focuses on zero-shot unlearning, which presents significant challenges due to the lack of remaining data. Secondly, alongside addressing the issue of over-unlearning, we investigate how unlearning can potentially enhance the model's performance by optimizing the influence.

3 ProposedMethod

3.1 Notations

Let f : Rd ↦→ RC be a classifier parameterized by θ := {wl }Ll=1 , and ril is the input feature w.r.t input xi at the l-th layer. Define θo as an empirical minimizer by fitting f on the training set D = {zi := (xi, yi)}Ni=1 with some loss function ℓ(x, y; θ). The training data D = Du ∪ Dr can be partitioned into two disjoint sets of unlearning data Du and remaining data Dr. Existing methods assume access to both Dr and Du. However, this assumption may be invalid in practice. In this paper, we consider a more stringent setting where we only have access to f(θo) and Du. Figure 1 provides an overview of the proposed framework.

3.2 Proxy Adversarial Data Generation

In zero-shot machine unlearning, we only have unlearning samples. However, it is hard to prevent over-unlearning without knowledge about the remaining samples. As such, our first step is to generate adversary samples as a proxy for the inaccessible remaining samples.

Starting with an unlearning sample xi ∈ Du, we generate an adversary sample xadv = xi + δ via optimizing the additive noise δ. The goal is to deceive the original model into predicting xadv as a pre-specified class ytarget ∈ ¯ c := [C]\yi in a C classification task. The optimization problem is formulated as follows:

The target label is determined by:

where fk(xi; θo) is the k-th component of logits. We use the second high prediction k as the ytarget for the adversary sample. This choice prioritizes efficiency as perturbing the sample towards this class typically requires less modification compared to forcing a more distant prediction. Essentially, we are perturbing xi cross decision boundaries. Repeating this process for each unlearning sample, we end with a set of adversary samples residing in regions of the rest classes, which serve as the basis for the subspace estimation in Section 3.3. Directly solving Eq. 1 is prohibitively hard due to high non-convexity. A plethora of algorithms have been proposed to approximate the solution [Zhou et al., 2022]. We use the method proposed by [Madry et al., 2017] in this paper.

3.3 Unlearning with Orthogonal Projection

Existing research reveals that gradients lie in a lowerdimensional subspace spanned by the inputs [Li et al., 2022; Saha et al., 2021]. Inspired by this, we project the unlearning gradients onto a designated subspace where the gradients are irrelevant to the remaining classes. This design effectively prevents over-unlearning.

We identify the subspace of class k by feeding a batch of adversary samples Bk = {(xi, yi) | (xi, yi) ∈ Dadv, yi = k}nadv i=1 through the original model f(θo). It is expected that using different values of nadv will produce various subspaces and affect the performance of the proposed method. However, as demonstrated in the experiment, setting nadv = 100 is sufficient for our needs. At each layer, we compute the subspace corresponding to class k at layer l as follows:

where Ukl contains the basis of the subspace, Rkl = [rl,1, rl,2, ..., rl,nadv ] represents the input feature maps. The definition of Rkl varies by layer type. For fully connected layers, Rkl is the raw input feature map. In convolutional layers, Rkl is a reshaped input feature map where each row concatenates multiple convolutional patches of the raw input [Liu et al., 2018]. For attention layers consisting of three modules, i.e., (Wq, Wk, andWv), we use the raw input of each module as Rkl and compute a separate subspace for each module.

To prevent over-unlearning, we project the unlearning gradients onto a subspace orthogonal to the subspaces of the remaining classes. Considering unlearn class c, we iterate over the rest classes k ∈ ¯ c to calculate the combined subspace S ¯ c as:

where S ¯ c is initialized as an empty set and S ¯ c l is the l-th element. The projection matrix is defined as Pl = I - S ¯ c l S ¯ cT l . At layer-l, the updating rule is:

The following theorem shows unlearning with Eq. 5 prevents over-unlearning.

Theorem 1. Denote f a neural network parameterized by θ := {wl }Ll=1 , where each wl represents the weights of the l-th layer. Assume the loss function Lr(θ) := 1 |Dr | ∑Dr ℓ(f(xi; θ), yi) has a Lipschitz continuous gradient with constant L and satisfies the Polyak-Łojasiewicz (PL) inequality [Polyak, 1963] with a constant µ. Under suitable conditions on the learning rate, unlearning with Eq. 5 ensures the unlearned model f(θu) will have approximately the same performance as f(θo), i.e.,

3.4 Influence-based Pseudo-label Optimization

In addition to preventing over-unlearning, we also propose optimizing the unlearning process's influence on the remaining samples Dr. Denote θ∗ = argminθ 1N ∑Nn=1 ℓ(zn; θ) as the empirical risk minimizer. After removing a point zi from the training set and the optimal parameters became θ∗-zi = argminθ 1N ∑zn̸=zi ℓ(zn; θ). The influence function gives an efficient approximation to the weight change θ∗ - θ∗-zi .

Definition 1 (Influence function). [Koh and Liang, 2017] When upweighting zi := (xi, yi) by ε, we get new parameter as θ∗ ε,zi = argminθ 1N ∑Nn=1 ℓ(zn; θ) + ϵℓ(zi; θ). The influence of upweighting zi on the parameters θ∗ is given by

whereHθ∗ = 1N ∑Nn=1 ∇2θ ℓ(zn; θ∗ ) is the Hessian of the loss function on the training data.

Assume we have access to Dr for now. Consider upweighting a pseudo-labeled unlearning sample zi := (xi, ypl) by ε.

The change in loss of the remaining sample x ∈ Dr is estimated as:

where the last step is the application of influence function Izi . Replace Ixi with Eq. 7, we get:

Removing a sample (xi, ypl) is equivalent to setting ε = - 1N . Therefore, we approximate the change of loss on Dr after unlearning xi as:

Calculating H-1 θo is costly for DNNs. Therefore, iterative methods [Sattigeri et al., 2022] are often used in practice to approximate H-1 θo ∇θL(Dr, θo).

We random initialize ypl ∈ RC and optimize it to decrease ∆L(Dr, θu) approximated by Eq. 10. However, minimizing ∆L(Dr, θu) alone may lead ypl collapse into the ground-truth label yi. As such, we add a regularization term to penalize the similarity between ypl and yi. This could be done via any similarity metric. Our experiment empirically finds that the Kullback-Leibler (KL) divergence suffices our needs. By optimizing ypl, we ensure the model will benefit from the unlearning process. This results in improved performance after unlearning. In a zero-shot unlearning setting, we approximate Dr with Dadv generated as described in Section 3.2. Our empirical results demonstrate that Dadv serves as an adequate proxy for Dr.

4 Experiment

4.1 Experimental Setup

Dataset and model architecture. Following previous works, we evaluate the proposed method on four benchmarks: Facescrub [Ng and Winkler, 2014], SVHN [Netzer et al., 2011], CIFAR-10 and CIFAR-100 [Krizhevsky et al., 2009]. We apply four representative network architectures in our experiments: AlexNet [Krizhevsky et al., 2012], VGG [Simonyan and Zisserman, 2014], ResNet [He et al., 2015], and ViT [Dosovitskiy et al., 2020].

Baselines. We compare our approach with several baselines. The first five methods assume access to Dr: (1) FT [Warnecke et al., 2021], (2) Neggrad [Kurmanji et al., 2024], (3) BadT [Chundawat et al., 2023a], (4) SalUn [Fan et al., 2023], and (5) Fisher [Golatkar et al., 2020]. To make a fair comparison, we included the results of their zero-shot versions. We also add two baselines that do not use Dr: (6) BU [Chen et al., 2023] and (7) GKT [Chundawat et al., 2023b]. Finally, we include the results of (8) Retrain.

Evaluation metrics and implement details. Following the literature, we assess the unlearned model with three metrics: 1) Accut: Accuracy on the testing set of unlearning classes. In a class unlearning setting, the unlearned model should have zero accuracy on the unlearning classes, matching a retrained model; 2) Accmia: Accuracy of membership inference attack (MIA). We train an attack model to predict the membership of unlearning samples in the training set. As noted by [Fan et al., 2023; Chen et al., 2023], the closer this metric is to that of the retrained model, the better the performance of the unlearning algorithm; 3) Accrt: Accuracy on testing set of remaining classes. We measure the degree of over-unlearning. An idea unlearned model should not decrease Accrt. All results are averaged over three different runs. We utilize projected gradient descent [Madry et al., 2017] to generate adversary samples Dadv in the experiment. Note our method is compatible with any adversary attack strategy.

4.2 Single-class Unlearning

We randomly select one class for SVHN and CIFAR10, two classes for Facescrub, and ten classes for CIFAR100 as the unlearning class(es) and compare our method with the baselines. Results in Table 1 reveal the following key findings:

1) Existing unlearning methods rely on Dr to prevent overunlearning. There exists a significant gap in AccDut between these methods and their zero-shot versions. For instance, on the SVHN dataset, the FT method fine-tunes the original model using a combination of Dr and random labeled samples fromDu. As a result, AccDut of the unlearned model is decreased to 0.95%, while AccDrt remains at 95.97%, which is comparable to the original model. However the AccDrt of FT-zs drops by more than 15.5%. This comparison highlights FT's reliance on Dr to prevent over-unlearning. This conclusion also applies to Neggrad, BadT, and SalUN.

2) ZS-PAG out-stands in zero-shot unlearning. For example, on the CIFAR-10 dataset, AccDut and AccDrt of ZSPAG are 1.40% and 85.47%, respectively. In comparison, the original model yields 63.97% and 83.80%, respectively. ZS-PAG effectively removed the information of the unlearning classes from the model and improved the model's performance in the remaining class. The improvement in AccDrt can be attributed to the optimized pseudo-labels in ZS-PAG, which maximize the influence of the unlearning process on the remaining samples. Notably, ZS-PAG improves AccDrt by 1.53%, 1.67%, and 2.23% on the FashionMNIST, CIFAR10, and CIFAR-100 dataset, respectively.

4.3 Multi-class Unlearning

We extend the comparison to multi-classes unlearning by randomly unlearning 3 classes from the SVHN dataset and comparing ZS-PAG against baselines. As shown in Table 2, the baseline methods exhibit a similar performance drop in AccDrt as seen in Table 1. What's worse, the over-unlearning issue worsens when unlearning more classes. For instance, the gap of AccDrt between BadT and BadT-zs is 17.79% in Table 1 for single class unlearning. And this gap increases to 74.48% when unlearning 3 classes. In contrast, ZS-PAG successfully prevents over-unlearning and demonstrates comparable even higher AccDrt than baselines, despite these methods using remaining samples during unlearning.

Comparing ZS-PAG to the original model across Tables 1 and 2, we observe increasing gains in AccDrt as more classes are unlearned: 0.11% for 1 class and 1.19% for 3classes. We accumulate a greater impact from each unlearning sample on the remaining samples when unlearning more classes, resulting in an improved overall outcome.

4.4 Unlearning Guarantee

MIA. Following the prior arts [Fan et al., 2023; Chen et al., 2023], we use the same setting as Sec. 4.2 and perform MIA against the unlearned model to probe the retained information about the unlearning classes in the unlearned model. Results are presented in Figure 2. Note a large deviation of Accmia from the retrained model may leak information about the unlearning samples. Therefore, Accmia values closer to the retrained model are more desirable. The optimal regions are highlighted in Figure 2. The baselines exhibit Accmia values either significantly higher or lower than that of the retrained model. For example, on the SVHN dataset in Figure 2b, FT achieves the highest Accmia among all unlearning methods. This indicates an unsuccessful unlearning. In contrast, the results of ZS-PAG lie close to the optimal region on all datasets, suggesting a successful unlearning.

Visualize attention map. We plot the GradCAM [Selvaraju et al., 2017] results on samples from the Facescrub dataset to evaluate the effectiveness of ZS-PAG. Two classes were unlearned from the original model to obtain the unlearned model. The results in Figure 3 demonstrate the effectiveness of our unlearning approach in selectively erasing target samples while preserving the model's performance on the remaining data. The first row shows heatmaps for the original model, and the second row corresponds to the unlearned model. For unlearning samples, the original model focuses on the eye regions, which are critical for identity recognition, whereas the unlearned model displays random attention, indicating successful forgetting. For the remaining samples, both models exhibit consistent attention to the eye regions, demonstrating that ZS-PAG preserves the model's functionality on non-target data.

Backdoor attack-based metric. Beyond MIA and attention maps, we additionally employ a backdoor attack-based metric to further evaluate the unlearning effectiveness of ZS-PAG and its robustness against over-unlearning, following the protocol in [Chundawat et al., 2023b]. Specifically, we train a ViT model on the CIFAR-10 dataset, implant a backdoor by adding visual triggers to samples from class 1, and relabel them as class 8 to simulate a targeted attack scenario. We apply ZS-PAG to unlearn class 1 and remove its influence. The results, shown in Tab. 3, demonstrate that the attack accuracy (Accattack) drops dramatically from 89.09 to 0.24 after unlearning. This significant reduction indicates the successful elimination of the backdoor functionality and further confirms the effectiveness of ZS-PAG in erasing sensitive information while preserving the integrity of the remaining model.

4.5 Ablation Studies

In this section, we analyze key components of the proposed ZS-PAG framework. These studies focus on (1) the distribution of adversarial samples, (2) the influence of adversarial attack, and (3) the contributions of subspace projection and pseudo-labeling. The results provide insights into the robustness and effectiveness of the proposed method.

Distribution of adversarial samples. To check how well the generated adversarial samples represent the remaining data, we visualize their distribution in relation to real samples. Using the SVHN and CIFAR-10 datasets with class 3 designated as the unlearning class, we generate adversarial samples to approximate the remaining classes. Figure 4 demonstrates that the adversarial samples closely align with real data distributions, particularly for classes adjacent to the unlearning class. In both datasets, we set class 3 as the unlearning class. The generated samples closely overlap with the real samples, demonstrating a good approximation. This proximity arises from the reduced perturbation required to shift samples to nearby decision boundaries. In comparison, distant classes require larger perturbations. Nevertheless, as shown in the following, a limited number of adversarial samples per class suffices to estimate class-wise subspaces effectively.

Influence of adversary attack. We estimate the inaccessible remaining samples using generated adversary samples in our method. A pertinent question arises: How does the adversary attack affect the effectiveness of our method? We conduct ablation studies using the same settings as in Section 4.2. These studies address this question by examining two aspects: 1) the influence of attack success rate (ASR), defined as the ratio of samples successfully identified as other classes; and 2) the choice of different attack methods.

Influence of ASR: We generate adversarial samples with varying noise bound ε. As shown in Figure 5a, increasing ε results in higher ASR due to larger perturbations. Notably, AccDrt remains stable around 85%, indicating that ZS-PAG is robust to variations in ASR. Additional experiments, as depicted in Figure 5b, confirm that as the number of generated adversarial samples per class nadv increases, the gap between the performance of the subspace-only method (Subspace + RL) and the original model narrows, demonstrating that a sufficient number of adversarial samples enhances subspace estimation and mitigates over-unlearning. This observation aligns with the results shown in Figure 5a, where we observe consistent performance across ASR levels ranging from 52% to 99%. In the context of class unlearning, a substantial number of samples are available for adversarial attacks. Consequently, even a low ASR provides enough adversarial sam-

(b)

Table 4: Influence of attack method. We evaluate the impact of different adversarial attack methods on the unlearning performance.

ples to meet our requirements.

Influence of different choices of attack methods: We compare four different attack methods, 1) PGD [Madry et al., 2017] (the default method in this paper), 2) FGSM [Goodfellow et al., 2015], 3) CW [Carlini and Wagner, 2017], and 4) DeepFool [Moosavi-Dezfooli et al., 2016]. Results in Table 4 show that all methods achieve comparable performance in terms of AccDrt , ranging from 84.46% to 85.49%. Minor differences in AccDut are observed. These results demonstrate that ZS-PAG is robust to the choice of attack method. This robustness stems from the fact that ZS-PAG primarily focuses on probing the decision boundaries of the model rather than requiring high-quality adversarial samples. By focusing on the general properties of decision boundaries, our method accurately identifies the subspaces needed for unlearning, regardless of the attack method used.

Contribution of subspace projection. The efficacy of subspace projection in mitigating over-unlearning is analyzed through three experiments: (1) vanilla Random Labeling (RL), (2) RL with a randomly generated subspace (Random Subspace+RL), and (3) RL with the estimated subspace derived from adversarial samples (Estimate Subspace+RL). As shown in Table 5, RL reduces remaining class accuracy AccDrt from from 95.52% to 80.41%, causing significant over-unlearning. Employing a random subspace exacerbates this issue, lowering AccDrt to 59.08%. In contrast, the estimated subspace effectively prevents over-unlearning, maintaining AccDrt at 95.01%. These results suggest that the unlearning gradient must be approached with caution. Mitigating over-unlearning requires identifying a subspace significant to the remaining samples..

Contribution of pseudo-labeling. We compare ZS-PAG with a subspace-only method that excludes pseudo-labeling. As illustrated in Figure 5b, the pseudo-labeling strategy consistently improves remaining class accuracy AccDrt , with performance gains ranging from 1.77% to 3.27%, depending on the number of adversarial samples per class nadv. This improvement underscores the effectiveness of influence-based pseudo-label optimization in enhancing the model's overall performance.

Computational cost of ZS-PAG. We conduct the experiment on an NVIDIA RTX 4090 GPU. We fix the unlearning epochs to 10 for a fair comparison. Tab. 6 indicates the running time and unlearning performance. ZS-PAG uses significantly less running time than Retrain and achieves the best unlearning performance, with the highest AccDrt . Compared to the second-best method, Neggrad, ZS-PAG incurs less computing overhead and achieves better unlearning performance.

5 Conclusion

This paper presents ZS-PAG, a novel approach to zero-shot machine unlearning. ZS-PAG approximates the inaccessible remaining samples with adversary samples and confines the unlearning process within a specified subspace. This effectively prevents over-unlearning. Additionally, ZS-PAG integrates influence-based optimization techniques to enhance the unlearned model's performance further. Our theoretical analysis provides robust support for this approach, and empirical results convincingly demonstrate the superior performance of ZS-PAG over existing methods. Moreover, our method exhibits robustness across various model architectures, underscoring its effectiveness and adaptability.

Acknowledgments

This research is partially supported by NSFC-FDCT under its Joint Scientific Research Project Fund (Grant No. 0051/2022/AFJ)