A Durable Machine Unlearning Framework to Nullify Recall of Sensitive Data on Incremental Training

1 Introduction

Data privacy is an immediate concern in our era of big data and large models. If training datasets contain sensitive information, the parameters of a trained model could inadvertently memorize individual data instances, which could be used by bad-faith actors to retrieve sensitive information from the model. In response to this, regulatory frameworks, such as GDPR [Hoofnagle et al., 2019], explicitly empower users to request the removal of specific sensitive data. In this context, Machine Unlearning (MU) [Nguyen et al., 2025] has been proposed as a principled paradigm that aims to eliminate the influence of a given forgetting dataset from a trained model (TRM) to produce an unlearned model (ULM).

∗Corresponding author

Rapid advances in MU research, while enhancing privacy, have simultaneously revealed potential security vulnerabilities-particularly the risk of privacy breaches through the recovery of erased information. This underscores a significant gap in understanding the potential threats faced by ULMs. Recently, relevant work [Hu et al., 2024; Sui et al., 2025] has emerged, which explicitly explores recall attacks against MU systems. Models always need to be incrementally tuned with newly collected data (see Figure 1), which may pose an unintentional risk of recalling sensitive information from the ULMs if the incremental training dataset contains similar or even the same unlearned samples. As a result, we have to rerun the MU procedure to remove sensitive information from the incrementally trained model (ITM) or even retrain the model from scratch with a refined dataset. Obviously, the additional rerun or retrain procedures cost significant computation time and resources, especially for large models.

The above challenge raises a fundamental question-"Can we introduce a durable machine unlearning mechanism that is capable of nullifying the recall of sensitive information when incremental training?" It is well understood that a model can only internalize information from a sample if that sample contributes correct gradients to updating parameters during training. Conversely, if gradients associated with a subset of samples are systematically suppressed or reversed, then the trained model is fundamentally unable to retain the information specific to those samples. To this end, we devise a Durable Unlearning Enhancement (DUE) framework that enables a post-MU learning mechanism to avoid restoring undesired sensitive information from new datasets when performing incremental training. As a result, ITMs will be free of rerunning MU, and even improve their performance through the proposed framework. As most MU models restrict the scope of the investigation to the area of image classification tasks [Bourtoule et al., 2021; Fan et al., 2024; Jia et al., 2023; Chen et al., 2023; Chen et al., 2024], we correspondingly study the DUE framework with these MU models in this area.

The DUE framework can work with current MU methods, which consists of three key components: (1) Sensitive Sample Recognizer, (2) Sensitive Sample Gradient Suppressor and (3) Sensitive Sample Probability Reverser. These components work together to identify sensitive samples and suppress the influence of their gradients on model updating. Current MU methods for image classification focuses on removing the effect at either the instance or class level. Instancelevel MU methods deal with the request to unlearn information related to an individual or a group of instances [Cha et al., 2024; Foster et al., 2024], while class-level MU methods aim to eliminate the influence of an image class [Graves et al., 2021]. As a result, we design a unified DUE framework for both instance-level and class-level post-MU training.

We summarize our main contributions as follows:

• To our knowledge, this is the first attempt study on how to avoid recalling sensitive information on incremental training, which can effectively avoid data privacy breaches and promote the robustness of the MU study.
• We propose DUE that is a post-MU training framework to avoid restoring unwanted sensitive information from new datasets when performing incremental training.
• We implement the DUE framework with three key components that can identify sensitive samples and suppress the influence of gradients to update the parameters of ULMs.
• We conduct extensive experiments on four real datasets and multiple state-of-the-art MU methods, showing that the proposed DUE framework can effectively nullify the recall of sensitive information after MU, and even improve the performance of ULMs.

2 RelatedWork

2.1 Machine Unlearning

The gold standard for exact unlearning is to retrain the model from scratch after removing specific data. Given the high cost of exact methods [Bourtoule et al., 2021], research has shifted towards approximate unlearning, which accepts a modest sacrifice in the precision of forgetting for substantial gains in efficiency. Gradient Ascent (GA) [Graves et al., 2021; Golatkar et al., 2020; Thudi et al., 2022; Miao et al., 2024] finetunes the model by maximizing the loss on the forget set, pushing its parameters away from those that memorized the data. Random Labeling (RL) [Golatkar et al., 2020] finetunes the model on the forget set using randomly assigned labels to disrupt and corrupt the learned associations. Influence-based Methods estimate the specific impact of the forget samples on the model's parameters. Unlearning is performed by subtracting this estimated influence, using tools like the Fisher Information Matrix (FF) [Becker and Liebig, 2022] or Influence Functions (IU) [Koh and Liang, 2017; Izzo et al., 2021]. Techniques like ℓ1-Sparse (L1-SP) [Jia et al., 2023] integrate weight sparsity constraints during unlearning, promoting the removal of connections specifically tied to the forget data. Many MU methods degrade model performance, and lead to "over-unlearning". Some recent work has explored more precise unlearning on forget samples. Boundary Unlearning (BU) [Chen et al., 2023] adjusts the model's decision boundary to mimic the behavior of a retrained-from-scratch model, aiming for precise forgetting with minimal collateral damage; SalUn [Fan et al., 2024] employs a "weight saliency" metric to identify and modify only the most critical parameters for forgetting, narrowing the performance gap with exact unlearning; UNSC [Chen et al., 2024] constrains the unlearning update to a null space defined by the remaining data, theoretically guaranteeing that the model's performance on the retain set is preserved throughout the forgetting process. Decoupled Distillation To Erase (DELETE) [Zhou et al., 2025] applies a mask to separate forgetting logits from retention logits to optimize both the forgetting and refined retention components simultaneously.

2.2 Post-unlearning Vulnerabilities

Recent research has revealed significant security and privacy vulnerabilities that persist or emerge after the unlearning process. [Chen et al., 2021] pioneered research on how machine unlearning jeopardizes privacy through membership inference attacks. [Hu et al., 2024] introduced unlearning inversion attacks, extending traditional model inversion attacks to the unlearning context. Their methodology identifies unlearning data leakage in deep neural networks by exploiting the residual information retained in model parameters after data removal. [Zhang et al., 2023] pioneered reconstruction attacks with conditional matching GANs, achieving high reconstruction accuracy by exploiting gradient residuals that persist after approximate unlearning. [Sui et al., 2025] proposed a membership recall attack (MRA), a model-agnostic attack framework, to effectively recover class memberships of forgotten instances from ULMs after unlearning.

Different from the above study of various post-unlearning attacks on ULMs, in this paper we aim to study how to construct a safe post-MU training framework to avoid recalling sensitive information.

3 Preliminaries

We first introduce the datasets and models used in our study, followed by a formal definition of the problem.

3.1 Involved Datasets

Training dataset Dtr : {Xtr, Ytr} denotes the data used to initially train models, where Xtr denotes the image set and Ytr denotes the corresponding label set.

Forgetting dataset Df : {Xf , Yf} is a subset of Dtr, that is, Df ⊂ Dtr. In MU, Df is a set of sensitive samples that should be unlearned, that is, the ULM cannot tell the true labels when Xf is input.

Incremental dataset Di : {Xi, Yi} denotes the new data for incremental training, where Xi contains very similar or even the same images in the forgetting set Xf .

Test dataset Dts : {Xts, Yts} is the unseen dataset for test.

3.2 InvolvedModels

Trained Model (TRM) M(ΘT ) is the model that has been trained on the training dataset Dtr.

UnlearnedModel (ULM)M(ΘU) is the model that has unlearned the forgetting dataset Df based on M(ΘT ).

Incrementally Trained Model (ITM) M(ΘI) is the model incrementally trained on dataset Di based on M(ΘU ).

DUE Model (DUE) M(ΘDUE) is the model incrementally trained on the dataset Di under the DUE framework.

3.3 ProblemFormulation

When the ULMM(ΘU) is incrementally trained on Di, the ITMM(ΘI) tends to recall sensitive information due to the sensitive data subset Ds ⊂ Di. To avoid such data privacy breaches, we aim to implement the DUE that is a post-MU training framework to identify sensitive samples of Ds and re-encode their gradients when updating M(ΘU). As a result, it will lead to a safe ITMM(¯ΘI) in which no information from Ds is retained. Moreover, we discuss the unified implementation of DUE framework on both class-level and instance-level post-MU training.

4 ProposedMethod

4.1 Overview

First, We obtain different ULMs {M(ΘU)} in terms of running various MU methods on a TRM M(ΘT ). Figure 2

shows the architecture of proposed DUE framework, which consists of three key components: (1) Sensitive Sample Recognizer (SSR), (2) Sensitive Sample Gradient Suppressor (SSGS) and (3) Sensitive Sample Probability Reverser (SSPR).

4.2 Sensitive Sample Recognizer

To avoid ULMs {M(ΘU)} restoring sensitive information, it is necessary to recognize sensitive samples when performing incremental training. Given a sample (xi, yi) ∈ Di, we check if it is a sensitive sample, i.e., (xi, yi) ∈ Ds. Here, we consider the ULMs having been removed sensitive information through class-level MU and instance-level MU, respectively.

Class-level Recognition. For class-level MU, it has eliminated the influence of a forgetting class from a TRM {M(ΘT )}. Therefore, we can simply check if a sensitive training sample (xi, yi) whose label yi is in the forgetting class set Cf .

Instance-level Recognition. For instance-level MU, all the samples in the forgetting set, i.e., (x, y) ∈ Df , should be recognized as sensitive data. Furthermore, given a sensitive sample x, we argue that similar images that are transformed (e.g., resize, flip, crop) from x should also be considered as sensitive samples. As a result, we take images Xf of Df as prototype instances, and propose a prototype-based contrastive learning algorithm to train the sensitive sample recognizer.

We use an image embedding model fθE (·), which can be obtained by a copy of TRMM(ΘT ) or a pretrained backbone (e.g. ResNet, ViT), to extract an embedding vector. Given a prototype image xp ∈ Xf , a transformed image xq ∼ xp, and a randomly selected image xn ∈ Xtr, we easily obtain their normalized embeddings.

Then, we set up the following loss:

where the first term is a triplet loss to optimize cosine similarity between xp and xn and that between xp and xq with a margin α in the embedding space; the second term aims to regularize cosine similarity between ep and eq at least β. When fθE (·) is trained, we can obtain the normalized embedding set for all forgetting images, i.e. prototypes, Ef = normalize(fθE (Xf )). Then, we can check if a training sample xt contains sensitive information.

4.3 Sensitive Sample Gradient Suppressor

In the image classification problem, gradient ∇L is derived from the cross entropy loss (CE).

where o denotes logits output and c is the target label.

For any sensitive sample x, i.e., gC(x) = 1 or gI(x) = 1, we need to suppress its gradient to update model parameters. In particular, we create a dummy logit matrix O = aIN where IN is an N × N identity matrix and a is a constant to scale the diagonal (a = 32 is used in this paper). Given a sensitive sample (x, c), we can immediately obtain the dummy logit vector oc = O[c, :]. That is, the probabilities p = softmax(oc) tend to output an approximate one-hot vector, i.e., the probability on c is close to 1. Then, we have the logits selected by g(x) = gC(x) ∨ gI (x).

Accordingly, we obtain the adaptive CE loss (Eq (8)) which tends to be zero when g(x) = 1, i.e., ∇LGS(x, c) = 0. As a result, the gradients associated with sensitive samples are successfully suppressed when updating ULM M(ΘU ).

4.4 Sensitive Sample Probability Reverser

In our experiments, we find that the ULMs can still recall sensitive information after incremental training, even when the above gradient suppressor is applied. This is because the information in forgetting samples could be shared in multiple classes (e.g., bird and airplane might share the background information of sky). As a result, the updating of ULMs on training samples that have features correlated with forgetting samples may indirectly recall the unlearned sensitive information.

In this paper, we additionally design a simple but effective probability reverser to enhance the MU result. Given the logits o = M(x|ΘU ) of sample (x, c), we compute the probabilities in terms of softmin, i.e.,

That is, the smallest logit will produce the largest probability. As a result, the CE loss LGR = CE(-o, c) aims to optimize the parameter to produce the smallest predicted probability on the true-label class. Finally, we have the following adaptive CE loss:

The above CE loss results in the gradient ∇LGR(x, c) to update ULMM(ΘU) towards the reverse direction to produce the highest probability on true label for sensitive samples, i.e., g(x) = 1 while dummy logit vector oc = O[c, :] is applied to generate zero loss, i.e., g(x) = 0.

4.5 Recall Nullified Incremental Training

With the above three components, we can conduct incremental training based on ULMM(ΘU). Given the incremental dataset Di, the loss for mini-batch learning is:

where B ⊂ Di denotes a mini-batch training data. The first term is the average loss to update ULMs with a batch of new samples where the gradient associated with sensitive samples has been suppressed (cf Eq (8)), while the second term is the average loss to enhance MU results in terms of probability reverse (Eq (11)). After incremental training with the proposed framework, we obtain the DUE model M(ΘDUE).

5 Experiments

5.1 Experiment Setup

Data Preparation

In our experiments, four real datasets are used to provide a comprehensive evaluation. CIFAR-10 [Krizhevsky et al., 2009] and STL-10 [Coates et al., 2011] are low-resolution image datasets with 10 classes, which are used for class-level post-MU evaluation. Oxford-IIIT Pet (Pet-37) [Parkhi et al., 2012] and Oxford 102 Flower (Flower-102) [Nilsback and Zisserman, 2008] are high-resolution image datasets, which are used for instance-level post-MU evaluation.

As shown in Table 1, we use the official splits of the training dataset Dtr for initial training, and we further split the official test dataset into incremental training dataset Di and our test dataset Dts for evaluation. Moreover, we select two classes in CIFAR-10 and STL-10 as the forgetting classes, and randomly sampling 10% of data from Dtr in CIFAR10's forgetting classes while 20% of data from Dtr in STL10's forgetting classes to construct the class-level forgetting datasets Df . Furthermore, we randomly sample individual forgetting instances from five classes in Pet-37 and Flower102 as the instance-level forgetting datasets Df . Then, we apply a set of transforms (including random flip, crop, and mixup [Zhang et al., 2018]) based on Df to construct a transformed image set as a part of incremental training dataset Di to simulate the cases of similar image recognition for instance-level DUE as described in Section 4.2.

Model Configuration

The DUE framework is model-agnostic, so we use EfficientNet (EFN) [Tan and Le, 2019] on CIFAR datasets, ResNet [He et al., 2016] on Pet-37, and Swin-Transformer (Swin-T) [Liu et al., 2022] on Flower-102 for a comprehensive study. We use SGD optimizer for MU methods with a momentum of 0.9, a weight decay of 0.005, and AdamW for our DUE framework. Other more detailed settings can be found in the appendix. For each comparison model, we carefully tuned its hyperparameters to achieve optimal performance.

Involved MU Methods

In the experiments, a set of SOTA MU methods, including GA [Graves et al., 2021], RL [Golatkar et al., 2020], FF [Becker and Liebig, 2022], IU [Koh and Liang, 2017], BU [Chen et al., 2023], L1-SP [Jia et al., 2023], UNSC [Chen et al., 2024], SalUn [Fan et al., 2024] and DELETE [Zhou et al., 2025], are involved to comprehensively evaluate the capability of recall nullification by the proposed DUE framework. In particular, UNSC does not support the Swin-T architecture, so the evaluation results are on CIFAR-10 only.

5.2 DUE for Class-level Post-MU Training

Firstly, we train an image classification model with Dtr and obtain the TRM M(ΘT ). Secondly, various MU methods are used to unlearn Df based on M(ΘT ) and obtain the corresponding ULMs M(ΘU). Then, we train each M(ΘU) with Di using or not using the DUE framework for comparison, which leads to the results of DUEM(ΘDUE) and ITM M(ΘI) shown in Table 2.

DUE Performance over Different MU Methods

Table 2 show the accuracy (Acc) on Df , i.e., forgetting set, Df ts , i.e., test set with forgetting classes only, and Dr ts i.e., test set without forgetting classes, respectively. And ∆Acc specify the improvement of DUE based on ULM, i.e., ∆Acc = AccDUE - AccULM

We compared the Acc of ULM and DUE on CIFAR-10 and STL-10 datasets with diverse MU models. From this table, all of the Acc on Df of TRM are 1.000 after training, while they drop to low Acc after MU (cf. the row of ULM). This shows that the MU methods can successfully mitigate the influence of Df from TRM. It is easily found that the Acc of ITMs on both Df and Df i are high, which demonstrates that almost all forgetting information is recalled after incremental training. In comparison, the proposed DUE framework consistently achieves performance comparable to that of ULMs after incremental training. Furthermore, the Acc of DUE on both Df and Df i are even lower than that of ULMs, which proves that DUE can enhance the MU results by successfully suppressing and reversing the gradient of sensitive samples to update ULMs during incremental training. More specifically, the Acc of ULMs on Df with FF, IU and BU are relatively high (above 0.45), which indicates that these MU methods can only unlearn the forgetting samples in part. Obviously, DUE effectively refines the MU results in terms of the Acc, i.e., the significantly negative of ∆Acc indicates the improvement of forgetting.

Furthermore, based on all the Acc between ITM and DUE on the test datasets Dr t s, we find that DUE improves the prediction accuracy on both CIFAR-10 and STL-10 for all MU models. This proves that the proposed DUE is an effective and versatile model-agnostic framework that can not only nullify the recall of sensitive data but also further improve prediction performance through incremental training.

Demonstration of Class-specific Efficacy

To intuitively demonstrate the capacity of DUE to nullify recall of the forgetting samples Xf , we further conducted detailed evaluations with respect to each forgetting class. Figure 3 demonstrate the comparison of the Acc of among ULM, ITM and DUE on CIFAR-10 for each forgetting class for six MU methods . By checking the improvement of Acc for each class, we can observe similar results as shown in Table 2, that is, ULMs can partially remove the influence of forgetting class samples but they are easily recalled by incremental training while DUE can not only nullify the recall but further enhance the forgetting effects.

Figure 4 compares the t-SNE results among ULM (left), ITM (middle) and DUE (right) on the joint image dataset Xp = Xf ⋃ X f i ⋃ X r ts . We observe that significant overlaps between both forgetting-class and non-forgetting-class instances. As a result, although ULMs partially remove the class information from forgetting samples, the inference capacity on non-forgetting-class instances is reduced due to the side effects of MU. In comparison, the boundaries between all classes are much clearer in terms of ITMs, which demonstrates the recall of forgetting-class instances after incremental training. By applying the DUE framework, we can find the boundaries between non-forgetting classes are relatively clear but there are apparent overlaps between forgetting classes.

5.3 DUE for Instance-level Post-MU Training

Analogously to the case of class-level post-MU training, we can easily obtain ULMs, ITMs, and DUEs on instance-level training datasets, Pet-37 and Flower-102.

DUE Performance over Different MU Methods

For the instance-level post-MU training case, we need to precisely unlearn the individual forgetting instances instead of the whole class. Table 3 compares the Acc on the forgetting datasets Df and Df r for various MU models. Similar results can be observed compared to the class-level results shown in Table 2. We find that almost all forgetting samples are recalled, i.e., the Acc is close to 1 through incremental training. After applying the DUE framework, the Acc for each MU method drops to small values, which again proves the efficacy of DUE to enhance MU results by as obtained from incremental training.

Visualization of Confusion Matrices

Figure 5 visualizes the normalized confusion matrices on Dp = Df ⋃ Df i ⋃ Dr ts w.r.t. ULMs (left), ITMs (middle), and DUEs (right). We can observe the highlights in the first column of each confusion matrix for ULM, which illustrates that MU methods have successfully unlearned the class information of forgetting instances. For ITMs, the diagonals of confusion matrices become clearly noticeable, that is, the class membership of forgetting samples has been successfully recalled. In comparison, the confusion matrices of DUE perform the best to eliminate the sensitive information of forgetting samples.

5.4 Ablation Study

In this section, we discuss the effectiveness of three key components of the DUE framework as presented in Section 4.1.

SSR: The component is used to recognize possible sensitive samples similar to prototype instances.

SSGS: The component is used to suppress the gradient of the samples recognized by SSR.

SSPR: The component is used to reverse the probability of the samples recognized by SSR.

Ablation for Class-level DUE

Table 4 reports the results of DUE on Dr ts and Df . From the results, we can easily find that only using either SSGS or SSPR can still partially nullify the recall of sensitive samples. However, as analyzed in Section 4.4, ULMs can probably recall sensitive information if only using SSGS due to the common features between forgetting and non-forgetting samples. As a result, jointly using both SSGS+SSPR achieves the best performance as shown in the last row.

Ablation for Instance-level DUE

Table 5 reports the results of DUE on forgetting sets Df ts and Df . Here, the ablation of SSR component specifies whether to apply the similarity matching based on prototypes or simply perform to exact matching the forgetting samples. Comparing the first case with the last one, we easily find that using SSR can effectively find similar training samples to avoid unlearned information is implicitly recalled. Therefore, using all components SSR+SSGS+SSPR is the most effective way to nullify the recall of sensitive information.

6 Conclusion

This paper initiates a study on how to avoid recalling sensitive information from the training dataset containing unlearned samples when post-MU incremental training is conducted. As a result, the DUE framework is proposed to identify sensitive samples and suppress the influence of gradients to update the parameters of ULMs when incremental training. Extensive experiments on four real datasets and multiple state-ofthe-art MU methods show that the proposed DUE framework can effectively nullify the recall of sensitive information and further enhance MU results. Consequently, this work establishes a new fundamental research direction in safe post-MU training to avoid data privacy breaches and promote the robustness of MU.

Acknowledgments

This work is partially supported by National Key R&D Program of China (2025CSJGG0039 / YDZX20263100004012) and National Natural Science Foundation of China (62276190).

Contribution Statement

Qingqing Cao and Liang Hu contributed equally to this work.