One-Turn Knockout: Traceable and Editable Proxy Unlearning Under Asymmetric Access Constraints

1 Introduction

Currently, personal data, including online posts, browsing histories, and travel records, are imperceptibly collected by various service platforms to train machine learning models such as recommendation systems [Yin et al., 2025; Liu et al., 2024] and scoical networks [Yan et al., 2025]. Along with the promulgation of data privacy regulations, machine unlearning (MUL) which ensures end-users the right to be forgotten attracts ever-increasing attention from both academia and industry. In detail, MUL aims to render the model predictions on the forgotten samples inaccurate without performance degradation on the retention data [Xu et al., 2024].

∗Corresponding Author.

Intuitively, retraining models from scratch based on the retention data, excluding forgotten samples, is straightforward and effective, but it incurs prohibitive computational costs. To mitigate this issue, approximate MUL methods provide efficient alternatives. Golatkar et al.(2020) located the parameters related to the forgotten samples and removed their accumulative gradients. Graves et al.(2021) randomly labeled the sensitive samples and rolled back the parameter updates for their batches. Li et al.(2024) designed mutual evolution strategy to forget. Chen et al.(2025b) adopted model distillation to extract forgotten student models. Despite successfully accelerate unlearning, they still require access to the original training data and model parameters during the unlearning procedure [Zhang et al., 2024; Shen et al., 2024].

However, as Figure 1 illustrated, in most practical scenarios, the model is trained by the model provider (MP) on aggregated data and then exposed to the service operator (SO) in the API-as-a-service manner. Consequently, the SO, who handles the unlearning requests, are not permitted to directly access the parameters and the training data of the target model. We refer to this setting as an asymmetric access constraint [Sun et al., 2022]. Given this constraint, on the one hand, existing MUL methods that depend on direct access to the training data or on operations at model parameters [Yang et al., 2025; Hu et al., 2024] cannot be applied. On the other hand, even though the SO can re-direct unlearning requests to the MP, the induced communication overhead and response latency are unacceptable. Therefore, this paper aims to investigate such a novel research question in MUL:

How to design a novel machine unlearning method for the service operator under the asymmetric access constraint, without the need of repeatedly accessing to the training data and model parameters?

Achieving this goal necessitates a shifted paradigm: we seek to construct a model-agnostic proxy representation space standing in for the parameter space of the target model. With such a proxy, unlearning requests can be handled by the SO without accessing the model or data. However, realizing this paradigm remains non-trivial due to the following challenges.

First, constructing a traceable and editable proxy space is inherently difficult. The principal barrier is the asymmetric access constraint per se, where the SO cannot access the model parameters and the retained data. This immutability rules out parameter-space editing techniques, while the inaccessibility of retained data makes it infeasible to pinpoint and isolate the exact contributions of the data to be forgotten [Golatkar et al., 2020; Foster et al., 2024b; Foster et al., 2024c; Yang et al., 2025]. Consequently, the proxy must be built to faithfully capture the model's knowledge and enable precise, sample-level edits, with limited access to data.

Second, given a traceable and editable proxy, how to surgically erase targeted samples without affecting others remains formidable. An over-aggressive erasure may degrade overall performance on retained data, whereas an insufficient update leaves residual traces of the forgotten samples, compromising the unlearning objective [Zhao et al., 2024]. The mechanism must, therefore, achieve a delicate balance between the forgetting effectiveness and the retained utility, and efficiently handle the sequential unlearning requests over time.

In this paper, we propose One Turn Knockout (OTK), a traceable and editable proxy MUL framework designed for the asymmetric access constraint. Procedurally, OTK consists of two stages. First, in the proxy post-training stage, OTK requires only one-turn access to construct a discrete codebookbased proxy representation to trace the learned distribution of the model. Each token in the proxy encodes the samplewise distributional statistics by uniquely binding to a codeword and records the training contribution via additive statistics updates. Second, in the unlearning stage, OTK erases the causal tokens strongly associated with the forget set via editable proxy, misleading the model's prediction on it without affecting retained knowledge. A theoretical analysis is provided to confirm the forgetting performance and the utility stability. Our contributions are as follows:

• We investigate, for the first time, machine unlearning with the asymmetric access constraint, and propose OTK unlearning framework that does not need to access to the training data and the model parameters.
• We propose the codebook-based space as the editable and traceable proxy for knowledge representation. Furthermore, we customize a series of efficient operations to identify and eliminate influential codes for the target sample.
• Extensive experiments conducted across diverse model architectures and datasets consistently demonstrate the superiority of OTK over existing methods.

2 Notation and ProblemFormulation

Notation. Let Dtrain = {(xi, yi)}Ni=1 and Dtest denote the training and test sets where samples are drawn i.i.d. from a joint distribution PX×Y. Let A : X → Y denotes the trained model where X denotes the input space and Y denotes a generic output space. The predictor Aθ,ϕ is composed of a feature extractor Eθ(·) : X → Z ∈ RD and a task head Hϕ(·) : Z → Y, where θ ∈ Rdθ and ϕ ∈ Rdϕ denote their respective parameters. Following existing work, unlearning methods access the feature extractor and task head through black-box forward queries [Sepahvand et al., 2025].

Problem Formulation. Given a trained model Aθ,ϕ and a designated forget set Df ⊂ Dtrain, the unlearning method U produces an updated model ˜ A = U(Aθ,ϕ,Df). The objective of the unlearning is to remove the influence of Df from the model Aθ,ϕ, while preserving predictive performance on the retained set Dr = Dtrain \ Df [Bourtoule et al., 2021]. In real-world applications, unlearning requests typically arrive sequentially over time [Ye and Lu, 2023; Fraboni et al., 2024]. Formally, consider a sequence of forget sets {D(t) f }Tt=1 , where each D(t) f ⊂ Dtrain is revealed at timestep t. The sequential unlearning process generates a sequence of model states { ˜ A(t) }Tt=0 according to

where the retained set evolves as D(t) r = D(t-1) r \ D(t) f with D(0) r = Dtrain. In this work, we design U to operate under an asymmetric access constraint, eliminating the need for repeated access to training data or model parameters.

3 Methodology

To address unlearning challenge under the asymmetric access constraint OTK is organized into two separated stages, as illustrated in Figure 2. In the first stage (left), our objective is to construct the traceable and editable proxy representation space that circumvents the problem of the service operator (SO) relying on access to the data and model, thereby enabling subsequent efficient unlearning. Specifically, the model provider (MP) inserts a codebook-based proxy module between the frozen encoder and task head, and only the proxy components are optimizable. With only a single access to the original training data, this proxy is post-trained such that both the data distribution and the training contribution of each sample become explicitly traceable in the proxy space. In the second stage (right), the SO handles sequential unlearning requests locally based on this proxy module. For each unlearning request, OTK estimates the distributional contribution of the forget set in the proxy space, identifies the tokens most strongly associated with the target samples, and selectively updates only those tokens to remove their influence, while keeping the rest of the model intact. The following subsections detail the codebook-based proxy representation (§3.1), the one-turn proxy post-training procedure (§3.2), the subsequent selective surgical erasure mechanism (§3.3).

Stage1 How to Construct a Traceable and Editable Proxy? Stage2 Handle Unlearning Requests via Proxy

3.1 Proxy Representation via Discrete Tokens

To enable precise and instantly local unlearning under restricted access, we introduce an explicit proxy representation composed of discrete and independently editable tokens [Yang et al., 2023]. Formally, given the model Aθ,ϕ, we augment it with a model-agnostic codebook placed between the encoder and the task head. The codebook-based proxy module is defined as

where C = {cj}Kj=1 denotes the learnable codebook of K tokens, q(·) is soft assignment operator, f↓ : RD → Rd and f↑ : Rd → RD are projection functions.

Given an input sample x, the encoder produces the representation z = Eθ(x) ∈ RD . Since direct quantization in high-dimensional latent space is inefficient and unstable, we first project z into a lower-dimensional space via ze = f↓(z) ∈ Rd . The projected feature is then softly assigned to the codebook tokens, yielding a proxy representation

where p(j | ze) denotes the assignment probability of ze to token j. Specifically, the probabilities are computed by

where τ controls the sharpness of token assignment. This soft assignment induces a discrete token distribution for each sample, enabling token-level attribution that supports subsequent proxy optimization and editing. Finally, the proxy representation is projected back via ˜ z = f↑(zq) and passed for prediction,i.e., ˆ y = Hϕ(˜ z). Throughout this process, the pretrained encoder Eθ and task head Hϕ remain frozen.

3.2 One-Turn Proxy Post-Training

As shown in Figure 2, the MP performs a one-turn posttraining procedure to optimize the proxy representation space. In this section, we seek to obtain a proxy that (i) stably records training footprints, (ii) maintains the behavior of model, and (iii) supports localized edits.

EMA-based Codebook Update. To enable subsequent unlearning, our key design is to record sample-wise distribution and contribution in the proxy space as additive statistics. We realize this recording by using codebook embeddings as discrete carriers of token-level statistics. Concretely, given a minibatch of samples B ⊆ Dtrain, we compute the soft assignments p(j | ze) (Eq. (4)) and form batch-wise additive statistics as follows:

We then update the statistics with exponential moving average (EMA) using momentum γ ∈ (0, 1) as follows:

Finally, the codeword is updated as following form:

where ε is a small constant for numerical stability.

Interpretation. The accumulated statistic pair (ˆ sj, ˆnj) represents an additive summary of how the training data occupy the proxy space through token j, where ˆnj records the effective mass assigned to token j, and ˆ sj aggregates the corresponding latent contributions. After the proxy post-training stage, the computed statistics {(ˆ sj, ˆnj)}Kj=1 serve as a proxylevel footprint of the training samples over the learned codebook space, which further enables efficient execution of subsequent unlearning operations in section(§3.3).

Objective Design. To construct a traceable and editable proxy space, we propose a disjoint objective to optimize the proxy post-training procedure. The objective consists of three complementary terms. The prediction consistent objective LPred enforces that the proxy induces the same prediction as the original model. Formally, LPred is defined as:

where z = Eθ(x) and ˜ z = f↑(zq). The geometric structure objective LRec is introduced to align the proxy space with the model's latent representation space. Specifically, it encourages the softly quantized proxy representation to remain close to the original continuous embedding:

where zq = ∑Kj=1 p(j | ze)cj. The commitment objective LCom enforces a stable and localized token attribution by encouraging each sample to commit to a small subset of proxy tokens, while simultaneously discouraging redundancy among codewords via a diversity regularizer:

where j ⋆ = argminj ∥ze - cj∥2 is the nearest token index. This term promotes low-entropy, peaked assignment distributions, ensuring that sample-level contributions are concentrated and traceable. The overall objective is given by:

Together, these objectives yield a proxy representation that is traceable and editable, thereby enabling precise and stable unlearning under asymmetric access.

Once the proxy module is post-trained, the MP provides the proxy along with the corresponding APIs to the the SO. This asymmetric deployment allows the SO to perform unlearning operations locally within the proxy space, reducing communication overhead without exposing the training data or model parameters of the MP.

3.3 Selective Surgical Erasure

With the trained proxy module deployed at the SO side, unlearning is carried out by operating directly on codebook embedding and token-level statistics in the proxy space. This procedure contains three steps: (i) estimating token-level statistics of the forget set, (ii) identifying a small set of causal tokens dominated by the forgotten data, and (iii) performing localized subtraction on the selected tokens.

Token-level Contribution Statistics. To perform unlearning, we need to quantify how a given set of samples occupies the proxy space. For this purpose, we consider an arbitrary subset T ⊆ Dtrain, which can represent the forget set, the retained set or the full training set at different stages.

According to the trained proxy, each sample x ∈ T induces a token assignment distribution p(j | ze) and a downsampled representation ze. We summarize the contribution of T in the proxy space using token-level sufficient statistics. Specifically, for each token j ∈ {1, . . . ,K}, we define:

where the expectation is empirically estimated using forward queries of the trained proxy. Collectively, these statistics form

which retains the full contribution information required for codeword rollback. The assignment masses further induce a token distribution

which provides a token-level assignment profile of T and is used for causal token identification. Prior to any unlearning requests, we initialize the system with the trained proxy (A,C (0) ) and set the initial retained set as D(0) r = Dtrain, together with its corresponding statistics Q(0) r and P (0) r .

Causal Token Selection. Consider the t-th unlearning request with forget set D(t) f with its token distribution P (t) f . Since the proxy attribution is additive, the token distribution of the updated retention set can be recursively maintained as

where N (t) f = |D(t) f | and N (t) r = N (t-1) r -N (t) f . To identify which tokens are primarily responsible for encoding the forget set, we define a token j to be causal if its relative contribution F (t) j exceeds a threshold δ:

This criterion isolates tokens whose attribution mass is dominated by the forget set, leveraging the structural separability encouraged during proxy training.

Surgical Erasure. After identified the causal token set J (t) f , we then remove the forget set's contribution from the proxy space. Based on the additive statistics Q(t) f estimated from the forget set, we update the retention statistics as

For each causal token j ∈ J (t) f , we then update the corresponding codeword via

with all non-causal codewords remain unchanged, i.e., c(t) j = c(t-1) j for j /∈ J (t) f and C (t) = {c(t) 1 , c(t) 2 , . . . , c(t) K }.

By subtracting the contribution of the forget set from the selected proxy tokens, we directly remove its influence from the proxy statistics. This operation modifies the proxy representation consumed by the frozen backbone, resulting in localized changes to the model's predictions. Crucially, unlearning is achieved without updating model parameters or accessing the original training data.

4 Experiment

In this section, we conduct extensive experiments aiming to answer the following questions.

• RQ1: What is the trade-off between forgetting effectiveness and retention performance achieved by OTK?
• RQ2: How well does OTK maintain model performance under sequential unlearning requests ?
• RQ3: What behavioral changes are induced by OTK after performing unlearning?

4.1 Experimental Setup

Datasets and Backbones. We conduct evaluations across diverse model architectures on seven datasets from various domains. In detail, for image classification, the unlearning target models include MobileNet [Li et al., 2022] and ResNet [He et al., 2016] of various scales trained on CIFAR-10, CIFAR-20, CIFAR-100, Flowers and Pets, in line with [Chundawat et al., 2023]. To verify the flexibility of OTK across domains, we further evaluate the performance on additional tasks, including question answering on Instruct GPT [Ouyang et al., 2022] and image generation benchmarks. For image generation, the target models contain trained VQ-VAE [Van Den Oord et al., 2017] and VQ-GAN [Feng et al., 2025; Yang et al., 2025] on Digits and LFW-face dataset.

Baselines. We evaluate OTK against three categories. (i) Critical anchors: unaltered original model (Baseline), Retraining (Retrain) as the gold standard, naive Fine-tuning

(Finetune) as the forgetting lower bound. (ii) Fisher Information Matrix based methods: Fisher [Golatkar et al., 2020], SSD [Foster et al., 2024b] and LFSSD [Foster et al., 2024c]. (iii) Re-optimization based methods: Lipschitz [Foster et al., 2024a] , amnesiac [Graves et al., 2021] (component-wise retraining), NegGrad [Golatkar et al., 2020] , BlindSpot [Tarun et al., 2023b] and UNSIR [Tarun et al., 2023a]. All experiments follow the same training and evaluation protocols, including identical train/test splits, consistent hyperparameter settings for each unlearning method, and five independent runs to report mean and standard deviation.

Metrics. We define the unlearning scenarios following SSD [Foster et al., 2024b]. In the full-class unlearning setup, the request targets all samples of the "truck" class in CIFAR10. In the sequential unlearning scenario, we construct a sequence of consecutive forgetting requests on CIFAR-100 and Flowers, each request removing one distinct class in turn. We report the accuracy on the forgotten classes Cf , the retained classes Cr, the overall test set Ct, and the membership inference attack (MIA) success rate. Following prior works, Retrain serves as the gold-standard reference, where an ideal unlearning method should minimize accuracy on forgotten classes and maximize retention performance while matching the MIA behavior of Retrain as closely as possible.

4.2 Forgetting-Retention Performance (RQ1)

To answer RQ1, we perform a comprehensive comparison across multiple datasets and backbones.

Robustness across backbone architectures. Table 1 reports full class unlearning results on CIFAR-10 using backbone networks with varying depth and parameter scale, ranging from lightweight models (e.g., MobileNet) to deep architectures (e.g., ResNet152). Across all backbones, OTK consistently maintains balance between forgetting effectiveness and retention performance. On smaller and shallower models, OTK achieves complete forgetting of the target classes while preserving, and in some cases slightly improving, retainedset accuracy. On average, OTK improves retained accuracy by approximately 0.33% compared to the original baseline, indicating that proxy-based editing does not introduce additional degradation even under limited model capacity.

Generalization across data distributions. Table 2 presents random class unlearning results on ResNet34 across datasets with different data scales and class granularities. On challenging datasets such as CIFAR-100 and Pets, which involve a larger number of classes and finer-grained visual distinctions, OTK demonstrates stronger generalization behavior. In particular, OTK achieves higher retained-set accuracy compared to most baselines, with an average improvement of up to 1.9% on the test set. These results indicate that OTK adapts well to complex decision boundaries.

Comparisonwith existingmethods. When comparedwith existing unlearning approaches, a clear trade-off pattern emerges. Methods that emphasize aggressive forgetting often achieve low Cf or MIA scores, but suffer from pronounced degradation in retained accuracy and test performance, indicating impaired generalization. In contrast, methods designed to preserve overall utility tend to maintain higher Cr and Ct, yet leave detectable traces of the forgotten data, as reflected by non-zero forgetting metrics.

4.3 Sequential Unlearning Performance (RQ2)

To answer RQ2, we evaluate the retained-set accuracy as a function of the number of unlearning rounds on (a) CIFAR100, (b) Flowers, and (c) Pets using ResNet-50 backbones.

In each round, a new disjoint subset of training samples is removed from the model. As shown in Figure 3, most baseline methods suffer from rapid performance degradation as unlearning requests accumulate. NegGrad and Lipschitz collapse within the first five rounds, while SSD and Finetune degrade steadily to near-zero accuracy after approximately 8-10 rounds. Amnesic exhibits a slower decline, but still drops from around 80% to 0% after 12 rounds. In contrast, OTK consistently preserves retained-set performance throughout the entire sequence, decreasing only from approximately 75% to 58% after 15 unlearning rounds, and remains close to the retraining upper bound. OTK is also substantially more efficient per unlearning request. As reported in Table 4, OTK requires only 3.7 seconds per request, compared to 9.7-11.9 seconds for NegGrad and 11.2-676.8 seconds for SSD, depending on model depth. Importantly, the runtime of OTK remains nearly constant across architectures, indicating that its unlearning cost is largely model-agnostic.

4.4 Case Study (RQ3)

We further conduct case studies on LLM-based QA and image generation, to qualitatively and quantitatively validate the generality of OTK beyond discriminative vision tasks.

LLM QA Unlearning. We simulate a personalized QA scenario using GPT-2 with 6,000 user profiles, where public and private information are separated at the attribute [Maini et al., 2024]. We report Forget (accuracy on private attributes), Leak (prompt-based extraction success rate) and Retain (exact match accuracy on public). The Baseline exhibits strong memorization of private attributes, confirming significant privacy leakage while NegGrad completely suppresses both Forget and Leak, indicating that aggressive gradient reversal effectively destroys both private and public knowledge representations. As shown in Table 4, OTK achieves complete forgetting and zero leakage on the private field, while maintaining strong utility, substantially outperforming joint training (0.80) and NegGrad (0.00). OTK achieves a favorable privacy-utility trade-off under different δ values. With δ = 1.5, it matches retraining in privacy removal while preserving high retained performance. Empirically, small δ leads to overly aggressive updates that slightly degrade retention, while δ = 1.5 yields the best balance between forgetting and utility. Further increasing δ to 2.0 weakens unlearning and results in residual memorization, indicating incomplete removal of sensitive associations.

5 RelatedWorks

Selective Unlearning. Machine unlearning aims to remove the influence of specific training samples while preserving the utility of the remaining knowledge. Existing methods mainly follow three paradigms. Retraining-based methods such as SISA [Bourtoule et al., 2021; Schelter et al., 2021] approximate exact forgetting by retraining on retained subsets, but incur substantial computational overhead. To avoid this issue, parameter-editing approaches identify the influence of the target samples and suppress parameters associated with them through weight perturbation, Fisher-based dampening, or local regularization [Chen et al., 2025a; Shibata et al., 2021; Sekhari et al., 2021]. Optimization-based methods further perform gradient rollback, adversarial forgetting, or representation erasure to reduce the influence of target data [Wang et al., 2024; Fan et al., 2024; Liu et al., 2023]. However, they still require access to model parameters, gradients, optimization trajectories, or retained training data during unlearning.

Asymmetric Access Constraints. With the rapid growth of data scale and increasingly constrained real-world deployment environments [Yin et al., 2023], recent studies have begun exploring restricted-access unlearning settings where the unlearning operators may only access deployed models through APIs without exposing model internals or original training data [Sun et al., 2022; Cao and Yang, 2015]. Data-free and restricted-access approaches attempt to relax dependence on retained data through score distillation, synthetic proxy generation [Chen et al., 2025b; Fraboni et al., 2024], or deployment-aware optimization [Zhang et al., 2024; Hu et al., 2024]. Meanwhile, recent scalable unlearning studies further investigate practical challenges such as sequential forgetting, utility preservation, and deployment efficiency under repeated requests [Sepahvand et al., 2025; Kim et al., 2024]. Despite effective, these methods still rely on provider-side coordination, auxiliary optimization, or partial access to model internals, limiting their applicability when both model parameters and original training data are inaccessible [Shi et al., 2024; Gupta et al., 2021].

6 Conclusion

Under asymmetric access constraints, we proposed OTK, a proxy unlearning framework that enables traceable and localized information erasure without data access. By tracking training contributions as additive token-level statistics, OTK supports efficient and precise unlearning through direct proxy edits. Extensive experiments across classification, language, and generative tasks demonstrate that OTK achieves effective forgetting with strong retention and low, model-agnostic runtime overhead. These results highlight OTK as a practical and scalable solution for real-world unlearning deployments. Many future directions could be discussed, particularly extending OTK to large language models, where capturing complex semantics within discrete proxies and exploring adaptive proxy architectures remains a significant challenge.

Acknowledgements

This research was funded by the National Natural Science Foundation of China (No.62572489).