Learning with Selective Forgetting

1 Introduction

Deep learning often suffers from a phenomenon called catastrophic forgetting. When a network is updated for a new task, its performance on previous tasks dramatically degrades. To mitigate this harmful effect, lifelong learning (or continual learning) has been explored, in which the network is updated to adapt to a new task (e.g., a new set of classes and a new instance) without forgetting the results of past learning. Major methods can be categorized into memory-replay-based [Rebuffi et al., 2017; Lopez-Paz and Ranzato, 2017], parameterfreezing-based [Mallya and Lazebnik, 2018; Mallya et al., 2018], and regularization-based [Kirkpatrick et al., 2017; Li and Hoiem, 2017; Zenke et al., 2017; Aljundi et al., 2018] approaches. Most existing methods have been designed to learn a highly expressive model for the new task while preserving all of the knowledge for the previous tasks.

Meanwhile, artificial intelligence is currently facing a new type of problem; as artificial intelligence has become more practical and connected to our everyday lives, various ethical issues such as privacy protection and data leakage prevention have become critical topics. This has brought new challenges to the field, covering learning from encrypted data [GiladBachrach et al., 2016], preventing learning of unintended information [Wang et al., 2019], privacy preserving localization [Speciale et al., 2019a; Speciale et al., 2019b], just to name a few. Even lifelong learning cannot avoid this issue either. Retaining the complete knowledge of all previous tasks is a double-edged sword - it possibly leads to the risk of data leakage and invasion of privacy. Moreover, it is not always necessary to have the complete knowledge of the previous tasks so desirable to have a mechanism for forgetting knowledge no longer needed. For example, a face recognition system at an office entrance gate would not need to remember the faces of staff who have transferred to other departments.

These observations motivate us to propose a new lifelong learning framework called Learning with Selective Forgetting (LSF), which aims to avoid catastrophic forgetting of previous tasks while selectively forgetting only specified sets of past classes. To the best of our knowledge, our study is the first to introduce the new forgetting problem to lifelong learning and proposes a solution to it. In this paper, we focus on task-incremental learning. The challenge is to forget only the specified classes while preventing catastrophic forgetting for the rest without using the original data of the previous tasks. Our method solves this issue by performing a special type of data augmentation that embeds a class-specific signal, called mnemonic code, in all the samples of the corresponding class when updating the model. This makes the class information tightly linked to the corresponding code, making it possible to forget arbitrary classes later on simply by discarding the codes corresponding to the classes. Experiments on common benchmark datasets demonstrate the remarkable superiority of our proposed method to existing approaches.

2 RelatedWork

2.1 Lifelong Learning

We briefly review three mainstream approaches in lifelong learning, memory-replay-based, parameter-freezing-based, and regularization-based, and we highlight the contributions of our work.

Memory-replay: The memory-replay-based approach uses a set of original samples of previous tasks when updating the model for a new task [Chaudhry et al., 2018b; Rebuffi et al., 2017; Lopez-Paz and Ranzato, 2017]. Instead of using the original samples, some methods train deep generative models to generate pseudo samples [Wu et al., 2018; Shin et al., 2017]. Several recent papers have proposed algorithms to solve the problem of data imbalance between the current task and the previous tasks [Zhao et al., 2020; Wu et al., 2019; Liu et al., 2020].

Parameter-freezing: The basic idea of this approach is to use different model parameters for each task. Several strategies have been proposed, such as networks switching the nodes or branches to be used depending on the tasks [Mallya and Lazebnik, 2018; Mallya et al., 2018] or adding new nodes or branches every time a new task is learned [Rusu et al., 2016; Aljundi et al., 2017]. A hybrid version of these approaches has also been proposed [Hung et al., 2019].

Regularization: This approach leverages the previous tasks' knowledge implicitly by introducing additional regularization terms. This approach can be grouped into data-drivenbased [Li and Hoiem, 2017; Hou et al., 2018; Dhar et al., 2019] and weight-constrain-based [Kirkpatrick et al., 2017; Zenke et al., 2017; Aljundi et al., 2018; Chaudhry et al., 2018a; Lee et al., 2017; Yu et al., 2020]. The former utilizes the knowledge distillation, while the latter introduces a prior on the model parameters.

Our method is categorized into the regularization-based approach. To summarize, the existing algorithms are designed to retain all the information of the classes for the past tasks. Unlike these, our contributions of this paper are to propose a new problem setting of lifelong learning that requires forgetting only specified classes and a solution to this problem.

2.2 Machine Unlearning

The concept of Machine Unlearning (MU) was first introduced by Cao et al. [Cao and Yang, 2015]. Its typical definition is to remove the effect of specified training samples without retraining the whole model so that the resulting model is indistinguishable from a model trained on a dataset without those samples. General approaches are to train multiple small models on separated subsets of the training data to prevent retraining the whole model [Bourtoule et al., 2019] or to utilize vestiges of the learning process, i.e., the stored learned model parameters and their gradients [Wu et al., 2020]. Specialized methods for some basic learning algorithms such as linear discriminant analysis [Guo et al., 2020] and k-means [Ginart et al., 2019] have also been presented. Inspired by differential privacy [Abadi et al., 2016], Eternal Sunshine of the Spotless Net [Golatkar et al., 2020] introduced a scrubbing procedure that removes information from the trained weights of deep neural networks using the Fisher information matrix. MixedLinear Forgetting [Golatkar et al., 2021] derived a tractable optimization problem by linearly approximating the amount of change in weights due to the addition of training data. Variational Bayesian inference also provides a compelling approach for MU [Nguyen et al., 2020].

Our work differs from these previous studies in the following two points. First, we focus on lifelong learning. To the best of our knowledge, this is the first work that considers the forgetting problem in the context of lifelong learning. Second, we address the problem of class-level forgetting, i.e., making a specified set of classes unrecognizable, rather than sample-level forgetting. This is a practical forgetting problem that has not yet been thoroughly studied in past MU literature.

3 Learning with Selective Forgetting

Let us begin with an introduction to a standard lifelong learning setting. Denote by {D1, · · ·Dk, · · ·DK} a sequence of datasets, where Dk = {(xik , yik )nk i=1 } is the dataset of the k-th task. xik ∈ X is an input and yik ∈ Y is its class label. While observing the datasets in a streaming manner, the purpose of standard lifelong learning is to learn a model fθ : X → Y parameterized by θ so that it can map a test input x of any learned tasks to its correct class label y.

Now we define our new problem illustrated in Figure 1, called Learning with Selective Forgetting (LSF). In this problem, each of the learned classes is assigned to either preservation set or deletion set. Formally,

• Preservation Set C P k : A set of classes learned in the past and should be preserved at k-th task.
• Deletion Set C P k : A set of classes still memorized and should be forgotten at k-th task (the complement of C P k ).

At the k-th task, we are given the dataset Dk and the preservation set C P k . We use index k for the new task and p for the previous tasks.

Definition 1 (LSF Problem). The Learning with Selective Forgetting (LSF) problem is defined as follows:

• Objective: Learn a model fθ : X → Y. This model fθ should map a test input x to its correct class label y if x is in the preservation set C P . Otherwise, fθ should map x to a wrong class label y′ ̸= y.
• Constraint: No original samples or generative models for the past tasks are available after the new task begins.

4 Method

We propose a method to solve the LSF problem. An overview of our method is shown in Fig. 2. Our method uses a multiheaded network architecture that has one head per task, which is a common architecture in lifelong learning [Li and Hoiem, 2017; Chaudhry et al., 2018a]. We first introduce mnemonic code, which is the key to solving the LSF problem, and then we present loss functions for learning our model. Finally, we empirically analyze the key properties of our method.

4.1 Mnemonic Code

The challenge of the LSF problem is to retain memorizing the classes listed in the preservation set while forgetting those in the deletion set without accessing the original dataset. Our idea aims to associate information of each class with a fairly simple code, called mnemonic code, and to use only that code to control whether the class will be retained or forgotten.

We implement this idea as a special type of data augmentation. An overview of the process is illustrated in the left-hand side of Fig. 2. When a new task is received, one synthetic image is generated per class with random pixel values as the class-specific mnemonic code and embedded in all the samples of the corresponding class. Formally, let {ξk,c} be a set of mnemonic codes, where ξk,c is the code for k-th task of the c-th class. During training for the k-th task, we generate an augmented sample ˜xik by embedding the mnemonic code ξk,c into the original sample xik of the c-th class such like mixup [Zhang et al., 2018]:

where λ is a uniform random variable in [0, 1]. Besides the set of the originals {(xik , yik )nk i=1 }, we also use the augmented samples {(˜xik , yik )nk i=1 } to update our model at the k-th task. Once the updating is done, we retain only the codes {ξp,c} for later tasks to control remembering and forgetting the classes learned in the past tasks.

The intuition behind this procedure is as follows. By training with such augmented data, the samples of the same class are aggregated around the corresponding mnemonic code in the feature space. We therefore can control whether or not to maintain the feature distribution around the code locally depending on whether or not to use the code later at updating the model for a new incoming task, leading it possible to remember or forget arbitrary classes by only using the corresponding codes but without using the original samples (we will show later analytic results in Sec. 4.3). This idea is inspired by a human learning technique called "mnemonics" that aids in memory retention by associating different types of information (e.g., images and words), hence the name.

Implementation of Mnemonic Code: We use random color patterns to generate our mnemonic code as shown in Fig. 2. Specifically, we assign a random color to each grid of an image of the same size as the original sample. Other types of codes are possible; however, we argue several strengths of using such a random code: i) the random pattern can be generated easily, ii) the patterns are i.i.d. for each class and each task, and iii) unlike existing memory-based-approaches that uses (a part of) the original samples, the pattern itself does not directly represent any information of the raw data, which is suitable for privacy protection and data leakage prevention purposes. Interestingly, as we will show later in Sec. 5.3, the performance of our random code is comparable to that of a content-based code, i.e., an average image of the original samples within the same class, which emphasizes the advantages of our version. One limitation so far is that our code has been customized for image data, but the idea itself can be readily extended to other data types, which will be a compelling future research direction.

4.2 Loss Function

We train the model with our mnemonic codes. As shown in Fig. 2, the total loss function L for training consists of four terms: classification loss LC, mnemonic loss LM, selective forgetting loss LSF , and regularization term LR. The first two are for learning a new task and the last two are for maintaining the previous tasks.

Below we detail each of them one-by-one.

Classification Loss LC: The classification loss for the k-th new task is given as

where Nk is the number of the training samples in the k-th task, l(x, y) is a loss function for the input x and its class label y. A typical choice would be softmax cross entropy (CE) or additive margin softmax (AMS) loss [Wang et al., 2018a; Wang et al., 2018b]. We use AMS for l(x, y), as we found it is better than CE.

Mnemonic Loss LM: In addition to the classification loss that uses the original samples Dk = {(xik , yik )nk i=1 }, we also use another loss using the augmented samples with our mnemonic codes ˜Dk = {(˜xik , yik )nk i=1 } for tying each code to the corresponding class. The loss function is given by

We use AMS for l(·, ·) as the classification loss.

Selective Forgetting Loss LSF : The aim of this loss function is to keep remember only the classes in the preservation set and to forget the others in the deletion set. This can be achieved by training with only the mnemonic codes corresponding to the classes in the preservation set and discarding the other codes. For convenience, let us denote by ξ ip the mnemonic code used to generate ˜xip (i.e., the code for the class of xip ). The loss function is

where Np is the number of training samples at the p-th task, and γSF is a balancing weight. l(·, ·) is the AMS function. Note that this loss function does not use any of the original samples. By ignoring the codes of the classes in the deletion set, these classes will experience catastrophic forgetting. This allows us to achieve the selective forgetting for the previous tasks without using any of the original samples.

Regularization Term LR: The regularization term is often introduced to prevent catastrophic forgetting. In this work, we consider using three existing regularization terms, namely, Learning without Forgetting (LwF) [Li and Hoiem, 2017], Elastic Weight Consolidation (EWC) [Kirkpatrick et al., 2017], and Memory Aware Synapses (MAS) [Aljundi et al., 2018]. LwF and EWC are originally designed to retain all classes, while we only need to memorize the classes included in the preservation set for our LWS problem. Thus, we make the following minor modifications to adapt LwF and EWC to our problem. The modified versions are distinguished from their original versions by "*", for example LwF∗ .

- LwF∗ [Li and Hoiem, 2017]: The regularization term of LwF∗ is defined as LLwF∗ = -γ ∑i∈CP k y′o (i) log ˆ y′o (i) , where γ is the weight for the term, and i is the index of the class label. We change the summation to only be taken over the preservation set, i.e., j ∈ C P k . y′o and ˆ y′o are the modified versions of recorded and current probabilities as in [Li and Hoiem, 2017] 1 .

1 The modified versions of recorded and current probabilities, i.e.,

• EWC∗ [Kirkpatrick et al., 2017]: The regularization term of EWC∗ is LEWC∗ = γ2 ∑q,p Fq,p(θq - ˆ θq,p)2 , where γ is the weight for the regularization term, Fq,p is the diagonal component of the Fisher matrix for the p-th previous task corresponding to the q-th parameter ˆ θq,p. We change the Fisher matrix to be evaluated only for the classes corresponding to the preservation set 2 .

- MAS [Aljundi et al., 2018]: The regularization LR for MAS is given by LMAS = γ2 ∑q,p Ωq,p(θq - ˆ θq,p)2 , where γ is the regularization strength, Ωq,p is the constraint strength, i.e., the importance parameter, for the p-th previous task for the q-th parameter, which is estimated by the sensitivity of the squared l2 norm of the function output to their changes.

Beyond the cases of using each of these individually as our regularization term LR, we can also consider combinations of them. Specifically, we test the following two versions of combinations for our method in the experiments.

which we denote OursE and OursM, respectively.

4.3 Analysis

In our preliminary analysis, we demonstrate that our mnemonic code can forget only the specified classes in the deletion set while maintaining the rest.

Setting: We use Permuted MNIST [Kirkpatrick et al., 2017], which is an artificial dataset often used for lifelong learning benchmarks. We prepare three tasks with different permutations; ten digit classes for each task (30 classes total). In this analysis, we always set three classes {"0", "1", '2"} at each task as the deletion set and the other seven classes as the

y′o and ˆ y′o, are given by y′o i = (y′o (i) )1/T /∑j∈CP k (y′o (j) )1/T and ˆ y′o i = (ˆ y′o (i) )1/T /∑j∈CP k (ˆ y′o (j) )1/T , where T is a hyperparameter for the distillation knowledge. We set T = 2 as in the original paper [Li and Hoiem, 2017]. The summation is only taken over the preservation set, i.e., j ∈ CP k .

2 In the case of multiple tasks, EWC requires storing the Fisher matrix for each task independently and performing regularization on all of them together [Chaudhry et al., 2018a].

preservation set, so whenever a new task comes, only {"0", "1", '2"} from the past tasks need to be forgotten, and the rest are required to be retained.

We compare four methods: 1) Vanilla, which only uses the classification loss LC , 2) EWC, 3) EWC∗ , and 4) OursE. The standard two-conv-two-FC CNN is used for all methods 3 .

Results: Figure 3 shows the accuracy vs. epoch plots (top) and t-SNE visualization results (bottom), where the features on the final layer of the shared backbone at the end of each task are visualized. The two accuracy plots show the performance for the deletion set (left) and the preservation set (right). In contrast to the others, we can see that Ours correctly reduces the accuracy of the past three classes to be forgotten and also maintains the accuracy of the seven classes to be retained. Vanilla forgets everything of the past and EWC remembers all. EWC∗ , which has been modified to apply regularization only to classes that should be preserved, tends to manage the task correctly, but is still inadequate. This proves that straightforward modifications of existing methods are not satisfactory and supports the unique effectiveness of our mnemonic code. Another important observation is that the mnemonic code is only embedded in the training images, which leads to a gap between the training and test images, however, this has no significant negative impact on the final classification accuracy.

The t-SNE plots also show that only Ours could keep the samples of the classes to be remembered agglomerated in the feature space and quickly scattered those to be forgotten, which is the desirable behavior for the problem. This implies that our randommnemonic code, despite its simplicity, is able to tightly link the samples of each class to the corresponding code in the feature space, and as a result, can control remembering/forgetting of the classes individually.

5 Experiments

5.1 Setting

Datasets: We use three widely used benchmark datasets for lifelong learning, i.e., CIFAR-100, CUB200-2011 [Wah et al., 2011], and Stanford Cars [Krause et al., 2013]. CUB200-2011 has 200 classes with 5,994 training images and 5,794 test images. CIFAR-100 contains 50,000 training images and 1,0000 test images overall. Stanford Cars comprises 196 cars of 8,144 images for training and 8,041 for testing. Unless otherwise noted, as the analysis in Sec. 4.3, the first 30% of classes for each task belongs to the deletion set, while the other classes belong to the preservation set.

Implementation Details: We used ResNet-18 [He et al., 2016] for the classification model. The final layer was changed to the multi-head architecture as shown in Fig. 2. We trained the network for 200 epochs for each task. Minibatch sizes were set to 128 for new tasks and 32 for past tasks in CIFAR-100, and 32 for new tasks and 8 for previous tasks in CUB-200-2011 and Stanford Cars. The weight decay was 5.0 × 10-4 . We used SGD for optimization. We employed a standard data augmentation strategy: random crop, horizontal flip, and rotation. In this experiment, we used Xavier's initialization.

3 The detailed configuration of the CNN is: Conv(3,32) - Conv(3,64) - MaxPool(2) - Dropout(0.25) - Linear(9216,120) - Dropout(0.5) - Linear(120,10), where Conv(k,c) denotes a convolution-ReLU layer with the kernel size k × k and output channel c, Maxpool(2) denotes max pooling with stride 2, and Dropout(p) denotes dropout with probability p.

Baselines: We compared our proposed method with LwF [Li and Hoiem, 2017], EWC [Kirkpatrick et al., 2017], and MAS [Aljundi et al., 2018], which are the popular regularization-based lifelong learningmethods. In the following experiments, γ for LwF/LwF∗ , EWC/EWC∗ , and MAS are set to 5, 100, and 5, respectively. The weight parameter for the selective forgetting loss γSF is set to 10. We compare the above methods, including their combinations. To sum up, the specific methods compared are as follows:

• Vanilla: Trained using only the classification loss LC,
• LwF: [Li and Hoiem, 2017],

- LwF∗ : Modified version of LwF,

• EWC: [Kirkpatrick et al., 2017],
• EWC∗ : Modified version of EWC,
• EWC∗+LwF∗ : Combination of EWC∗ and LwF∗
• MAS: [Aljundi et al., 2018]
• MAS+LwF∗ : Combination of MAS and LwF∗
• OursE: Our method with EWC∗ and LwF∗
• OursM : Our method with MAS and LwF∗

Evaluation Metric: In our LSF problem setting, the goal is to forget the deletion set and preserve the preservation set. There is no suitable evaluation metric for evaluating the performance on this setting, because it contains a new criterion, selective forgetting. We introduce a new evaluation metric S, called Learning with Selective Forgetting Measure (LSFM).

LSFM is calculated as the harmonic mean of the two standard evaluation measures for lifelong learning [Chaudhry et al., 2018b]: the average accuracy Ak for the preservation set and the forgetting measure Fk for the deletion set, i.e.,

The average accuracy Ak is evaluated for the preservation set after the model has been trained up untill the k-th task. The specific definition is given by Ak = 1k ∑kp=1 ak,p, where ak,p is the accuracy for the p-th task after the training for the k-th task is completed. Ak is evaluated only for the preservation set. Similarly, the forgetting measure Fk is computed for the deletion set after completing the k-th task. This is given by Fk = 1k ∑kp=1 f p k , where f p k = maxl∈1···k al,p - ak,p, which represents the largest gap (decrease) from the past to the current accuracy for the p-th task. This is evaluated only for the deletion set 4 . The ranges of Ak and Fk are both [0, 1].

We report the averages of Sk, Ak and Fk over k after the last task has been completed, which are denoted by S, A, and F, respectively.

4 In the first task (i.e., the number of previous tasks is zero), no class belongs to the deletion set, so Fk and Sk are not defined.

5.2 Comparative Results

Overall Results: Table 1 shows the comparative results of all the methods. We can clearly see that OursE is the best and OursM is the second best among all the methods in SF . No other method that is better in terms of both A and F. This is mainly due to the advantage of our mnemonic codes; as we verified in our preliminary analysis, the codes enable accurate control over whether each class should be retained or forgotten on a class-by-class basis.

Per-Task Results: To visualize the performance changes over the tasks, we show Sk, Ak and Fk at each task in Fig. 4 5 . We can see that OursE consistently achieves the best S on all the datasets. We can draw several observations from the results. First, any single existing method (like LwF, EWC, and MAS) cannot work well. While most of the methods maintain satisfactory performance in terms of Ak, Vanilla, EWC, and EWC∗ suffer from catastrophic forgetting, showing a decrease in accuracy with each new task added. These three methods look better in Fk than the other baselines, however, they forget all the classes whether they are in the preservation set or the deletion set, which is not desirable behavior. Conversely, LwF remembers all the classes, and thus has high Ak but sacrifices Fk. Second, even a combination of the existing methods, namely EWC∗+LwF∗ , cannot yield satisfactory performance. This indicates that a straightforward idea to apply the strong regularization terms only to the preservation set is not sufficient to maintain adequate performance. These observations emphasize the difficulty of maintaining both Ak and Fk together. Unlike these methods, OursE shows consistently high accuracy in both Ak and Fk. This proves the effectiveness of our mnemonic code and learning strategy in overcoming the new problem.

5 Due to space limitations, we report only the results of Vanilla, EWC, LWF, EWC∗ , LWF∗ , and OursE in this figure.

Results for Varying Number of Tasks/Classes: Table 2 shows the results on CIFAR-100 under the various numbers of tasks/classes. Ours is the best in S in all the cases. We also evaluated the performance of the methods when the ratio of the number of classes in the deletion set to that of all the classes, rdel, is varied in the range from 0.1 to 0.9. From the results shown in Table 3, S of our two methods are higher than those of the other methods for all the ratios. These results show the strong robustness of the proposed method to the various settings.

5.3 Sensitivity Analysis

We analyze the sensitivity of the performance to the hyperparameters, including the weight for the selective forgetting loss LSF and the mnemonic loss LM. We also evaluate the performance with different types of mnemonic codes.

Effectiveness of LSF : Figure 5 shows the performance for various γSF in Eq. (5). First, as γSF is decreased, the performance decreases. This suggests that LSF has a significant contribution to improving the performance. For larger γSF , the performance is high and stable, indicating that tuning of the the value is not severe.

Effectiveness of LM: We compared the performance of the proposed method with and without the mnemonic loss LM. The results are shown in the left side of Table 4. It clearly shows that the loss LM significantly improves the performance, demonstrating the effectiveness of LM.

Mnemonic Code Choice: We evaluated the effectiveness of our choice for the mnemonic code, i.e., the random pattern

Table 4: Effectiveness of Mnemonic Loss LM and Mnemonic Code. Left: Effectiveness of the mnemonic loss, Right: Performance with different mnemonic code types.

code, by comparing it with the average code which is the average image of each class. The right side of Table 4 shows the results. We can see that the performance of these two methods is highly comparable. As we discuss in Sec. 4.1, the random code has several advantages compared with the average code. The results further emphasizes the merit of using the random code for lifelong learning with selective forgetting.

6 Conclusion

We opened up a new framework for lifelong learning called Learning with Selective Forgetting (LSF), which allows a model to continuously learn from new tasks while selectively forgetting undesirable class information. Our key contribution was the proposal of a simple and effective idea called mnemonic code. The code is a class-specific random signal embedded in each sample of the same class, which makes it possible to control the remembering and forgetting of the arbitrary class without using the original samples. Thorough experiments proved that our method could achieve significantly better performance than existing methods on this new problem. We believe that this paper will bring a new and practical direction of lifelong learning to the community and give the first baseline for the new problem.