Recent Advances in Concept Drift Adaptation Methods for Deep Learning

1 Introduction

In many real-world tasks, data arrive continuously in a stream manner, such as weather prediction [Elwell and Polikar, 2011], call record analysis [Charles et al., 1977], network intrusion detection [Andresini et al., 2021] and real-time stock market trading data analysis [Hu et al., 2015]. The distribution of these data often changes over time. For instance, the weather prediction rules may vary radically depending on seasons. The characteristics of spam may occasionally change, leading to the redefinition of spam. The above phenomenon is called concept drift. In [Ghomeshi et al., 2019], concept drift is divided into four categories, i.e., abrupt drift, gradual drift, incremental drift and recurrent drift. As shown in Fig. 1, classification performance will deteriorate when concept drift occurs, since classification models usually cannot spontaneously adapt to the new data distribution. To address this problem, a series of concept drift adaptation methods have been proposed to update models online to handle four different concept drift situations.

∗Corresponding author: yuanwei@mail.hust.edu.cn

There have been some good reviews of concept drift adaptation methods [Lu et al., 2020] [Gopu and Godandapani, 2015] [Barros and Santos, 2018]. They mainly consider traditional machine learning models and seldom involve deep learning ones. Inspired by this, we study the recent advances in concept drift adaptation methods developed for deep learning models. In addition to the well-known stability-plasticity dilemma 1 [Ditzler and Polikar, 2013], deep learning oriented concept drift adaptation faces two main challenges: heavier sample shortage and higher latency of model updating. In fact, updating deep learning models requires lots of samples, which is difficult to satisfy in practice. Moreover, updating deep learning models usually consumes longer time and results in higher latency of concept drift adaptation [Jan et al., 2020] [Kantchelian et al., 2013] [Pendlebury et al., 2018]. Up to now, a variety of concept drift adaptation methods have been developed through exploiting the complexity and the flexibility of deep learning models, which have attracted increasing attention from the research community.

In this survey, we thoroughly investigate the existing concept drift adaptation methods for deep learning. Our main contributions are summarized as follows:

1. We analyze mathematically different concept drift situations, and divide the existing adaptation methods for deep learning into two main categories and four subcategories.
2. We examine the representative methods in each subcategory, analyze their pros and cons, and compare their performance on widely used datasets under different concept drift.
3. We propose the main challenges faced by existing studies, and point out the future directions of deep learning model oriented concept drift adaptation studies.

2 Background

2.1 The Definition of Concept Drift

Concept drift is a phenomenon in which the statistical characteristics of the target variable change over time [Baena-Garc et al., 2006]. Suppose X is the feature space and y is the label. Then the emergence of concept drift between times t and t + 1 can be described by

1 Concept drift adaptation should balance the learning focus between the past information and the incoming distribution changes.

where Pt(X, y) and Pt+1(X, y) represent the data distribution at time t and t + 1, respectively.

2.2 The Sources of Concept Drift

Considering Pt(X, y) = Pt(X)Pt(y | X), we attribute concept drift to the following three reasons depicted in Fig. 2.

1. Pt(X) changes and Pt(y | X) remains unchanged. As described in Fig. 2(b), the feature space is accordingly changed, but the decision boundary is not impacted. This is known as virtual drift.
2. Pt(X) remains unchanged and Pt(y | X) changes. As shown in Fig. 2(c), the decision boundary also changes accordingly. This is called real drift. Since we have

the probabilistic sources of drift can be attributed to: (1) label prior drift Pt(y) ̸= Pt+△t(y), and (2) likelihood drift Pt(X|y) ̸= Pt+△t(X|y).

1. Both Pt(X) and Pt(y | X) changes over time. As depicted in Fig. 2(d), virtual and real concept drift exist simultaneously in this case. In practice, this case is more common than the previous ones.

2.3 The Categories of Concept Drift

Concept drift fall into four categories: abrupt, gradual, incremental and recurrent, which are depicted in Fig. 1.

• Abrupt drift. It indicates data distribution changes at a precise point in time, which is defined by the inequality below. For example, some people's hobbies may significantly change suddenly for some reason.

where δ is a threshold defined.

• Incremental drift. It indicates data distribution changes from one to another over a period of time, characterized by the following inequalities. It includes multiple sources, but the difference among sources is very small. Furthermore, incremental drift is not necessarily monotonically increasing.

• Gradual drift. As defined by the inequalities below, gradual drift means both source of samples are active at some point. As time goes by, the probability of sampling from the source c1 becomes lower and sampling from c2 source becomes higher. For example, the function of a device fails occasionally, until a new failure mode completely takes over.

where α(t) = β(c(t)) holds. Here β denotes the Bernoulli distribution. α(t) has a probability c(t) of 0 and 1 - c(t) of 1. c(t) represents an incremental drift. Pt+△t(X, y) will become dominant over time.

• Recurrent drift. It reveals that the data distribution reverts back to the original distribution over time, as shown in the following formula. For instance, some rare weather phenomena, like hurricanes, can occur repeatedly in certain places.

3 Methods

As shown in Fig. 3, we divide the existing concept drift adaptation methods for deep learning into two categories, i.e., model parameter updating and model structure updating.

3.1 Model Parameter Updating

The model parameter updating methods preserve network structure but update the parameters, i.e., weights of neural network, to adapt to concept drift.

Fully Parameter Updating

Fully parameter updating means that all parameters are updated through training model with new data. [Soleymani and Paquet, 2020] proposes a pre-trained Convolutional Neural Networks (CNN) model through initial training over historical data, and then updates model parameters (i.e., weights) with an online learning scheme for the purpose of concept drift adaptation. In [Ryan et al., 2019], model parameters are fully updated with new data and a new meta learning method is introduced to accelerate model retraining.

The authors of [Saadallah and Morik, 2021] leverage a meta learning method to learn the weights of their ensemble model to adapt to concept drift. Adadelta optimizerbased deep neural networks (ADODNN) proposed in [Priya and Uthra, 2021] uses Adadelta optimizer to update network weights, and optimizes hyper parameters to further improve classification performance.

DAREM [Darem et al., 2021] employs an incremental learning method to update model parameters for concept drift adaptation. Since its training data contains both new and old data, this method alleviates the catastrophic forgetting problem. DAREM can be used to handle gradual drift and incremental drift. However, it suffers a slow convergence rate and cannot tackle abrupt drift.

In [Sahoo et al., 2018], the authors focus on DNNs and propose Hedge Back propagation (HBP) to counter concept drift. As described in Fig. 4, this method connects hidden layers to the output layer, each of which represents different concept at different level. It assigns a weight to every hidden layer, and its prediction function can be expressed as

where α(l) is the weight of the l-th layer, f is the output of the l-th layer, and L is the number of hidden layers. Weights can be updated with back propagation to adapt to concept drift.

Furthermore, Recurrent Neural Networks (RNNs) are also studied by existing concept drift adaptation methods. Causal ConvLSTM [Yen et al., 2019] uses BPTT (Back-Propagation Through Time) to update model parameters with recent samples. In [Fekri et al., 2020], the tree-structured Parzen Estimator (TPE) is used to optimize hyper parameters, and the weights of a RNN are then updated for concept drift adaptation. This method neither retains historical data nor requires retraining.

Partially Parameter Updating

Many neural networks adapt to concept drift by using the BPTT method on the latest samples, so that their parameters can be redefined on the latest samples. Unfortunately, these methods may lead to catastrophic forgetting, in which new learning is followed by almost complete forgetting of what was previously learned. A common solution is to update only a part of network parameters to accommodate concept drift, which is called partially parameter updating in this survey.

Selective Ensemble-Based Online Adaptive Deep Neural Network (SEOA) [Guo et al., 2021] selects shallow classifiers from frozen base classifiers and adds them into the ensemble model to adapt to concept drift. When the data fluctuation ratio is small, the base classifier whose weight is lower than a specified threshold is frozen.

[Yang et al., 2019] and [Kirkpatrick et al., 2016] employ the fisher information matrix to prevent the weight of the new task from moving away from the old one. In [Yang et al., 2019], the authors exploit the attention based fisher information matrix to overcome the catastrophic forgetting problem.

In [Disabato and Roveri, 2019], the authors study learning CNNs under concept drift. They take the two-layer hypothesis test [Yu et al., 2019] to find the layer where concept drift occurs first, and then updates the subsequent layers for concept drift adaptation. To overcome the limited data challenge, [Diez-Olivan et al., 2021] generates synthetic data via a kernel density estimation method. And these data can be used to fine-tune the last layer, which helps quickly adapting to concept drift.

It is worth noting that model training is accelerated when only a part of model parameters are updated. Hence the above methods can deal with the abrupt drift.

3.2 Model Structure Updating

The model structure updating methods adjust network structure (e.g., network width or depth) to adapt to concept drift.

Network Width Adjusting

The network width adjusting methods modify network width to accommodate concept drift. The original part of their network remains unchanged, and their model can be incrementally learned by adding new branches or units for the purpose of concept drift adaptation. Here we introduce two key methods for network width adjusting.

Adding Branches. Progressive Neural Networks (PNN) [Rusu et al., 2016] has been proposed as a new network structure for concept drift adaptation. [Budiman et al., 2016] designs an Adaptive Convolutional ELM method (ACNNELM) which enhances CNNs with a hybrid Extreme Learning Machine (ELM) model. Unfortunately, the number of parameters increases sharply with the number of tasks (i.e., concept drift adaptation), and the design of different tasks requires prior knowledge. The highly complex network structure will lead to slow convergence of model training, so they cannot well adapt to concept drift.

To solve the above problem, [Kauschke et al., 2019] constructs a discriminative classifier to identify the area where the classifier is incorrectly classified, and then trains a new classifier (called patch network) on the misclassified data. The patch network takes advantage of the middle layers of the original neural network to extract features and representations that may be critical to classification.

Adding Units. This kind of methods adjusts neural networks by adding some hidden units. Dynamically Expandable Networks (DEN) [Lee et al., 2017] selectively retrains old tasks and adapts to new tasks by adding new hidden units. However, its network complexity increases with the number of learning tasks. To overcome this challenge, Deep Evolving Denoising Autoencoder (DEVDAN) [Ashfahani et al., 2020] chooses to explore the flexible network structure, whose hidden units can be inserted or deleted for the purpose of concept drift adaptation. The authors of DEVDAN point out that Mean Square Error (MSE) has some problems for concept drift adaptation: (1) it calls for all data to understand the reconstruction capability, and (2) it cannot examine the reconstruction power on unseen samples. Therefor, DEVDAN exploits the NS formula to evaluate the generalization power of network structure under the assumption on normal distribution:

where P( ˜X) is the probability density function, X and z are the input and reconstruct, respectively. It can be turned into a bias-variance formula:

where z is a random variable, Bias(z)2 and the V ar(z) can be expressed as (X-E[z])2 and E [z2 ]-E[z]2 , respectively.

In practice, the term Bias(z)2 keeps decreasing with the increase of training sample number. DEVDAN alarms the occurrence of concept drift, once a rise of Bias2 is captured. At this moment, DEVDAN can adapt to concept drift through adding new hidden units or deleting old hidden units. However, DEVDAN relies on the strong assumption that the incoming data follows normal distribution. To relax this assumption, [Pratama et al., 2019b] and [Pratama et al., 2019a] exploit the Autonomous Gaussian Mixture Model (AGMM) to capture concept drift.

Network Depth Adjusting

It is shown in [Eldan and Shamir, 2015] that increasing network depth better enhances the generalization performance of neural networks than increasing network width. Therefore, the network depth adjusting methods usually can better adapt to concept drift. Fast Hoeffding Drift Detection Method (FHDDM) [Pratama et al., 2019c] builds a deep stack network with the feature enhancement method, and adds a hidden layer for concept drift adaptation. The hidden layer is built on the new incoming data between warnings and determinations on the occurrence of concept drift. Each hidden layer has an output that is weighted to get the final label. What's more, FHDDM sets a decaying factor ρd to adjust the weight of each layer:

where π is the step size. When ρd is low, FHDDM can adapt to the gradual or incremental drift, but result in slow adaptation to abrupt drift. If the hidden layer makes a wrong prediction, a penalty is imposed as follows:

where Xd is the prediction of the d-th layer.

Another popular method is Autonomous Deep Learning (ADL) [Ashfahani and Pratama, 2018]. This method combines network width and network depth adaptation. Moreover, ADL utilizes the local error derivative back propagated from the layer most related to the current concept. Thus ADL can adapt to abrupt drift. However, ADL assigns a softmax layer to every layer of neural network, making it limited to classification applications.

Network with Dynamically Evolved Capacity (NADINE) [Pratama et al., 2019d] is proposed to handle concept drift in the non-classification applications. NADINE constructs a Multilayer Perceptron (MLP) model whose network depth can be adjusted to adapt to concept drift. Moreover, NADINE freezes the parameters of some layers and updates the other layers with Stochastic Gradient Descent (SGD), in order to avoid the catastrophic forgetting. Its learning rate can be adjusted adaptively, and the value of its learning rate is related to the correlation between the current concept and the hidden layers. However, the error back propagation used in this method has a major drawback, i.e., the slow convergence of model training.

To accelerate convergence, Stacked Auto Encoder-Deep Neural Network (SAE-DNN) [Zhang et al., 2020] expands the neural network by Random Vector Functional Link (RVFL) structure, and the parameters of the expanded layers are dynamically assigned by the new incoming data with the group lasso regularization and the L2 regularization.

4 Datasets and Evaluation

4.1 Available Datasets

The datasets used in concept drift studies include the synthetic dataset and the real dataset 2 . Representative datasets are given in Table 1, which gives the basic properties about drift types, features, category number and sample amount.

• SEA [Ashfahani and Pratama, 2018]: This dataset contains the samples of two classes, and every sample has three features (or attributes). The relationship between the first two features can be characterized by a binary classification, described as f1 + f2 < θ. In this problem, f1 and f2 are two relevant features, and θ is the threshold value for distinguishing these two classes. Through changing θ from 4 to 7 and then back to 4, abrupt drift and recurrent drift can be implemented.
• Hyperplane [Dong et al., 2015]: This dataset contains the samples of two classes, and every sample has 10 features. The decision boundary is given by a hyperplane ∑dj=1 wjxj = wo. The position and orientation of that hyperplane can be changed by continuously conducting wo = wo +ϵ, where ϵ is a small number. In this way, incremental drift can take place.
• Rotated MNIST [Lopez-Paz and Ranzato, 2017]: This dataset is built by rotating the handwritten digits in the traditional MNIST dataset to arbitrary angles from -π to π. Accordingly, abrupt drift can be introduced into this dataset.
• Permuted MNIST [Ashfahani et al., 2020]: Similar to Rotated MNIST, this dataset is also generated from MNIST. More specifically, every sample in this dataset is obtained by permuting some pixels in an image fromMNIST. In this way, abrupt drift and recurrent drift occur in this dataset.
• LED [Guo et al., 2021]: This dataset contains the data used to predict the numbers on the seven-segment LED display. It contains 24 attributes and 17 of them are irrelevant. It can generate gradual and abrupt concept drift dataset which called LED gradual and LED abrupt, respectively.
• Tree [Brzezinski and Stefanowski, 2014]: The Tree dataset is a fast changing dataset, whose data are generated

2 https://github.com/vlosing/driftDatasets/tree/master

from a random tree generator. This dataset contains 15 abrupt drift over 100000 instances.

The above datasets are all constructed with synthetic data. There exist some datasets for concept drift studies containing real data, e.g., Weather [Elwell and Polikar, 2011], Elec [Wang et al., 2020] and KDDCUP [Yang et al., 2019]. Weather is a real dataset which contains the real data used for weather prediction. Every sample in Weather has 8 features. Elec contains the real data that are collected to predict the electricity price of Australian New South Wales Electricity Market. KDDCUP is composed of the real data that can be used to predict network intrusion events. CoverType [Gama et al., 2003] contains 54 features and 7 classes. It collects the US Forest Service (USFS) Region 2 Resource Information System (RIS) data. These datasets often contain two or more types of concept drift, e.g., abrupt and recurrent drift.

4.2 The EvaluationMetric

The studies on concept drift mainly use the following metrics to evaluate their methods: 1) classification accuracy, 2) execution time (ET), 3) the number of parameters (NoP), 4) the number of hidden layers per time step (NoHL) and 5) the number of hidden nodes per time step (NoHN).

4.3 Performance Analysis

We summarize the performance of representative concept drift adaptation methods in Tables 2 and 3 3 , respectively. For fair comparison, we only consider the methods evaluated on the same datasets in existing literature. We aim to analyze the influencing factors for different types of concept drift.

The partially parameter updating methods can mitigate the problems of limited data and catastrophic forgetting. They are able to adapt to abrupt drift owing to their fast convergence of model training. However, when the new data distribution is remarkably different from the old one, their adaptability will be limited. For instance, the performance of partially parameter updating methods is usually not very good on KDDCUP dataset.

Although the adding branch based methods can adapt to concept drift, their convergence is slow due to their complex network structure. So they cannot handle both gradual and abrupt drift. As shown in Table 3, the ET of these methods (e.g.,PNN) is long. This accounts for why some experimental results of ET (denoted by "-" in Table 3) cannot be obtained after running the corresponding methods for days due to the computational constraint. What's more, the number of their parameters grows rapidly with the increase of the added branches, making them difficult to work in the rapidly changing environment. The adding unit based methods in Table 3 adjust network structure based on NS formula. Therefore, they can adapt to the mixed concept drift (e.g., the concept drift in P. MNIST).

The network depth adjusting methods can deal with the abrupt and recurrent drift through adapting their network depth. Moreover, the flexibility of their network structure makes it possible to cope with gradual drift. As shown in Table 3, the ET of these methods is shorter than that of the adding branch based methods. However, it is difficult for these methods to work in the scenarios where multiple types of concept drift exist simultaneously. For instance, [Pratama et al., 2019c] and [Ashfahani and Pratama, 2018] control the process of weight updating with a decaying factor. When the decaying factor is small, it can adapt to gradual drift, but cannot adapt to abrupt drift. This makes them perform poorly on the real-data datasets (e.g.,Weather) .

3 All experimental results are from the original references.

5 Discussion

5.1 Major Challenges

The research of deep learning model oriented concept drift adaptation is still in its early stage and is faced with the following challenges:

(1) When concept drift occurs, the number of samples with new distribution is limited. However, deep learning requires a large amount of data for model training. Accordingly, deep learning models are unable to rapidly converge and quickly adapt to the new distribution. Under this situation, the performance of deep learning models will inevitably deteriorate.

(2) Existing studies on concept drift adaptation usually deal with only one type of concept drift. However, various concept drift often occur simultaneously in practice. How to use a method to tackle different types of concept drift simultaneously is still an open issue.

5.2 Future Directions

• Relying on fewer samples with new distribution. As mentioned above, a main challenge in concept drift adaptation is the shortage of the data with new distribution. Fewshot learning provides a promising solution to this problem. Few-shot learning is a machine learning method where its training dataset contains limited information. Few-shot learning attempts to build accurate models with less training data. Hence, introducing few-shot learning may help quickly adapting to new data distribution, especially when training data are insufficient.
• Integrating lightweight classifiers to counter concept drift. Ensemble algorithms can reduce the deviation and variance of neural networks. Moreover, they can prevent overfitting by combining multiple individual models. In general, the prediction ability of ensemble classifier is better than that of single classifier. Applying ensemble learning to concept drift adaptation seems to be a promising and interesting direction. A simple way is to adjust the weights of base classifiers to react to new data distribution. One can add new base classifiers trained over new data into the ensemble model, then remove or decrease the weights of outdated base classifiers. In this situation, how to choose lightweight and diverse base classifiers is of great importance. More efforts should be devoted to reducing the computational complexity of this method.

6 Conclusion

How to handle concept drift is an important research topic in deep learning field. In this article, we review the general methods of concept drift adaptation for deep learning models. We summarize the sources of concept drift, analyze the deep learning oriented concept drift adaptation methods, and point out the future research directions. We hope this survey can provide researches with the state art of the knowledge of concept drift in deep learning and provide guidelines for the future.

Acknowledgments

This work was supported partially by the National Natural Science Foundation of China (Grant No. 62172177 and 61571205), and in part by the Fundamental Research Funds for the Central Universities (Grant No. 2021JYCXJJ037).