Recurrent Concept Drifts on Data Streams

1 Introduction

With the emergence of Industry, 4.0, more and more processes are monitored digitally, thus continuously generating tremendous quantities of data [Dreyfus et al., 2022]. Data accessibility enables the implementation of impactful online data-driven machine learning models [Dreyfus et al., 2022]. However, these data-driven models are impacted by concept drifts: input distribution shifts in the underlying data [Dreyfus et al., 2022]. Concept drifts can differ due to impact, transition type, reach of change, recurrence, and blips/outliers/noise [Gunasekara et al., 2023].

The literature describes real and virtual concept drifts considering impact. The former affects the decision boundary of the model. The latter does not influence the decision boundary. Hence, the model is unaffected [Gunasekara et al.,

2023]. Concerning transition type, the drifts are categorized into sudden (abrupt), gradual, and incremental drifts, considering the evolution of the relationship between features and the target and the speed of change. In sudden or abrupt drifts, the current data distribution changes to a new one within a short period [Gunasekara et al., 2023]. For gradual drifts, this transition happens gradually [Su´arez-Cetrulo et al., 2023]. Here, one could observe instances from both distributions for a certain period. The transition time is very long for incremental drifts [Gunasekara et al., 2023]. With recurrent concept drifts, a particular data distribution reoccurs in the stream after a certain period. Recurrent concept drifts could also have different transition types going from one distribution to another. Literature also categorizes concept drifts considering their reach of change: drifts that affect all of the features are regarded as global drifts, and drifts that affect some of the features are called local drifts [Su´arez-Cetrulo et al., 2023]. Random blips/outliers/noise are situations where, for a very short period, few instances do not belong to the current distribution popup in the stream [Su´arez-Cetrulo et al., 2023].

This survey mainly focuses on recurrent concept drifts. Due to the cyclic nature of many physical phenomena, Stream Learning (SL) methods that can handle recurrent concept drifts were used in many real-world applications. A SL method that supports recurrent concept drifts was used to detect air quality using readings of neighboring sensors [Halstead et al., 2022b]. Here, inference in air quality data was complex due to spatial non-linearities and abrupt temporal changes. Spatiotemporal relationships, environmental and contextual features such as meteorological conditions (wind), urban activity (traffic and heater use), and points of interest (locations of factories) were not available for the model. Thus, the model was required to adapt to these recurrent contexts dynamically. Furthermore, SL with recurrent concept drifts support were used in two energy optimization problems [Wu et al., 2023]. The first, tries to optimize between energy storage and energy usage during tariff periods. The second one attempts to predict HVAC (heating, ventilation, and air conditioning) electricity, considering factors such as temperature and humidity. Factors such as seasonality and human activity induce recurrent concept drifts in both cases. In another real-world problem, per-concept models were used to predict the outlet temperature in a Vertical Roller Mill systemwith recurring concepts due to regulating strategies [Sun et al., 2021]. Depending on the regulating strategy, previous concepts could reemerge. In another instance where species activity registered on a sensor that changed depending on the time of the day, custom models related to each species produced better results in determining the species than a generic model [Moreira dos Reis et al., 2018]. Furthermore, recent research on data-driven model maintenance in industry 4.0 highlights the importance of SL on data streams with recurrent concept [Dreyfus et al., 2022].

Considering the above and many real-world applications, this survey focuses on SL for data streams with recurrent concept drifts. The survey by [Su´arez-Cetrulo et al., 2023] mainly focuses on SL in general and explains different methods to handle recurrent concept drifts. However, it did not discuss evaluation methods, benchmark datasets, and software implementations for SL on data streams with recurrent concept drifts. A recent survey [Gunasekara et al., 2023] looks at Online Continual Learning (OCL) from a Stream Learning perspective. Though it refers to [Su´arez-Cetrulo et al., 2023] for the latest SL work for recurrent concept drifts, the main emphasis there is how SL could improve Online Continual Learning. Hence, it misses a thorough investigation of SL on data streams with recurrent concept drifts. This survey attempts to fill the above-mentioned survey gaps by explaining design components, evaluation methods, benchmark datasets, and software implementations for SL on data streams with recurrent concept drifts. The survey also dwells into similar settings such as concept evolution with recurrent classes for SL and Online Continual Learning to find intersections and differences with those research fields.

This paper is structured as follows. First, we provide definitions of recurrent concept drifts. Considering their design components we explain popular SL methods for recurrent concept drifts in the following section. In the subsequent sections, we explore different evaluation methods, open-source software, benchmark datasets and future directions. Concluding remarks are presented in the final section.

2 Problem Statement

Consider an online learning algorithm A learning from an online (potentially infinite) non-Independent and Identically Distributed (IID) data stream. Here, the data stream can be defined as a stream of unknown distributions D = D1,D2, . . . ,DN over X × Y , where X and Y are input and output random variables. The transition of Dj → Dj+1

could happen such that Dj ̸= Dj+1, to result in a concept drift [Gama et al., 2014]. Specifically, it is a drift in the conditional probability distributions, where Pj(y|X) ̸= Pj+1(y|X) [Gama et al., 2014; Gao et al., 2020]. Furthermore, if Dj-k = Dj+1, this new distribution can be considered a recurrence of a k-th previous distribution. Such a drift is identified as a recurrent concept drift in literature [Su´arezCetrulo et al., 2023].

Recurrent concept drifts could also have different transition types going from one distribution to another. Figure 1a illustrates abrupt recurrent concept drifts. Here, the distribution shifts from one concept to the other are abrupt. On the other hand, figure 1b explains incremental recurrent concept drifts, where instances slowly change over to the new concept from the previous concept, and this incremental transition of concepts is present throughout the recurrent drifts. With gradual recurrent concept drifts, concepts change gradually. During the drift period, instances from both concepts appear in the stream for some time. This is illustrated in figure 1c with instances from the blue concept appearing in the stream with the pink concept until it completely switches over to the pink concept. Then, again, instances slowly transition into the blue concept while instances from the pink concept gradually disappear from the stream. Partial recurrent concept drifts are when a given recurrent concept is only seen partially at the subsequent recurrence. For example, in figure 1d, the pink concept is partially present in the subsequent recurrence, while the blue concept is fully present in the next recurrence. Figure 1e explains an evolving recurrent concept drift where a given recurrent concept slowly evolves into a new concept over a period of time. Here, one can see the pink recurrent concept slowly evolving into the orange concept throughout the recurrences.

On the other hand, the recurrence of a given concept could have many forms, depending on the time of recurrence. As shown in figure 2a, recurrences follow a periodic pattern in periodic recurrent drifts. Seasonal weather changes are a good example of periodic recurrent drifts, where each season occurs around the same time of the year. In semi-periodic recurrent drifts, only certain concepts reoccur periodically. For example, in figure 2b, only the blue concept reoccurs regularly, while other concepts follow a random recurrence pattern. The recurrence pattern for each concept is hard to determine in a data stream with random recurrent drifts. This is illustrated with random recurrences for each concept in figure 2c.

Researchers working in the field should be aware of the different intricacies in recurrent concept drifts, considering drift transition type and concept recurrence pattern which is highlighted in figures 1 and 2.

3 Handling Recurrent Concept Drifts

The literature identifies many methods to detect drifts in the input distribution: methods based on differences between two distributions, methods based on sequential analysis and methods based on statistical process control [Gunasekara et al., 2023]. Works by [Khamassi et al., 2018] and [Gama et al., 2014] contain thorough reviews of drift detectors for Stream Learning. Once a drift is detected, they discuss many methods to identify recurrent concepts. Conceptual equivalence assumes when two classifiers behave similarly in prediction for a time window, both classifiers describe the same concept [Yang et al., 2006]. The idea of concept similarity was initially proposed in REDLLA [Li et al., 2012] to detect recurring drifts in the absence of labeled data. Though the authors did not name this as such, the approach aimed to recognize similar concepts using Euclidean distances between concept clusters representing different concepts. Different distance measures for concept similarity are discussed in other research [Angel et al., 2016; Gomes et al., 2013]. RCD [Gonc¸alves Jr and De Barros, 2013] holds a pool of buffers and compares the current buffer against all to find similar concepts using a multivariate nonparametric statistical test.

Managing a concept history [Yang et al., 2006; Alippi et al., 2013], which contains a classifier for each concept, is popular among the research community. This idea of a concept history is described using different terminology like pool of classifiers [Hosseini et al., 2013], concept list [Li et al., 2012] and concept repository [Wu et al., 2021; Wu et al., 2022] in SL for data streams with recurrent concept drifts.

3.1 Explicit Handling of Recurrences

Some SL methods that deal with recurrent concept drifts build a classifier for each data batch and use a repository/ensemble management technique to maintain the repository/ensemble. These methods do not employ a drift detector to detect drifts. LEARN++.NSE trains a new classifier for each batch of data and combines the classifiers using weighted voting [Elwell and Polikar, 2011]. For recurring concepts, it assumes that the weights of learners relating to the recurrent concept will increase, and therefore, the final prediction will consider relevant knowledge. PMRCD also builds a classifier for each data batch. The method maintains a pool of ensembles for each concept [Hosseini et al., 2012]. A maximum number of concepts and classifiers were assumed to handle the memory limitations. The authors proposed a mechanism to manage the classifier pool by merging and splitting concepts. A similar approach of maintaining a classifier pool and selecting the best ensemble was discussed in Dynse [Almeida et al., 2018]. ASE also considered dynamically handling the ensemble size for recurrent concept drifts [Duda et al., 2017]. Considering the current data chunk, it evaluates whether adding an item to the ensemble increases the accuracy for the entire data stream. The method uses Kullback-Leibler discrepancy to measure the suitability of the elements. GraphPool [Ahmadi and Kramer, 2018] maintains a pool of concepts by applying a merging mechanism whenever necessary: after receiving a new batch of data, it extracts a concept representation considering the correlation among features. Then, it compares the current representation to the representations in the pool using a statistical multivariate likelihood test. GraphPool merges all the corresponding concepts if they are similar. Like PMRCD, it also uses Na¨ıve Bayes classifiers. Furthermore, GraphPool maintains the transition among concepts via a first-order Markov chain. This information is used in prediction. It can use either a single classifier or a weighted majority vote among classifiers for prediction.

3.2 Meta Learning

Independently of the number of base learners, some SL methods for recurrent concept drifts act as a wrapper algorithm to determine the best model or models for the current concept after detecting a drift. Thus, they act as metalearners [Su´arez-Cetrulo et al., 2023]. RCD creates a new classifier for each concept and stores a sample of data used to build it [Gonc¸alves Jr and De Barros, 2013]. When it detects a new concept, the algorithm compares the incoming data to previous ones using a non-parametric multivariate statistical test to verify if both contexts come from the same distribution. If so, the corresponding classifier is reused. Concept Profiling Framework (CPF) [Anderson et al., 2016] handles recurring concepts using a classifier pool. It evaluates which model to reuse when a concept drift is detected using conceptual equivalence with classification accuracy. New models are added into the pool in case of a drift and pruned considering their reuse frequency. CPF is shown to be very effective on synthetic data with clear recurrent drifts but fails to outperform RCD on real-world benchmarks [Anderson et al., 2019]. One of the limitations of CPF is that it relies on a fixed buffer size to determine what model to reuse. This makes it slower to react to drift than the other approaches [Anderson et al., 2019]. An enhanced version of CPF was proposed in ECPF [Anderson et al., 2019]. It trains both a new and used classifier on new data and retains the more accurate classifier when concept drift occurs. ECPF creates a copy of the old classifier so that training on new data does not impact the old classifier. PEARL [Wu et al., 2022] utilizes an exact technique and a probabilistic graphical model with Lossy Counting to replace drifted trees in an ensemble with relevant trees from a repository. The exact technique uses pattern matching to find the set of drifted trees that co-occurred in the past for predictions. At the same time, the probabilistic graphical model captures tree replacements among recurrent concept drifts and replaces the exact technique when stable.

3.3 Concept Clusters

Unsupervised concept representations and distance metrics for concept similarity were also used by the SL community working on data streams with recurrent concept drifts to identify recurrences [Li et al., 2012; Wu et al., 2012; Gomes et al., 2012]. In REDLLA [Li et al., 2012], when growing a tree, the k-Means clustering algorithm produces concept clusters and labels unlabeled data at the leaves. The deviation between current and historic concept clusters was used to identify concept drifts and recurring concepts. A similar approach was proposed in SUN [Wu et al., 2012]. There instead of k-Means, authors used k-Modes as the clustering algorithm. A framework using the Context Spaces Model with Context Information was proposed in ContexTrac to represent different concepts [Gomes et al., 2012]. However, they did not elaborate on a method to extract Context Information from a concept. A clustering-based semi-supervised framework, ESCR [Zheng et al., 2021], uses Jensen Shannon divergence on classification confidence score [Haque et al., 2016] to detect recurrent concept drifts. It detects recurring concept drift by looking for any significant change in classifier confidence scores. Then, it determines the possibility of recurring concept drift via Jensen-Shannon divergence by calculating the distance between two confidence score distributions. ESCR performed better than REDLLA and RCD on some synthetic datasets. However, its performance was poor compared to other baselines when the dataset contains irrelevant attributes. CDMSE [Li et al., 2021] works with missing labeled data. There, the predicted class labels by an ensemble model were partitioned into clusters for each data chunk to infer their class labels. Then a concept drift detection method based on the divergence of distributions between adjoining data chunks was used to distinguish recurring concept drifts. The method performed slightly better than SUN and REDLLA on different percentages of unlabeled data. CCP [Katakis et al., 2010] is a very early method that used data stream clustering for SL on data streams with recurrent concept drifts. It proposes a general framework for classifying data streams by exploiting stream clustering to build and update an ensemble of incremental classifiers dynamically. Data stream clustering framework UClust [Namitha and Santhosh Kumar, 2020] was proposed to handle unlabeled data streams with recurrent concepts. Clusters detected through CluStream [Aggarwal et al., 2003] were used to detect drifts and identify concept recurrences. In the CDCMS framework, clustering in the model space was used to build a diverse ensemble and identify recurring concepts [Chiu and Minku, 2020]. The authors argue that diversity accelerates adaptation to different types of drifts when the new concept is similar to the past concepts.

3.4 Drift Prediction

Some methods attempt to predict the next drift or the concept, considering the recurrent nature of the concept's appearance in the stream. These methods try to either proactively influence the drift detection mechanism or re-actively correct the detection signal by the drift detector. MM-PRec [Angel et al., 2016] trains a meta-learner that uses a Hidden Markov Model to predict when a drift will happen and the most suitable concept for each situation if it is recurrent. To measure concept similarity, the authors used a function based on fuzzy logic. The extra computing required to train the meta-model was identified as the method's main drawback. Predictive Change Confidence Function (PCCF) for modeling recurrent changes and predicting change points was derived using the average time between changes and its standard deviation [Maslov et al., 2016].The PCCF models recurrent streams as convolutions of Gaussian distributions of the time intervals between changes. The method can be used to post-process a detection by a drift detector or dynamically adjust the sensitivity of a drift detector. Later, BLPA [Maslov et al., 2017] used PCCF to improve Bayesian Online Change Point Detector (BOCPD) [Adams and MacKay, 2007] for recurrent concept drifts. BOCPD was also used to develop Change Point Recurrence Distribution (CPRD) as an empirical estimate of the recurrent behavior of observed change points [Reich et al., 2019a; Reich et al., 2019b]. ProSeed [Chen et al., 2016] uses a probabilistic network that uses stream volatility patterns to predict future changes. Like PCCF, this method also works independently of the drift detection technique. ProSeed was incorporated into the drift detector SEED [Huang et al., 2014] to yield a proactive drift detector. Experimental results showed that ProSeed performed better than reactive drift detectors for data streams with reoccurring volatility trends. The same approach was used in ProChange [Koh et al., 2018] to improve a drift detector using Hellinger distance to detect virtual drifts and Hoeffding inequality to detect real drifts for unlabeled transactional data. Metadata Drift Predictor (MDP) proposes a dynamically adapting drift detector using drift-related metadata clustering [Anderson et al., 2018]. MDP allows the drift detector to be more sensitive when metadata is similar to past drifts and more conservative when metadata is dissimilar. In their empirical evaluations, MDP performed more accurately compared to ProSeed. Nacre proposes a framework that contains a recurrent drift classifier, a sequence predictor, and a drift coordinator for smooth adaptation of recurrent concept drifts [Wu et al., 2021]. The recurrent drift classifier maintains a concept repository for previously learned concepts. The drift sequence predictor predicts the next drift point based on the previous drift intervals. The drift coordinator manipulates the recurrent drift classifier and the drift sequence predictor to adapt to drifts proactively. Nacre uses PEARL [Wu et al., 2022] as the recurrent drift classifier.

3.5 Meta Features

Apart from performance statistics such as accuracy, error rate, kappa, area under the curve, or classifier confidence score, some methods use different meta-features to improve SL on data streams with recurrent concept drifts. CCP stores the running mean and standard deviation for each numeric attribute and the fraction of instances for each bin in the case of nominal attributes over a sliding window. These features are identified as 'Conceptual Vectors' in CCP. GraphPool takes this idea further by storing a covariance matrix of attributes for each class. Cluster feature vector [Zhang et al., 1996] was used to store information about each cluster in UClust. Authors of FiCSUM [Halstead et al., 2023a] argue that no single meta-feature can fully represent a concept. They propose a general framework for combining various meta-features into a single representation. They propose a method for efficiently computing, storing, and querying an arbitrary metafeature set as a single representation. They also propose a method for dynamic learning of meta-features that distinguishes concepts for a given dataset. According to the authors, FiCSUM enables feature selection methods, such as mutual information, to be applied to concept representation meta-features [Halstead et al., 2023a]. The idea was initially proposed in 2021 however, in the initial version, it was identified that introducing irrelevant meta-features may reduce performance. A new data structures which facilitates feature selection to learn the relevant meta-features was introduced in the later version[Halstead et al., 2023a]. SELeCT [Halstead et al., 2022a] maintains a distinct internal state for each classifier. It uses meta-features described in FiCSUM to describe a state. Considering the previous states, it selects the best classifier for the current state of the stream. SELeCT contains three components: a method for representing a concept as a system state, a method for computing state priors and likelihoods, and a continuous state selection statistical test to select the active state for an incoming observation.

Table 1 summarises how the above-mentioned-design components were used in different SL methods for data streams with recurrent concept drifts. It further shows that some solutions employ many techniques to address the issue, whereas others use only a few. We can also observe that techniques involving clustering and concept meta-features have gained traction in recent years.

4 Evaluation

It is crucial to correctly evaluate a newly proposed method or a drift detector to understand its strengths and weaknesses. Except for typical prequential evaluation and data stream cross-validation along with accuracy, kappa, and other metrics [Gunasekara et al., 2023], novel techniques proposed for SL on data streams with recurrent concept drifts require evaluation methods that demonstrate their suitability for such data streams. The literature explains a few evaluation methods for SL on recurrent concept drifts. They mainly fall into three categories: i) evaluation methods that evaluate the general performance of a method on a data stream with recurrent concept drifts, ii) evaluation methods that evaluate the model selection for each concept on a data stream with recurrent concept drifts, iii) evaluation methods which evaluate drift detection methods for data streams with recurrent concept drifts.

4.1 Evaluating Relative Performance

Other than reporting typical prequential evaluation of a newly proposed classifier A on a data stream with recurrent concept drifts, this evaluation compares the performance of classifier A against a baseline classifier B. Early work proposed a variant of Q statistic [Gama et al., 2013]: Qi = log(Berri /Aerri ), where the index i refers to the time-stamp, and Aerr and Berr refers to the error rate of the classifiers under comparison [Gama and Kosina, 2014]. In more recent work, Cumulative Accuracy Gain (CAG) [Wu et al., 2022]: ∑((accuracy(A) - accuracy(B)) was proposed to evaluate a new method for recurrent concept drifts. CAG represents a sum of accuracy differences against a baseline at a given sample frequency. Apart from accuracy, CAG evaluation could also be applied using kappa statistics. In both the above evaluation methods, the base classifier B selection greatly influences the final evaluation.

4.2 EvaluatingModel Selection for Each Concept

Most of the SL methods proposed for data streams with recurrent concept drifts keep a pool of classifiers/buffers for each concept and attempt to retrieve the most suitable one from the pool for a given concept. To evaluate this, three measurements of context identification are discussed in the literature [Halstead et al., 2021]. Proposed measurements consider concepts that are previously known and their context and model are use as nominal and discrete for a given dataset. Here, context refers to an underlying condition that results in a concept. For example, a stream may be in a given context A for a period of observations and then swap to context B over the subsequent period. At the same time, a system may use model A it has created over a period of time and then swap to model B.

F1c measures the context linkage considering model use in these cases. It measures the strength of the relationship between each < model, context > pair, e.g. < Model A, rain >, using recall (the fraction of rain samples which occurred under Model A) and precision (the fraction of samples under Model A where it was also raining). For given s and c, where s is the time steps a given model was active and c is the time steps a given underlying context was present. The authors calculated the recall(R) and precision(P) of s on c: R(s, c) = |{t|t∈s and t∈c}| |c| , P(s, c) = |{t|t∈s and t∈c}| |s| and further calculated the F1 score: F1(s, c) = 2 R(s,c)P(s,c) R(s,c)+P (s,c) .

The above is used to calculate three transparency measures. i) Average Context F1c: is the maximal F1 score obtained considering the cooccurrence of a single model and each context, averaged across all underlying contexts: F1c = ∑c∈C 1 |C| maxs F1(s, c). For any given underlying context, this measures the approximate average F1 score a single model hopes to achieve. ii) Average System F1s: is the maximal F1 score obtained by considering availability afterward.

Our primary contribution is a method that, given an unlabeled

(context) to be followed by the sample. Contrarily to other statistical

the cooccurrence of each model and a single context, averaged across all models, weighted by length of active time: F1s = ∑s∈S |s| |S| maxc F1(s, c). This measures the average F1 score a model chosen randomly has on its closest context match, given only the system's model history. iii) s ≈ c: the number of models which achieve a recall and precision above 80% on one underlying context. These models have the potential to predict underlying contexts. tests for matching data distribution [10], our method is robust regarding extreme changes in P(Y). Given the unlabeled deployment sample, our method provides not only its context but also an estimate of the current class distribution ˆ P(Y). The utility of such estimate is two-fold. First, it is the direct answer to quanti￿cation queries. Second, we can use it to adjust the decision threshold of the classi￿ers, providing accurate results for applications where the class priors vary signi￿cantly. We make a comprehensible evaluation of our proposal and show that:

According to the authors, F1c and F1s have similar properties to the standard F1 measure, which ranges from 0 to 1. The s ≈ c measure ranges from 0 to the maximum number of underlying concepts present in the stream. A high F1c indicates that changes in concept were successfully identified. In FiCSUM, F1c is identified as C-F1 [Halstead et al., 2023a]. (1) We can identify the correct context under varying class distributions with high accuracy; (2) For quanti￿cation problems, our method is signi￿cantly more accurate than the state-of-the-art under changes in P(X). Additionally, other quanti￿cation methods can bene￿t from our context identi￿cation approach; (3) Our method is more accurate than the state-of-the-art in

4.3 Evaluating Drift Detection on Synthetic Data and total absence of class labels; (4) Our method is signi￿cantly more accurate than a single

data stream classi￿cation with data drifts in P(X ) and P(Y )

MDP explains an evaluation framework for drift detection on synthetic datasets [Anderson et al., 2018]. As one can specify when the true drifts occur, they can be compared to when the drifts are detected. False positive rate FP is the proportion of drifts detected when no drift occurred since the last drift detection. Here, the true positive rate (TP) = 1 - FP. The false negative rate FN is defined as the proportion of true drifts followed by another drift before a drift detection. These measures are compared to highlight the tradeoff between Type I (FP) and Type II (FN) errors in each method. The authors also considered the drift detection delay, the mean number of instances between an actual drift, and the subsequent drift detection. Similar evaluations on drift detection and drift detection delay are also presented in other work as well [Maslov et al., 2016; Huang et al., 2014; Koh et al., 2018]. model that ignores contexts altogether, for both quanti￿cation and classi￿cation; (5) Our proposal is e￿cient and prone to an incremental implementation, ￿tting stream settings. This paper is organized as follows. Section 2 presents a motivation for this research; Section 3 reviews the literature regarding the tackled problem; Section 4 describes our proposal; Section 5 explains our experimental setup; Section 6 presents and discusses our ￿ndings; Section 7 discusses our proposal's e￿ciency; and Section 9 presents our ￿nal conclusions and prospects for future work. 2 MOTIVATION In the last years, our research group has designed a sensor for ￿ying insects [27]. The sensor uses infrared light to capture the movement of the insect wings and machine learning to classify the insect signals into species and sex. This sensor can augment existing mosquito traps, creating a device that can attract, capture, and also count the insects. Hence, such

5 Open Source Software For control, classi￿cation is the primary task. Real-time classi￿cation allows us to create a trap that only captures insects of

trap can be a valuable tool for mosquito control and surveillance.

Most of the methods discussed in section 3 have a GitHub fork of a popular SL framework like Massive Online Analysis (MOA), scikit-multiflow, or River with the implementation. LEARN++.NSE is available as a learner in MOA. PCCF and ProChange are available as separate Python programs. MOA implementations of CPF, ECPF, MDP, and PEARL are available as separate repositories. scikit-ika, which is based on scikit-multiflow, contains implementations of PEARL and Nacre. FALL [Halstead et al., 2023b] is a modular adaptive learning platform based on River. Thus it allows one to use different classifiers and drift detectors fromRiver. It contains modules to calculate some of the meta-features discussed in the section 3.5 such as the ones used in FiCSUM. FALL also implements the selection mechanism and C-F1 context evaluation explained in section 4.2. interest, such as a disease vector or agricultural pest. Other species can be released, reducing the environmental impact of the device. For surveillance, quanti￿cation is the primary task. Counting the number of captured insects by species and sex over time gives us an estimate of the insect population in the trap area. Heat maps

6 Benchmark Datasets

Some of the commonly used synthetic data generators such as SEA, Hyperplane, Agrawal, Random Tree, LED and different concept drift simulators are available in MOA. Although realworld datasets Electricity and Sensor [Angel et al., 2016] are widely used in SL literature for data streams with recurrent concept drifts, their exact concept recurrences are unclear.

P(X)) such as temperature, humidity, air pressure, age, availability

of water and food and so forth. However, temperature is a prominent

and highly variable. It depends on two primary factors: the local

An aggravating factor is that the class distribution is unknown

availability of the species of interest and the insect circadian rhythm. The circadian rhythm is a biological process that governs peaks of activity and resting. For many insect species, these peaks occur at dawn and dusk, as shown in Figure 2. On the other hand, real-world datasets Aedes-Culex, AedesSex, Arabic-Digit, CMC, Handwritten-QG, and Wine contain clear recurring contexts [Moreira dos Reis et al., 2018]. For example, figure 3 shows the different temperature contexts that influence the wing-beat frequency of Aedes aegypti mosquitoes in the Aedes-Culex dataset. FiCSUM uses these contexts to identify resulting concepts. The building electricity demand simulation dataset used in [Wu et al., 2023] also contains recurrent concepts.

7 Future Directions

Considering the recent developments in SL and machine learning in general, this section aims to explore possible intersections of those fields with SL on data streams with recurrent concept drifts.

7.1 Unlabeled Data and Concept Evolution

1984 Section 2 definition of a recurrent concept drift at concept j+1 only considers the recurrent concepts where Pj(y|X) ̸= Pj+1(y|X). On the other hand, concept evolution only considers evolving target variables. Here at j + 1, new set of classes emerge compared to j: Pj(y) ̸= Pj+1(y) [Gao et al., 2020]. Here, the distribution shift is in the label data instead of the input data. Apart from completely new (previously not seen) classes at j + 1, [Masud et al., 2011; Gao et al., 2020] considers concept evolution with the reappearance of any previously seen class that was not available at j. There are only a few works in this area of concept evolution with recurrent classes. Most of the techniques discussed in section 3 could be applied in this setting. Thus, we see this as a natural expansion of current SL on data streams with recurrent concept drifts.

Figure 2: Histograms representing the circadian rhythm of Aedes aegypti and Culex quinquefasciatus mosquitoes. For this application, we possess plenty of labeled training data. We gather these data in the laboratory using insectaries, which are containers with a sensor attached. Each insectary maintains Apart from one recent work [Namitha and Santhosh Kumar, 2020], there has not been much attention paid to unlabeled data streams with recurrent concepts among the research community. Considering that most real-world data is unlabeled, this could be an exciting research direction the SL community working on recurrent concept drifts could peruse.

7.2 Online Continual Learning

Online Continual Learning also considers online learning of tasks with different label/input distributions [Gunasekara et

l

ncremental

y3 y4 y5 y6

t P(X)

ifar10*

ifar100*

* Others

*

g P(X)

ally

iCub

is also able to benefit from low accuracy during training (i.e. this e↵ect improves the metric). Instead, we believe the best CL-algorithm is one that has high accuracy, which can be improved by including new data, without sudden performance drops. Ideally, this would be measured continuously during training, and the area below the accuracy curve used as a metric. Yet practically this has a high computational cost, and testing at well-chosen discrete intervals is a reasonable approximation. For further discussions, see [23, 24, 12]. 3. Continual Learning Benchmark for Autonomous 3.1. SODA10MDataset Both tracks build on the SODA10M dataset [25], which contains 10M unlabelled images and 20k labelled images. Image data consists of dash-camera recorded footage, obtained from vehicles driving through four Chinese cities, with images recorded at 10 second intervals. Ordering images chronologically largely entails visual footage of a car exploring the city and its neighbourhoods. The image label set has bounding box annotations for 6 object classes and covers different 'domains' (cities, weather conditions, time of day and road type) - see Figure 2. See Figure 4 for some examples images (arranged in tasks for CLADD, as will be discussed in Sec. 3.4). While self- and to [22], the best algorithm is one that has the highest final accuracy, however we note that the BWT metric is also able to benefit from low accuracy during training (i.e. this e↵ect improves the metric). Instead, we believe the best CL-algorithm is one that has high accuracy, which can be improved by including new data, without sudden performance drops. Ideally, this would be measured continuously during training, and the area below the accuracy curve used as a metric. Yet practically this has a high computational cost, and testing at well-chosen discrete intervals is a reasonable approxiable data. 3.1. SODA10MDataset Both tracks build on the SODA10M dataset [25], which contains 10M unlabelled images and 20k labelled images. Image data consists of dash-camera recorded footage, obtained from vehicles driving through four Chinese cities, with images recorded at 10 second intervals. Ordering images chronologically largely entails visual footage of a car exploring the city and its final accuracy, however we note that the BWT metric is also able to benefit from low accuracy during training (i.e. this e↵ect improves the metric). Instead, we believe the best CL-algorithm is one that has high accuracy, which can be improved by including new data, without sudden performance drops. Ideally, this would be measured continuously during training, and the area below the accuracy curve used as a metric. Yet practically this has a high computational cost, and testing at well-chosen discrete intervals is a reasonable approximation. For further discussions, see [23, 24, 12]. 3. Continual Learning Benchmark for Autonomous able data. 3.1. SODA10MDataset Both tracks build on the SODA10M dataset [25], which contains 10M unlabelled images and 20k labelled images. Image data consists of dash-camera recorded footage, obtained from vehicles driving through four Chinese cities, with images recorded at 10 second intervals. Ordering images chronologically largely entails visual footage of a car exploring the city and its neighbourhoods. The image label set has bounding box annotations for 6 object classes and covers different 'domains' (cities, weather conditions, time of day and road type) - see Figure 2. See Figure 4 for some examples images (arranged in tasks for CLADD, as will be discussed in Sec. 3.4). While self- and Wanderlust CLOC or continual learning in computer vision, with a focus on classification problems. We hange during training. The yellow circles areas are proportional to how many papers ch benchmark, see Section 2.2 for details. Black circles are recently proposed, more cates a 'split' version of the dataset. 'Others' refers to a set of benchmarks that were marks where multiple datasets with the same labels are used. For more details and to [22], the best algorithm is one that has the highest final accuracy, however we note that the BWT metric is also able to benefit from low accuracy during training (i.e. this e↵ect improves the metric). Instead, we believe the best CL-algorithm is one that has high accuracy, which can be improved by including new data, without sudden performance drops. Ideally, this would be measured continuously during training, and the area below the accuracy curve used as a metric. Yet practically this has a high computational cost, and testing at well-chosen discrete intervals is a reasonable approxiable data. 3.1. SODA10MDataset Both tracks build on the SODA10M dataset [25], which contains 10M unlabelled images and 20k labelled images. Image data consists of dash-camera recorded footage, obtained from vehicles driving through four Chinese cities, with images recorded at 10 second intervals. Ordering images chronologically largely entails visual footage of a car exploring the city and its Wanderlust proposed and common benchmarks for continual learning in computer vision, with a focus on classification problems. We tions X and the available labels Y change during training. The yellow circles areas are proportional to how many papers CVPR '21 and NeurIPS '21 used each benchmark, see Section 2.2 for details. Black circles are recently proposed, more at are currently less popular. '*' indicates a 'split' version of the dataset. 'Others' refers to a set of benchmarks that were aper and 'sequences' refers to benchmarks where multiple datasets with the same labels are used. For more details and oys200 iCub CLAD-C CORe50.NI Wanderlust Clear CLOC chmarks for continual learning in computer vision, with a focus on classification problems. We labels Y change during training. The yellow circles areas are proportional to how many papers OCL considers two settings with different types of distribution shifts: Online Domain Incremental Continual Learning (ODICL) and Online Class Incremental Continual Learning (OCICL). ODICL is similar to SL on data streams with concept drifts where distribution shifts are in the input data [Gunasekara et al., 2023]. On the other hand, OCICL focuses online learning on tasks with different label distributions. It is similar to SL with concept evolution. Considering the similarity in two settings: SL and OCL [Gunasekara et al., 2023] explains how some of the ideas from SL on recurrent concept drifts could be useful in OCL. Specifically, drift detection and prediction to detect and predict tasks, model repository management techniques to manage a pool of Neural Network (NN)s and model selection techniques to select a Neural Network for prediction on instances from a past task. For data streams with recurrent concept drifts, if the start and end points of a concept and its relevant recurrent concept are known, then one can track the performance of recurrent concept explicitly. This process can provide useful insights for each recurrent concept. It can help identify which recurrent concept performs best with which proposed algorithm. Similar to current stream learning evaluation methods, this new evaluation can be applied with accuracy, kappa, and other evaluation metrics discussed in SL literature [Gunasekara et al., 2023]. One could also use a similar metric as forgetting to evaluate the model improvement over different reincarnations of the same concept.

Class Incremental y1 y2 y3 y4 y5 y6

Domain Incremental

X

X

CLOC

Wanderlust Permuted MNIST Rotated y1 y2 y3 y4 y5 y6 Permuted MNIST Rotated MNIST y1 y2 y3 y4 y5 y6 y1 y2 y3 y4 y5 y6 y1 y2 y3 y4 y5 y6 Domain Incremental tion evaluation methods discussed in section 4.2 only consider selecting the correct model for the relevant concept. They do not give us an indication of the model's predictive performance.

CLOC

Domain Incremental

Figure 1: Overview of proposed and common benchmarks for continual learning in computer vision, with a focus on classification problems. We depict how data distributions X and the available labels Y change during training. The yellow circles areas are proportional to how many papers published in ICCV '21, CVPR '21 and NeurIPS '21 used each benchmark, see Section 2.2 for details. Black circles are recently proposed, more realistic benchmarks that are currently less popular. '*' indicates a 'split' version of the dataset. 'Others' refers to a set of benchmarks that were only used in a single paper and 'sequences' refers to benchmarks where multiple datasets with the same labels are used. For more details and explanation of dataset placements, see Appendix. to [22], the best algorithm is one that has the highest final accuracy, however we note that the BWT metric of our two challenge benchmarks that utilise the available data. MNIST Clear X Sequences Figure 1: Overview of proposed and common benchmarks for continual learning in computer vision, with a focus on classification problems. We depict how data distributions X and the available labels Y change during training. The yellow circles areas are proportional to how many papers published in ICCV '21, CVPR '21 and NeurIPS '21 used each benchmark, see Section 2.2 for details. Black circles are recently proposed, more realistic benchmarks that are currently less popular. '*' indicates a 'split' version of the dataset. 'Others' refers to a set of benchmarks that were only used in a single paper and 'sequences' refers to benchmarks where multiple datasets with the same labels are used. For more details and explanation of dataset placements, see Appendix. of our two challenge benchmarks that utilise the availX Figure 1: Overview of proposed and common benchmarks for continual learning in computer vision, with a focus on classification problems. We depict how data distributions X and the available labels Y change during training. The yellow circles areas are proportional to how many papers published in ICCV '21, CVPR '21 and NeurIPS '21 used each benchmark, see Section 2.2 for details. Black circles are recently proposed, more realistic benchmarks that are currently less popular. '*' indicates a 'split' version of the dataset. 'Others' refers to a set of benchmarks that were only used in a single paper and 'sequences' refers to benchmarks where multiple datasets with the same labels are used. For more details and explanation of dataset placements, see Appendix. to [22], the best algorithm is one that has the highest of our two challenge benchmarks that utilise the availMNIST Rotated MNIST CLAD-C CORe50.NI Clear Sequences Clear Sequences Figure 1: Overview of proposed and common benchmarks for continual learning in computer vision, with a focus on classification problems. We depict how data distributions X and the available labels Y change during training. The yellow circles areas are proportional to how many papers published in ICCV '21, CVPR '21 and NeurIPS '21 used each benchmark, see Section 2.2 for details. Black circles are recently proposed, more realistic benchmarks that are currently less popular. '*' indicates a 'split' version of the dataset. 'Others' refers to a set of benchmarks that were only used in a single paper and 'sequences' refers to benchmarks where multiple datasets with the same labels are used. For more details and explanation of dataset placements, see Appendix. of our two challenge benchmarks that utilise the availMNIST* Permuted MNIST Rotated MNIST CORe50 Toys200 iCub CLAD-C CORe50.NI Clear Cub200* Others Sequences CLOC Permuted MNIST Rotated MNIST y1 y2 y3 y4 y5 y6 Cub200* Others Sequences Figure 4: Continual Learning Datasets [Verwimp et al., 2023]. al., 2023]. But in OCL, task reoccurrence is not guaranteed. Here, the learning system is forced to remember old tasks. Hence, after training on a given task, instances from all the previous tasks are considered in the evaluation [Gunasekara et al., 2023]. Thus, forgetting of previous tasks after learning a new task can be calculated. Novel evaluation methods that specifically address a given algorithm's suitability when learning from a data stream with recurrent concept drifts could be an area that the research community could further explore. When evaluating the performance of a new algorithm, it is best to avoid a baseline algorithm, as evaluation is a bit subjective to the performance of the selected baseline. Possible inspirations could come from Online Continual Learning evaluations [Gunasekara et al., 2023]. Online Domain Incremental Continual Learning specifically is quite similar to Stream Learning [Gunasekara et al., 2023]. Different tasks with distribution shifts in the input data appear in the stream. The evaluation method is aware of the drift points of the stream. After learning a new task, the OCL method is evaluated against separate test sets of all the already learned tasks. Apart from accuracy, one interesting metric discussed in OCL is forgetting [Gunasekara et al., 2023]. Here, the current task's performance of the OCL method is compared against previous tasks' best performance, with lower forgetting indicating a model improvement over past tasks.

X

X

Driving

Driving

Driving

Driving

highest

metric

g train-

ad, we

igh ac-

w data,

In this section, we describe the design and development of our continual learning benchmark, using SODA10M [25], an industry-scale dataset for autonomous driving. Self-driving vehicles can change urban mobility significantly, but there are still challenges to overcome. One such challenge is demonstrating the versatility of AI-based automated driving systems to cope with challenging and dynamic real-world scenarios. Several corporations are now driving many thousands of miles a day autonomously, creating streams of sensor measurements that form a natural source of continual learning data. This problem setting leads to natural and gradually changing distribution shifts, an excellent benchmark for CL-algorithms. Next, we introduce the SODA10M dataset, and provide a detailed overview semi-supervised learning are interesting research directions [26], we leave incorporating the unlabelled images for future work, and make use of the labelled dataset portion exclusively for our benchmark challenges. 3.2. Challenge Subtracks As our goal involves working towards more realistic, real-world settings for continual learning, we approach the design of the challenge benchmarks with this mindset. As referred to in Section 2.1, there currently exist two main axes along which continual learning realworld task realism increases: firstly, the problem formulation itself and secondly, more realistic context shifts. Ideally, we combine these aspects into a single comprehensive benchmark. Yet, given that continual object mation. For further discussions, see [23, 24, 12]. 3. Continual Learning Benchmark for Autonomous In this section, we describe the design and development of our continual learning benchmark, using SODA10M [25], an industry-scale dataset for autonomous driving. Self-driving vehicles can change urban mobility significantly, but there are still challenges neighbourhoods. The image label set has bounding box annotations for 6 object classes and covers different 'domains' (cities, weather conditions, time of day and road type) - see Figure 2. See Figure 4 for some examples images (arranged in tasks for CLADD, as will be discussed in Sec. 3.4). While self- and semi-supervised learning are interesting research directions [26], we leave incorporating the unlabelled images for future work, and make use of the labelled dataset portion exclusively for our benchmark challenges. As our goal involves working towards more realistic, In this section, we describe the design and development of our continual learning benchmark, using SODA10M [25], an industry-scale dataset for autonomous driving. Self-driving vehicles can change urban mobility significantly, but there are still challenges to overcome. One such challenge is demonstrating the versatility of AI-based automated driving systems to cope with challenging and dynamic real-world scenarios. Several corporations are now driving many thousands of miles a day autonomously, creating streams of sensor measurements that form a natural source of continual learning data. This problem setting leads to natural and gradually changing distribution shifts, an excellent benchmark for CL-algorithms. Next, we introduce the SODA10M dataset, and provide a detailed overview semi-supervised learning are interesting research directions [26], we leave incorporating the unlabelled images for future work, and make use of the labelled dataset portion exclusively for our benchmark challenges. 3.2. Challenge Subtracks As our goal involves working towards more realistic, real-world settings for continual learning, we approach the design of the challenge benchmarks with this mindset. As referred to in Section 2.1, there currently exist two main axes along which continual learning realworld task realism increases: firstly, the problem formulation itself and secondly, more realistic context shifts. Ideally, we combine these aspects into a single comof our two challenge benchmarks that utilise the available data. 3.1. SODA10MDataset mation. For further discussions, see [23, 24, 12]. 3. Continual Learning Benchmark for Autonomous In this section, we describe the design and development of our continual learning benchmark, using SODA10M [25], an industry-scale dataset for autonomous driving. Self-driving vehicles can change urban mobility significantly, but there are still challenges neighbourhoods. The image label set has bounding box annotations for 6 object classes and covers different 'domains' (cities, weather conditions, time of day and road type) - see Figure 2. See Figure 4 for some examples images (arranged in tasks for CLADD, as will be discussed in Sec. 3.4). While self- and semi-supervised learning are interesting research directions [26], we leave incorporating the unlabelled images for future work, and make use of the labelled dataset portion exclusively for our benchmark challenges. As our goal involves working towards more realistic, lacements, see Appendix. lgorithm is one that has the highest ever we note that the BWT metric efit from low accuracy during trainimproves the metric). Instead, we of our two challenge benchmarks that utilise the available data. 3.1. SODA10MDataset Both tracks build on the SODA10M dataset [25], which contains 10M unlabelled images and 20k labelled 21 used each benchmark, see Section 2.2 for details. Black circles are recently proposed, more r. '*' indicates a 'split' version of the dataset. 'Others' refers to a set of benchmarks that were s to benchmarks where multiple datasets with the same labels are used. For more details and has the highest of our two challenge benchmarks that utilise the availOn the other hand, some of the real-world OCL datasets such as: CORe50 and Online Domain Incremental Continual Learning version of CORe50 [Gunasekara et al., 2022] and many of the datasets discussed in [Verwimp et al., 2023] can be used to evaluate SL models for data streams with recurrent concept drifts. Figure 4 contains a summary of different realworld OCL datasets. As per figure 4, some datasets contain clear distribution shifts going from one task to the other, and in most of those datasets, the start and end of the task are clearly defined to support the OCL evaluation. Such realworld datasets can be very useful in evaluating SL methods proposed for data streams with recurrent concept drifts. 7.3 Evaluation for Recurrent Concept Drifts 8 Conclusions This survey focuses on Stream Learning methods for data streams with recurrent concept drifts. It highlights the importance of SL on data streams with recurrent concept drifts considering concept recurrence in the data streams generated by the increasing digitization processes. The survey explains the currently available methods, their design components, evaluation methods, benchmarks, and the availability of software implementations. It highlights the need for evaluation methods that do not use a baseline algorithm. Furthermore, it inquires into Online Continual Learning and Concept Evolution with recurrent classes to find possible intersections and directions for future research.

would

he area

practireal-world settings for continual learning, we approach the design of the challenge benchmarks with this mindset. As referred to in Section 2.1, there currently exist two main axes along which continual learning realworld task realism increases: firstly, the problem formureal-world settings for continual learning, we approach the design of the challenge benchmarks with this mindset. As referred to in Section 2.1, there currently exist two main axes along which continual learning realworld task realism increases: firstly, the problem formuimages. Image data consists of dash-camera recorded footage, obtained from vehicles driving through four Chinese cities, with images recorded at 10 second inBoth tracks build on the SODA10M dataset [25], Though we see the importance of explainability in recurrent concept drifts, where one explains the reasons for recurrences, explainable Stream Learning itself is still in its infancy. We hope future work in explainable Stream Learning would lay the foundations for explaining recurrent concept drifts.

4 to overcome. One such challenge is demonstrating the versatility of AI-based automated driving systems to cope with challenging and dynamic real-world scenarios. Several corporations are now driving many thousands of miles a day autonomously, creating streams of sensor measurements that form a natural source of continual learning data. This problem setting leads to natu3.2. Challenge Subtracks prehensive benchmark. Yet, given that continual object 4 Both tracks build on the SODA10M dataset [25], which contains 10M unlabelled images and 20k labelled images. Image data consists of dash-camera recorded footage, obtained from vehicles driving through four Chinese cities, with images recorded at 10 second into overcome. One such challenge is demonstrating the versatility of AI-based automated driving systems to cope with challenging and dynamic real-world scenarios. Several corporations are now driving many thousands of miles a day autonomously, creating streams of sensor measurements that form a natural source of continual learning data. This problem setting leads to natu3.2. Challenge Subtracks -algorithm is one that has high acbe improved by including new data, formance drops. Ideally, this would uously during training, and the area curve used as a metric. Yet practie BWT metric y during train). Instead, we at has high acable data. 3.1. SODA10MDataset The main limitation of the performance evaluation methods discussed in section 4.1 for SL on recurrent concept drifts is that they require a baseline method. Thus, the performance of the proposed algorithm is dependent upon the performance of the baseline method. Here, one could choose a general SL algorithm as the baseline to evaluate an algorithm specifically proposed for recurrent concept drifts. This does not allow us to evaluate how accurately the proposed algorithm performs on recurrent concept drifts. On the other hand, model selec-

ral and gradually changing distribution shifts, an excel-

lent benchmark for CL-algorithms. Next, we introduce

ral and gradually changing distribution shifts, an excel-

lent benchmark for CL-algorithms. Next, we introduce

the SODA10M dataset, and provide a detailed overview

ding new data,

sting at

pproxi-

.

h computational cost, and testing at

tervals. Ordering images chronologically largely en-

lation itself and secondly, more realistic context shifts.

lation itself and secondly, more realistic context shifts.

Ideally, we combine these aspects into a single com-

prehensive benchmark. Yet, given that continual object

the SODA10M dataset, and provide a detailed overview Ideally, we combine these aspects into a single comprehensive benchmark. Yet, given that continual object

which contains 10M unlabelled images and 20k labelled

te intervals is a reasonable approxi-

tails visual footage of a car exploring the city and its tails visual footage of a car exploring the city and its images. Image data consists of dash-camera recorded 8035

ally, this would

discussions, see [23, 24, 12].

g, and the area

4

4

tervals. Ordering images chronologically largely en-

neighbourhoods. The image label set has bounding

neighbourhoods. The image label set has bounding

footage, obtained from vehicles driving through four

Wanderlust