Survey on Online Streaming Continual Learning

1 Introduction

Stream Learning (SL) focuses on efficiently learning from streaming data by learning from one instance at a time. The requirements for Stream Learning are: to predict at any given moment, dynamically adapt to underlying data distribution changes (concept drifts), and be computationally efficient when learning and predicting [Bifet et al., 2018]. Data streams are often assumed to be IID, but in most cases, data streams are often non-IID as the data distribution changes over time.

Continual Learning (CL), attempts to learn from a non-IID data stream to preserve and extend already accrued knowledge [Mai et al., 2022]. The learning algorithm is expected to strike a balance between stability and plasticity as the stream undergoes distribution shifts. Furthermore, in Online Continual Learning (OCL) this learning happens online; thus, the learning algorithm is only allowed a single pass over the data [Mai et al., 2022].

We identify Online Streaming Continual Learning (OSCL) as the intersection between Stream Learning and Online Continual Learning. OSCL allows well-researched SL fields OCL

End of the distribution signals provided to the model only at task-incremental testing.

Figure 1: Comparison between SL and OCL settings.

SL: uses drift detectors discussed in section 2.1 to detect distribution shifts, and evaluation is discussed in section 2.3. OCL: uses evaluation metrics (equations 1, 2, 3, and 4) discussed in section 2.3. The models in this setting do not detect distribution shifts. OSCL proposes some of the techniques used in SL to be used in OCL.

such as efficient stream learners, concept drift detection, and adaptation to enhance or develop OCL methods. This survey attempts to understand the intersection of those two closely related fields, considering the underlying setting, evaluation methods, and applications. It also suggests future directions for OSCL taking into account recent advances in SL.

The rest of the paper is organized as follows. Section one explains Stream Learning considering the setting, drift adaptation, methods, evaluation, and applications. The next section explores Continual Learning considering the same criteria except for drift adaptation. Figure 1 compares Stream Learning and Online Continual Learning. Section 4 explains some of the intersection points between these two fields. These intersection points lay the ground for OSCL setting. This is further explained in this section considering new trends and future directions. Finally, we provide our conclusions in the last section.

2 StreamLearning

In Stream Learning, a model learns from an evolving data stream (non-IID data), processing one instance at a time. The learner must predict at any given moment using limited processing and memory [Bifet et al., 2018; Gomes et al., 2017b]. Also, it should adjust to distribution changes in the underlying input distribution [Bifet et al., 2018; Bifet and Gavalda, 2007]. A shift in the data distribution is identified as a concept drift in literature.

2.1 Concept Drift

Concept drifts can be categorized according to their impact on the decision boundary, the evolution of the relationship between features and the target, speed of change, reach, and recurrence [Su´arez-Cetrulo et al., 2023].

• Effect on the decision boundary (impact): the literature describes real and virtual concept drifts. The former effects the the decision boundary of the model. This affects the model. The latter does not affect the decision boundary. Hence the model is unaffected [Ram´ırezGallego et al., 2017].
• Evolution of the relationship between features and the target and the speed of change: in the literature, drifts

are categorized into sudden (abrupt), gradual, and incremental drifts, considering the evolution of the relationship between features and the target and the speed of change. With sudden or abrupt drifts, the current data distribution changes to a new one within a short period [Ram´ırez-Gallego et al., 2017]. In the case of gradual drifts, this transition happens gradually [Su´arezCetrulo et al., 2023]. Here for a certain period, one could observe instances from both distributions. The transition time is very long with incremental drifts, and there may not be a statistical difference between adjacent instances [Ram´ırez-Gallego et al., 2017]. Figure 2 shows how the drift types mentioned above evolve.

• Reach of change: drifts that affect all of the features are considered global drifts [Su´arez-Cetrulo et al., 2023], and drifts that affect some of the features are called local drifts [Khamassi et al., 2015].
• Recurrent concept drifts: if a particular data distribution reoccurs in the stream after a given period, it is considered a recurrent concept drift [Su´arez-Cetrulo et al., 2023].
• Random blips/outliers/noise: are situations where, for a very short period of time, few instances which do not belong to the current distribution popup in the stream [Su´arez-Cetrulo et al., 2023].

Drift Detectors

Many types of drift detectors are explained in the literature. [Souza et al., 2020] explain three types of drift detectors for Stream Learning.

• Methods based on differences between two distributions: These methods compare the difference between two data windows. A reference window with old data and a detection window with recent data are compared using a statistical test to discard the null hypothesis that both data belong to the same distribution. Drift detectors based on fixed-size windows usually suffer from a delay in detection [Souza et al., 2020]. Works such as ADaptive sliding WINdow (ADWIN) [Bifet and Gavalda, 2007] use dynamic windows.
• Methods based on sequential analysis: These are methods founded on the Sequential Probability Ratio Test (SPRT)[Wald, 1947]. CUSUM and Page-Hinkley [Page, 1954] are good examples of drift detectors of this type.
• Methods based on statistical process control: These methods consider the classification problem a statistical process and monitor the evolution of some performance indicators like error rate to apply heuristics to find change points. For example, DDM [Gama et al., 2004] has three different states for the classification error evolution: in-control when the error is in the control level, out-of-control when the error is increasing significantly compared to the recent past, and warning, when the error is increasing but has not reached the out-of-control level. Where DDM only looks at the magnitude of the errors, EDDM [Baena-Garcıa et al., 2006] also considers the distance in time between consecutive errors.

We would like to direct the reader to work by [Khamassi et al., 2018] and [Gama et al., 2014] for a thorough review of drift detectors for Stream Learning.

2.2 Methods

Similar to batch learning, Stream Learning methods can be categorized into supervised, semi-supervised, and clustering. In supervised SL, it is assumed that target values are available for each instance. In semi-supervised Stream Learning, this assumption is relaxed for some instances. Target values may only become available at a later time or not be available at all. Clustering assumes the unavailability of target variables. Akin to batch learning, supervised SL has two main categories: classification and regression. There are quite a few popular classification methods proposed in SL. Starting with simple but effective learners like Naive Bayes (NB) and Hoeffding Tree (HT) to ensemble learners like Adaptive Random Forest (ARF), Streaming Random Patches (SRP), and Continuously Adaptive Neural Networks for Data Streams (CAND)[Gunasekara et al., 2022c]. HT [Hulten et al., 2001] builds a tree using the Hoeffding bound to control its split decisions with a given confidence. Later an adaptive version of it was introduced to replace the branches when the data stream is evolving [Bifet and Gavalda, 2009]. Ensemble methods have shown great success in Stream Learning [Gomes et al., 2017b]. They allow the use of efficient SL base learners like HT in a bagging or random forest setting with efficient drift detectors like ADWIN [Gomes et al., 2019]. ARF is an online random forest implementation for Stream Learning which uses effective re-sampling methods and drift adaptation mechanisms to cope with different types of concept drifts [Gomes et al., 2017a]. SRP trains base models on random subsets of features and instances identified as patches [Gomes et al., 2019]. It uses the same drift adaptation strategy as in ARF but produces better results than ARF. CAND trains a pool of simple NNs and uses the one with the smallest estimated loss for prediction. It employs ADWIN, an estimator to estimate each NN's loss. As CAND uses NNs as it base learners, it works well on high-dimensional data. We would like to direct the reader to [Gomes et al., 2017b], which contains an extensive taxonomy of data stream ensemble classifiers.

Many data stream regression methods are explained in the literature [Choudhary et al., 2021]. Hoeffding Tree Regressor (HTR) is an adaptation of the incremental tree algorithm HT for regression. Like HT, HTR uses the Hoeffding bound to control its split decisions. HTR relies on calculating the reduction of variance in the target space to find a split candidate. Fast Incremental Model Trees with Drift Detection (FIMT-DD) learn model trees from an evolving data stream with drift detection [Ikonomovska et al., 2011]. It uses the variance reduction split criterion for splitting and the PageHinckley test for drift detection. More recent ensemble methods for streaming regression include Adaptive Random Forest Regressor (ARF-REG) [Gomes et al., 2018], and SelfOptimising K-Nearest Leaves (SOKNL) [Sun et al., 2022]. SOKNL claim to have superior accuracy compared to ARFREG.

Label availability in a streaming setting can be categorized into four groups: (i) Immediate and fully labelled, (ii) Delayed and fully labelled, (iii) Immediate and partially labelled, (iv) Delayed and partially labelled [Gomes et al., 2022]. The majority of data stream Semi-Supervised Learning (SSL) is devoted to understanding (iii). However, [Gomes et al., 2022] highlights the importance of understanding the delayed and partially labelled (iv) setting. Furthermore, the authors categorize streaming SSL methods into: (i) intrinsically SSL, (ii) self-training, and (iii) learning by disagreement. Intrinsically SSL methods exploit the unlabelled instances directly as part of their objective function or optimization procedure [Gomes et al., 2022]. Self-training methods are based on the idea that a classifier learns from its previous mistakes and then reinforces itself [Gomes et al., 2022]. It can act as a wrapper algorithm that uses any arbitrary classifier. Learning by disagreement works by learners teaching other learners. Models are trained with multiple viewpoints of the same data 1 , which results in disagreeing models. The key idea behind learning by disagreement is to generate multiple learners and let them collaborate to exploit the unlabelled data [Gomes et al., 2022].

Data stream clustering can be categorized into: partition clustering, micro-cluster-based clustering, density-based clustering, and hierarchical clustering [Bahri et al., 2021]. Instances from a stream are divided into segments without a class label. The objective of this type of SL is to discover patterns in the stream in an online fashion with a minimum amount of resources. Also, algorithms deployed in this setting should be able to cope with the evolving nature of the stream. The survey by [Zubaro˘glu and Atalay, 2021] contains a recent and extensive study on this field.

2.3 Evaluation

Several methods are explained in the SL literature for evaluating a model. The most popular one is the test-then-train approach [Bifet et al., 2018; Gama et al., 2013]. As the name suggests, the evaluation uses the incoming instance to test the model first and later train the model. Here the current predictive evaluation is affected by the previous evaluations. This may be desirable when one is interested in the model's overall performance. Test-then-train is also known as prequential evaluation in the literature. The prequential evaluation may not be reliable in conveying the current predictive performance of the model. Therefore prequential evaluation can be equipped with a sliding window, or a fading factor, to gracefully forget the performance on instances from the distant past [Bifet et al., 2018; Gama et al., 2013]. For partly labelled data, prequential evaluation is still applicable as the loss can be calculated on just the labelled subset of instances [Gomes et al., 2022]. Data stream cross-validation was introduced by [Bifet et al., 2015] where models are trained and tested in parallel on different folds of the data. Continuous reevaluation considers the verification latency in the streaming setting with partially delayed labels [Grzenda et al., 2020a; Grzenda et al., 2020b]. This evaluation attempts to evaluate how fast a model can transform from an initial possibly incorrect prediction to a correct prediction prior to the availability of the true label.

1 This could be achieved through techniques such as bootstrapping aggregation.

There are several metrics explained in the literature to measure the performance of an SL classification algorithm. The most popular one is accuracy. If the data stream is imbalanced, accuracy can be misleading; sensitivity and specificity are better measurement alternatives [Bahri et al., 2021]. The kappa statistic compares the model's prequential accuracy against the chance classifier (one that randomly assigns to each class the same number of instances as the model under consideration) [Bifet et al., 2018]. On the other hand, the kappa M compares the current model's performance against the majority class classifier [Bifet et al., 2018]. Kappa temporal attempts to capture the temporal dependencies in a data stream by comparing the model performance against a "nochange" model, which predicts the next instance using the current instance's label [Bifet et al., 2018] 2 . For delayed label situations, when multiple predictions are made for a single instance, accuracy and kappa values can be aggregated to produce immediate measures until the true label is available [Grzenda et al., 2020b; Gomes et al., 2022].

Regression SL uses two main evaluation metrics: (i) Root mean squared error (RMSE) and (ii) Mean absolute error (MAE) [Bahri et al., 2021]. We direct the reader to [Bifet et al., 2018; Bahri et al., 2021] for thorough reviews of regression evaluation methods and [Kremer et al., 2011] for clustering evaluation methods. Furthermore, data stream evaluation also considers computing and memory usage [Bifet et al., 2018].

2.4 Application

SL has been used in many situations where learning happens from an evolving data stream. [Souza et al., 2020] used SL on data generated by optical sensors, which measure the flying behavior of insects to identify disease vector insects. Also [Gao and Lei, 2017] used SL methods for online crude oil price prediction. SL was used to predict power production considering environmental conditions by [Lobo et al., 2020]. The study by [ ˇZliobait˙e et al., 2016] contains some interesting applications of SL for monitoring and control problems. It includes application tasks such as traffic management, activity recognition, communication monitoring, controlling robots, intelligent appliances, intrusion detection, fraud detection, and insider trading. The study also contains some interesting areas where SL could provide solutions. We like to direct the reader to [ ˇZliobait˙e et al., 2016] for a broader understanding of SL applications.

3 Continual Learning

The literature has thoroughly documented that an NN receiving non-IID data forgets past knowledge when confronted with a concept shift [Kirkpatrick et al., 2017; Mai et al., 2022]. CL attempts to learn with minimal forgetting of past concepts [Kirkpatrick et al., 2017; Mai et al., 2022]. In OCL, this learning happens online. Three main continual learning settings are described in the literature: task-incremental, class-incremental, and domain-incremental.

2 These measurements are thoroughly explained in [Bifet et al., 2018]

• Task-incremental: In this setting, output distributions are demarked by external task ids, available for training and testing. In this setting, the model can use the external task-id signal at test time [Mai et al., 2022].
• Class-incremental: Each distribution consists of classes that are unavailable in other distributions (tasks). This setting adapts a single-head NN configuration. Here, output distributions differ from task to task [Mai et al., 2022].
• Domain-incremental, on the other hand, assumes output distribution from one task to the other to be the same while having different input distributions [Mai et al., 2022].

In both class-incremental and domain-incremental settings, an external task-id that separates one task from another is assumed to be unavailable at test time. The availability of this signal at training is optional. However, some CL methods rely on this signal during training. Online Class Incremental Continual Learning (OCICL) and Online Domain Incremental Continual Learning (ODICL) assume class-incremental and domain-incremental OCL settings, respectively.

3.1 Methods

CL algorithms use three popular approaches to avoid catastrophic forgetting in NNs: regularization, replay, and parameter isolation.

• Regularization methods: algorithms like Elastic Weight Consolidation (EWC) [Kirkpatrick et al., 2017] and Learning without Forgetting (LWF) [Li and Hoiem, 2017] adjust the weights of the network in such a way that it minimizes the overwriting of the weights for the old concept. EWC uses a quadratic penalty to regularize updating the network parameters related to the past concept. It uses the Fisher Information Matrix's diagonal to approximate the importance of the parameters [Kirkpatrick et al., 2017]. EWC has some shortcomings: 1) the Fisher Information Matrix needs to be stored for each task, 2) it requires an extra pass over each task's data at the end of the training [Mai et al., 2022]. Though different versions of EWC address these concerns [Mai et al., 2022], [Chaudhry et al., 2018] seems suitable for online CL by keeping a single Fisher Information Matrix calculated by a moving average. LWF uses knowledge distillation to preserve knowledge from past tasks. Here, the model related to the old task is kept separate, and a separate model is trained on the current task. When the LWF receives data for a new task (Xnew, Ynew), it computes the output (Yold) from the old model for the new data Xnew. During training, assuming that ˆ Yold and ˆ Ynew are predicted values for Xnew from the old model and new model, LWF attempts to minimize the loss: αLKD(Yold, ˆ Yold) +LCE(Ynew, ˆ Ynew) + R [Mai et al., 2022]. Here LKD is the distillation loss for the old model, and α is the hyper-parameter controlling the

Examples of the three CL scenarios. (x, y, task-ID) represents (input images, target label and task identity). The main distin

to i, and we assume there are T tasks in tot

Online

Task-ID

Train & Test

No

No

Train & Test No No Optional No Optional ð3Þ Y i1 f g - Yi f g U data sequence (see example in Table 3). Average Accuracy Ai ð Þ ¼ 1 i Xi j¼1 ai;j strength of the old model against the new one. LCE is the cross-entropy loss for the new task. R is the general regularization term. Due to this strong relation between old and new tasks, it may perform poorly in situations where there is a huge difference between old and new task distributions [Mai et al., 2022].

ð Þ is t

Task-ID

No

No

m a disjoint space which is separated by task-ID.

enotes that output space are from a disjoint space which is separated by task-ID.

io as

ting,

class

available during both train and test times. For class

Þ - P Xt

U

ð Þ

mental

data to i, and we assume there are T tasks in total.

new task consists of new instances with nonstationarity (e.g., noise) of all the seen classes.

P Yi1

f g - Yi

Þ - P Yi

U

U

P Yi1

P Xi1

ð

ð

Þ - P Yi

ð Þ

Þ - P Xt

ð Þ

Y i1

ð

f g

ð Þ

, mutually exclusive sets of classes comprise each data

U

mental

ong

U

emental

Di, meaning that there is no duplicated class among

tput

opts

k distributions. Thus P Yi1

same for all distributions since this setting adopts

Table 2

d Di , following [27]. P X

arated by task-ID.

io

een Di1 and Di

U

ð Þ, but the output

Task-ID

w instances with nonstationarity (e.g., noise) of all the seen classes.

Þ - P Yi

U

ð Þ

mental

U

mental

emental Train & Test

U

U

Table 2

ty (e.g., noise) of all the seen classes.

30

over-fitting on replay buffer data 3 . The approach seems to improve all replay methods in general.

• Parameter-isolation: The intuition behind parameterisolation methods is to avoid interference by allocating separate parameters for each task [Mai et al., 2022]. There are two types of parameter-isolation-based methods: fixed architecture and dynamic architecture. Fixed architecture only activates the relevant part of the network without changing the NN architecture [Mai et al., 2022]. Dynamic architecture, on the other hand, adds new parameters for the new task while keeping the old parameters [Yoon et al., 2017; Mai et al., 2022]. Continual Neural Dirichlet Process Mixture (CN-DPM) [Lin, 2013] trains a new model for each new task and leaves the existing models untouched so that at a later point, it can retain the knowledge of the past tasks. It composes of a group of experts where each expert contains a discriminative model and a generative model. Each expert is responsible for a subset of the data. The group is expanded based on the Dirichlet Process Mixture using Sequential Variational Approximation [Mai et al., 2022].

We would like to direct the reader to a survey by [Mai et al., 2022] for in-depth detail about those methods.

3.2 Evaluation

There are many evaluation metrics defined in the CL literature. On a stream with T tasks, after training the NN from tasks 1 to i, let ai,j be the accuracy on the held-out test set for task j. Average accuracy (Ai) at task i is defined as:

[Chaudhry et al., 2018]. Average forgetting (Fi) at task i is defined as:

, where

Here fk,j is the best test accuracy the model has ever achieved on task j prior to learning task k. ak,j is the test accuracy on task j after learning task k [Chaudhry et al., 2018]. The positive influence of learning a new task on previous tasks' performance is measured by Positive Backward Transfer (BWT):

[Mai et al., 2022]. The positive influence of learning a given task on future tasks' performance is defined as Forward Transfer (FWT):

3 A well-documented issue in replay methods [Zhang et al., 2022].

[Mai et al., 2022]. Further to the above metrics, run-time and memory usage are also considered when evaluating OCL methods [Mai et al., 2022].

3.3 Applications

Recent research has focused on using ODICL methods to avoid costly retraining in practical situations where the model is confronted with a concept shift. ODICL has been used in X-ray image classification to avoid retraining on distribution shifts due to unforeseen shifts in hardware's physical properties [Srivastava et al., 2021]. Also, it has been used to mitigate bias in facial expression and action unit recognition across different demographic groups [Kara et al., 2021]. Furthermore, ODICL was used to counter retraining on concept shifts for multi-variate sequential data of critical care patient recordings [Armstrong and Clifton, 2021]. The authors highlight some replay method's infeasibility due to strong privacy requirements in clinical settings. This concern is further highlighted in the empirical study by [Mai et al., 2022]. Practical implementations such as [Kara et al., 2021] and [Armstrong and Clifton, 2021] use the end of the task signal to employ OCL methods such as EWC and LWF. However, on the other hand, practical implementation in [Srivastava et al., 2021] assumes a gradual distribution shift in the input data distribution where instances from both the new and old tasks could appear in the stream for a certain period.

4 Online Streaming Continual Learning

In Stream Learning, the objective is to adjust to the current concept in the stream efficiently. On the other hand, OCL has dual learning objectives: adjust to the current concept while preserving knowledge about previous concepts. Both settings assume data is non-IID. In Stream Learning, it is assumed that model should detect distribution changes and adapt accordingly. However, in OCL, the end of the concept signal is provided at training time, even though some replay methods may not use it. This end-of-the-concept signal is only provided for task-incremental CL at test time. Figure 1 also shows the differences between these two settings. Contrary to the differences, these two fields have many intersection points. We identify these intersection points as Online Streaming Continual Learning (OSCL). In OSCL, we mainly identify how well-researched Stream Learning techniques and methods could be used to enhance OCL.

SL on recurrent concept drifts attempts to adjust to an evolving data stream where some concepts could reemerge at a later stage of the stream [Su´arez-Cetrulo et al., 2023]. The setting is similar to OCL but without the additional learning objective of preserving old knowledge explicitly. Hence evaluation in this setting does not consider how to measure forgetting of past knowledge. Most of the methods explored in this setting keep a fixed-size pool of classifiers [Su´arez-Cetrulo et al., 2023]. Various mechanisms are explored in the literature on how to maintain this pool [Su´arez-Cetrulo et al., 2023; Anderson et al., 2019] and how to use it for prediction [Su´arez-Cetrulo et al., 2023; Almeida et al., 2018]. This "pool of classifiers" is also known as "concept history", "concept list", and "concept repository" in the literature [Su´arez-Cetrulo et al., 2023]. Measures like concept equivalence and concept similarity were introduced to identify the current concept in the data stream from the concept pool.

• Conceptual equivalence assumes that when two classifiers behave similarly on a given time window, both describe the same concept [Yang et al., 2006].
• Concept similarity: recognizes similar concepts using Euclidean distances between concept clusters [Li et al., 2012]. Thus it can detect recurring drifts in unlabelled data.

Measures such as concept equivalence and concept similarity could be used for model selection and data retrieval from the replay buffer in OCL. When handling recurrent concepts, predicting the following concept is helpful so the learner can adjust to the incoming concept ahead of time. [Chen et al., 2016] proposed a method that used a probabilistic network to predict future changes. [Maslov et al., 2016] proposed a method to use patterns acquired during previous drifts to predict the time of the next drift. The method assumed a Gaussian distribution for the duration of the concepts. A recent survey by [Su´arez-Cetrulo et al., 2023] discusses the above and many more exciting topics on SL for recurrent concept drifts.

Most of the OCL methods rely on externally provided endof-concept signals (task ids) at training 4 . It is critical for an autonomous learning agent to detect these concept shifts and adjust accordingly. While OCL research has explored different methods to preserve old knowledge when adjusting to new concepts, SL has done an excellent job of understanding how to detect distribution shifts, especially through different drift detection methods on streams with different drift types (abrupt, gradual, incremental, and recurrent) and different label conditions (available for all instances/ delayed label/ no label). OCL could utilize well-researched drift detectors to detect the end-of-concept signal. Recent research in that direction includes Online Domain Incremental Pool (ODIP) [Gunasekara et al., 2022b] and Online Domain Incremental Networks (ODIN) [Gunasekara et al., 2022a], which use ADWIN as an end-of-task signal generator in an ODICL setting. These methods use this internal signal to adjust to newly perceived concepts automatically. Recent work by [Davalas et al., 2022] has explored the use of drift detectors to identify when to use the data from the instance buffer and how to use it in an OCL setting. Having well-researched SL knowledge on different distribution shifts and different drift detectors would allow OCL algorithms to be more effective in practical OCL scenarios, like the gradual distribution shifts in x-ray images [Srivastava et al., 2021].

4 Due to internal instance buffers, some OCL models may not need task ids at train time. The performance of these models is heavily dependent upon the size of this instance buffer [Mai et al., 2022]

Furthermore, semi-supervised SL could enhance OCL to be more practical in situations where label data is not always immediately available. Works like ORDisCo [Wang et al., 2021] and CURL [Rao et al., 2019] have already started exploring this research area. Semi-supervised SL methods under Self-training and learning by disagreement categories [Gomes et al., 2022] could easily be deployed on real world OCL settings where labels are not always present. However, many opportunities exist considering the breadth of semisupervised SL methods discussed by [Gomes et al., 2022].

Streaming clustering is another exciting area that could be explored in future OCL work. Here, well-established streaming clustering algorithms [Zubaro˘glu and Atalay, 2021] could solve interesting OCL problems as streaming clustering algorithms extract patterns from evolving data. One could use a streaming clustering algorithm to extract tasks from an unsupervised OCL setting. The extracted information, such as task information, could be used for model selection and data retrieval from the replay buffer. The most exciting aspect of the streaming clustering algorithms for OCL is that they are well-studied for evolving data streams.

On the other hand, SL could adapt the dual learning objective in OCL (adjust to the current concept while preserving knowledge about previous concepts). This would allow one to evaluate the breadth of well-studied SL methods for OCL. There is some emerging work in this area where catastrophic forgetting is explored in HTs [Korycki and Krawczyk, 2021]. Implementing the OCL evaluation discussed in [Mai et al., 2022] on popular SL platforms such as MOA [Bifet et al., 2010] and river [Montiel et al., 2021] would speed up this area of research.

Table 1 summarizes the above-discussed synergies and differences between Stream Learning and OCL. It points out the differences in the settings and lists some future research directions in Online Streaming Continual Learning.

5 Conclusion

SL attempts to adjust to an evolving data stream with multiple concepts efficiently. The primary learning objective of the model is to performwell on the current concept. For the same setting, OCL, on the other hand, attempts to do well on the current concept while preserving the knowledge of past concepts. Drift detection techniques could be used to detect new concepts in OCL. So that OCL models could be trained without an external end-of-concept signal (task id). Also, mechanisms explored in SL for recurrent concept drifts could give new insights into predicting future drifts real world in OCL settings. Some of the techniques and methods explored in semi-supervised SL could allow OCL to be applicable in situations where labels are not always available. Furthermore, popular SL frameworks can employ the dual learning objective discussed in OCL. This would allow the OCL community to evaluate numerous SL methods and techniques for OCL.