Enhancing Efficient Continual Learning with Dynamic Structure Development of Spiking Neural Networks

1 Introduction

Children are able to incrementally learn new tasks to acquire new knowledge, however, this is a major challenge for Deep Neural Networks (DNNs) and Spiking Neural Networks (SNNs). When learning a series of different tasks sequentially, DNNs and SNNs forget the previously acquired knowledge and fall into catastrophic forgetting [French, 1999]. Despite some preliminary solutions that have recently been proposed for DNNs-based continual learning, there is still a lack of in-depth inspiration from brain continual learning mechanisms and exploration on SNNs-based models.

* Corresponding author.

The studies attempt to address the continual learning problem of DNNs under task incremental learning (recognition within the classes of a known task) and class incremental learning (recognition within all learned classes) scenarios. Related works can be roughly divided into three categories:

1. Regularization. Employing maximum a posterior estimation minimizes the changes of important weights [Li and Hoiem, 2017; Kirkpatrick et al., 2017; Zenke et al., 2017]. These methods require strong model assumptions, such as the EWC [Kirkpatrick et al., 2017] supposing that new weights are updated to local regions of the previous task weights, which are highly mathematical abstractions and poorly biologically plausibility.
2. Replay and retrospection. Reviewing a portion of the samples of the old tasks while learning the new task [LopezPaz and Ranzato, 2017; van de Ven et al., 2020; Kemker and Kanan, 2018], is currently considered as the superior class incremental learning method. The samples of old tasks are stored in additional memory space or generated by additional generation networks, resulting in extra consumption.
3. Dynamic network structure expansion. [Rusu et al., 2016; Siddiqui and Park, 2021] proposed progressive neural networks that extend a new network for each task, causing a linear increase in network scale. To reduce network consumption, a sub-network of the whole is selected for each task using pruning and growth algorithms [Yoon et al., 2018; Dekhovich et al., 2023], evolutionary algorithms [Fernando et al., 2017] or reinforcement learning (RL) algorithms [Xu and Zhu, 2018; Gao et al., 2022]. However, these methods require storing a mask for each sub-network, which to some extent amounts to storing a separate network for each task, rather than a brain-inspired overall network capable of performing multiple sequential tasks simultaneously.

To the best of our knowledge, there is little research on SNNs-based continual learning. Spiking neural networks, as third-generation neural networks [Maass, 1997; Zhao et al., 2022b], simulate the information processing mechanisms of the brain, and thus serve well as an appropriate level of abstraction for integrating inspirations from brain multi-scale biological plasticity to achieve child-like continual learning. The existing HMN algorithm [Zhao et al., 2022c] uses a DNN network to decide the sub-network of SNN for each task, and is only applicable to two-layer fully connected networks for the N-MNIST dataset. There is still a lack of SNNs-based continual learning methods that could incorporate in-depth inspiration from the brain's continual learning mechanisms, while achieving comparable performance with DNNs under complex continual learning scenarios.

Structural development mechanisms allow the brain's nervous system to dynamically expand and contract, as well as flexibly allocate and invoke neural circuits for efficient continual learning [Silva et al., 2009]. Motivated by this, this paper proposes Dynamic Structure Development of Spiking Neural Networks (DSD-SNN) for efficient and adaptive continual learning. DSD-SNN is designed as an SNN architecture that can be dynamically expanded and compressed, empowering a single network to learn multiple incremental tasks simultaneously, overcoming the problem of needing to assign masks to each task faced by DNNs-based continual learning methods. We validate the effectiveness of our proposed model on multiple class incremental learning (CIL) and task incremental learning (TIL) benchmarks, achieving comparable or better performance on MNIST, N-MNIST, and CIFAR100 datasets. Especially, the proposed DSD-SNN model achieves an accuracy of 77.92% ± 0.29% on CIFAR100, only using 37.48% of the network parameters.

The main contributions of this paper can be summarized as follows:

• DSD-SNN dynamically grows new neurons to learn newly arrived tasks, while extremely compressing the network to increase memory capacity and reduce computational overhead.
• DSD-SNN maximally utilizes the previously learned tasks to help quickly adapt and infer new tasks, enabling efficient and adaptive continual learning (no need to identify separate sub-network mask for each task).
• The experimental results demonstrate the remarkable superiority of DSD-SNN model on performance, learning speed, memory capacity and computational overhead compared with the state-of-the-art (SOTA) SNNs-based continual learning algorithms, and comparable performance with DNNs-based continual learning algorithms.

2 RelatedWork

This paper mainly focuses on dynamic network structure expansion algorithms based on structural plasticity, which can be divided into progressive neural networks (PNN) and subnetwork selection algorithms. In fact, the existing network structure expansion algorithms are mostly DNNs-based continual learning, with little exploration on SNNs.

Progressive neural networks. [Rusu et al., 2016] first proposes the progressive neural network and applies it to multiple continual reinforcement learning tasks. The PNN expands a new complete network for each new task and fixes the networks of the old tasks. In addition, lateral connections are introduced between the networks to effectively leverage the knowledge already learned. PIL [Siddiqui and Park, 2021] extends the PNN to large-scale convolutional neural networks for image classification tasks. However, the PNN algorithms extremely increase the network storage and computational consumption during continual learning. In contrast, as development matures and cognition improves, the number of brain synapses decreases by more than 50% [Huttenlocher, 1990], forming a highly sparse brain structure perfect for continual learning. The PNN blindly expand the structure causing catastrophic effects in the case of massive sequential tasks.

Sub-network selection algorithm. A part of the network nodes is selected to be activated for a given task. PathNet [Fernando et al., 2017] is first proposed to select path nodes (each node contains a set of neurons) for each task using the genetic algorithm. RPS-Net [Rajasegaran et al., 2019a] randomly activates multiple input-to-output paths connected by convolutional blocks, and chooses the highestperforming ones as the final path. In addition, RCL [Xu and Zhu, 2018] employ additional RL networks to learn the number of neurons required for a new task, while CLEAS [Gao et al., 2022] uses RL to directly determine the activation and death of each neuron. HMN [Zhao et al., 2022c] uses a hybrid network learning framework that uses an ANN modulation network to determine the activation of neurons for a SNN prediction network, but is only applied to small-scale networks for simple scenarios. A sub-network mask learning process based on pruning strategy is proposed by [Dekhovich et al., 2023], which is applied to CIL combined with the replay strategy. The above algorithms select sub-networks for each task separately, failing to maximize the reuse of acquired knowledge to support new task learning.

To solve this problem, DER [Yan et al., 2021] prunes a sparse convolutional feature extractor for each task, and then merges the output of the convolution extractor into the previous tasks. CLNP [Golkar et al., 2020] grows new neurons for a new task based on the old network, and DEN [Yoon et al., 2018] expands when the already learned network is insufficient for the new task, while reusing the existing neurons. These several works require storing an additional subnetwork mask for each task, which both increases additional storage consumption and is not consistent with the overall developmental learning process of the brain.

Considering the various limitations of existing works above, the DSD-SNN proposed in this paper, which is a pioneering algorithm on SNNs-based continual learning, enables the capacity of a single network to learn multiple sequential tasks simultaneously, while reusing the acquired knowledge and significantly increasing the memory capacity.

3 Method

3.1 Continual Learning Definition

We are expected to sequentially learn Γ tasks, Γ = {T1, ..., TN}. Each task Ti takes the form of a classification problem with its own dataset: DTi = {(xj, yj)}NTi j=1 , where xj ∈ χ, yi ∈ {1, ...,CTi }, χ is the input image space, NTi and CTi are the number of samples and classes of task Ti. For the task incremental learning scenario, Ti is knowable in the testing process, setting requires to optimize:

where θ is the network parameters. When Ti is unknown in testing, more complex class incremental learning scenarios solve the following problems:

3.2 DSD-SNNArchitecture

The design of the DSD-SNN algorithm is inspired by the dynamic allocation, reorganization, growth, and pruning of neurons during efficient continual learning in the brain. As depicted in Fig. 1, the proposed DSD-SNN model includes three modules (random growth, adaptive pruning, freezing neurons) to accomplish multi-task incremental learning.

Random growth. When a new task is coming, the DSDSNN model first randomly assigns and grows a portion of untrained empty neurons to form a new pathway. And the new task-related classification neurons are added to the output layer as shown in Fig. 1. Newly grown neurons receive the output of all non-empty neurons of the previous layer (both newly grown neurons and already frozen neurons in the previous tasks). Therefore, all features learned from previous tasks can be captured and reused by the neural pathways of the new task. Then, the DSD-SNN algorithm can take full advantage of the features learned from the previous task to help the new task converge quickly, while the newly grown neurons can also focus on learning features specific to the new task.

Adaptive pruning. During the learning process of the current task, the DSD-SNN algorithm adaptively detects relatively inactive neurons in the current pathway based on synaptic activity and prunes those redundant neurons to save resources. The pruned neurons are re-initialized as empty neurons that can be assigned to play a more important role in future tasks. Pruning only targets those neurons that are newly grown for the current task and does not include neurons that were frozen in the previous tasks. Adaptive pruning can substantially expand the memory capacity of the network to learn and memorize more tasks under a fixed scale.

Freezing neurons. The contributing neurons that are retained after pruning will be frozen, enabling the DSD-SNN model to learn new tasks without forgetting the old tasks. The frozen neurons can be connected to newly grown neurons to provide acquired knowledge. During the training of new task Ti, all input synapses of the frozen neuron are no longer updated, only the newly added output synapses to the new neurons can be updated. The DSD-SNN model with neuron growth, pruning, and freezing can memorize previous knowledge and reuse the acquired knowledge to learn new tasks for efficient continual learning.

The deep SNN with multiple convolutional and fully connected layers is constructed to implement task incremental learning and class incremental learning, as shown in Fig. 2. During the training process, we sequentially input training samples of each task and update the synapses newly added to the network. In the testing process, test samples of all learned tasks are fed into our overall multi-task continual learning network, so that a single DSD-SNN model can achieve all tasks without the need to identify separate sub-network mask for each task.

To address more complex class incremental learning, we add a two-layer network as the task classifier. The task classifier receives inputs from the classes outputs of the continual learning network, and outputs which task the current sample belongs to (as in the red box in Fig. 2). According to the inferred task ˆ Ti obtained from the task classifier, the DSD-SNN model chooses the maximum output class of the ˆ Ti task in the continual learning network as the predicted class.

3.3 DSD-SNNComputational Details

So far in this section, we have described how our model efficiently and adaptively accomplishes continual learning. We now introduce the detailed growth and pruning scheme that we use throughout this paper.

Neuronal Growth and Allocation

During brain development, neurons and synapses are first randomly and excessively grown and then reshaped based on the external experience [Jun and Jin, 2007; Elman et al., 1996]. In the DSD-SNN model, the SNN is first initialized to consist of N l neurons in each layer l. In the beginning, all neurons in the network are unassigned empty neurons Nempty. When the new task Ti arrives, we randomly grow ρ%×N l neurons from the empty neurons for each layer, denoted as Nnew. After training and pruning for task Ti , all retained neurons Nnew are frozen, added to Nfrozen.

To better utilize the acquired knowledge, the newly grown neurons N lnew in each layer not only receive the output of the new growth neurons N l-1 new in the previous layer, but also receive the output of the frozen neurons N l-1 frozen in the previous layer, as follows.

Where → represents the input connections. For the frozen neurons N l-1 frozen , growth does not add input connections to avoid interference with the memory of previous tasks.

Note that we do not assign task labels to frozen and new growth neurons in either the training or testing phase of continual learning. That is, the DSD-SNN algorithm uses the entire network containing all neurons that have learned previous tasks to do prediction and inference. Thus, our model is able to learn multiple sequential tasks simultaneously without storing separate sub-network masks.

Neuronal Pruning and Deactivation

Neuroscience researches have demonstrated that after the overgrowth in infancy, the brain network undergoes a long pruning process in adolescence, gradually emerging into a delicate and sparse network [Huttenlocher and others, 1979; Huttenlocher, 1990; Zhao et al., 2022a]. Among them, input synapses are important factors to determine the survival of neurons according to the principle of "use it or lose it" [Furber et al., 1987; Bruer, 1999; Zhao and Zeng, 2021]. For SNN, neurons with input synapse weights close to 0 are more difficult to accumulate membrane potentials beyond the spiking threshold, resulting in firing spikes less and contributing to the outputs less. Therefore, we used the sum of input synapse weights Sl i to assess the importance of neurons i in the l layer as in Eq. 4.

Where Wij is the synapse weights from presynaptic neuron j to postsynaptic neuron i, Ml-1 is the number of presynaptic neurons.

During the training of new tasks, we monitor the importance of newly grown neurons Nnew and prune redundant neurons whose values of Si are continuously getting smaller. Here, we define a pruning function as follows:

Where Norm(Sl i ) refers to linearly normalize Sl i to 0 ∼ 1. α = 2 and ρp control the pruning strength. ρp includes ρc and ρf for the convolutional and fully connected layers, respectively. P l i is initialized to 5. γ = 0.99 and η controls the update rate as [Han et al., 2022b]. e- epoch η decreases exponentially with increasing epoch, which is consistent with the speed of the pruning process in biological neural networks that are first fast, then slow, and finally stable [Huttenlocher and others, 1979; Han et al., 2022a].

The pruning functions are updated at each epoch, then we prune neurons with the pruning function P l i < 0. We structurally prune channels in the convolutional layer and prune neurons in the fully connected layer, removing their input connections and output connections.

3.4 SNNs Information Transmission

Different from DNNs, SNNs use spiking neurons with discrete 0/1 output, which are able to integrate spatio-temporal information. Specifically, we employ the leaky integrate-andfire (LIF) neuron model [Abbott, 1999] to transmit and memorize information. In the spatial dimension, LIF neurons integrate the output of neurons in the previous layer through input synapses. In the temporal dimension, LIF neurons accumulate membrane potentials from previous time steps via internal decay constants τ. Incorporating the spatio-temporal information, the LIF neuron membrane potential U t,l i at time step t is updated by the following equation:

When the neuronal membrane potential exceeds the firing threshold Vth, the neuron fires spike, and its output Ot,l i is equal to 1; Conversely, the neuron outputs 0. The discrete spiking outputs of LIF neurons conserve consumption as the biological brain, but hinder gradient-based backpropagation. To address this problem, [Wu et al., 2018] first proposed the method of surrogate gradient. In this paper, we use Qgategrad [Qin et al., 2020] surrogate gradient method with constant λ = 2 to approximate the spiking gradient, as follows:

Overall, We present the specific procedure of our DSD-SNN algorithm as Algorithm 1.

Algorithm 1: The DSD-SNN Continual Learning.

Input: Dataset DTi for each task Ti;

Initialize empty network Net;

Constant parameters of growth ρ% and pruning ρc, ρf .

Output: Prediction Class in task Ti (TIL) or in all

tasks (CIL).

for each sequential task Ti do

Growing new neurons to Net as Eq. 3;

for epoch = 0; epoch < E; epoch + + do

SNN forward prediction Net (DTi ) as Eq. 7;

SNN backpropagation to update new connections as Eq. 8;

Assessing importance for newly grown neurons as Eq. 4;

Calculating the neuronal pruning function as Eq. 5 and Eq. 6;

Pruning redundant neurons with P l i < 0; end

Freezing retained neurons in Net;

end

4 Experiments

4.1 Datasets andModels

To validate the effectiveness of our DSD-SNN algorithm, we conduct extensive experiments and analyses on the spatial MNIST [LeCun et al., 1998], CIFAR100 [Xu et al., 2015] and neuromorphic temporal N-MNIST datasets [Orchard et al., 2015] based on the brain-inspired cognitive intelligence engine BrainCog [Zeng et al., 2022]. The specific experimental datasets and models are as follows:

• Permuted MNIST: We permute the MNIST handwritten digit dataset to ten tasks via random permutations of the pixels. Each task contains ten classes, divided into 60,000 training samples and 10,000 test samples. As for the SNN model, we use the SNN with two convolutional layers, one fully-connected layer, and the multi-headed output layer.
• Permuted N-MNIST: We randomly permute the NMNIST ( the neuromorphic capture of MNIST) to ten tasks. And we employ the same sample division and the same SNN structure as MNIST.
• Split CIFAR100: The more complex natural image dataset CIFAR100 is trained in several splits including 10 steps (10 new classes per step), 20 steps (5 new classes per step). SNN model consisting of eight convolutional layers, one fully connected and multi-headed output layer are used to generate the predicted class.

For the task classifier, we use networks containing a hidden layer with 100 hidden neurons for MNIST and N-MNIST, and 500 hidden neurons for CIFAR100. To recognize tasks better, we replay 2000 samples for each task as [Rebuffi et al., 2017; Rajasegaran et al., 2019b; Rajasegaran et al., 2019a]. Our code is available at https://github.com/BrainCog-X/BrainCog/tree/main/examples/Structural Development/DSDSNN.

4.2 Comparisons of Performance

As shown in Fig. 3a, our DSD-SNN model maintains high accuracy with increasing number of learned tasks. This demonstrates that the proposed model overcomes catastrophic forgetting on all MNIST, neuromorphic N-MNIST and more complex CIFAR100 datasets, achieving robustness and generalization capability on both TIL and CIL. To validate the effectiveness of our dynamic structure development module, we compare the learning process of DSD-SNN with other

DNNs-based continual learning and transfer them to SNN as

Fig. 3b. The experimental results indicate that DSD-SNN re-

alizes superior performance in learning and memorizing more

incremental tasks, exhibiting larger memory capacity com-

pared to the DNNs-based continual learning baselines.

The comparison results of the average accuracy with existing continual learning algorithms based on DNN and SNN are shown in Table 1 and Table 2. In the TIL scenario, our DSD-SNN achieves an accuracy of 97.30% ± 0.09% with a network parameter compression rate of 34.38% for MNIST, which outperforms most DNNs-based algorithms such as EWC [Kirkpatrick et al., 2017], GEM [Lopez-Paz and Ranzato, 2017], and RCL [Xu and Zhu, 2018]. In particular, our algorithm achieves a higher performance improvement of 0.70% over the DEN [Yoon et al., 2018] model (which is also based on growth and pruning). For the temporal neuromorphic N-MNIST dataset, our DSD-SNN algorithm is superior to the existing HMN algorithm which combines SNN with DNN [Zhao et al., 2022c]. Meanwhile, our DSD-SNN model achieves 92.69% ± 0.57% and 96.94% ± 0.05% accuracy in CIL scenarios for MNIST and N-MNIST, respectively.

From Table 2, our DSD-SNN outperforms PathNet [Fernando et al., 2017], DEN [Yoon et al., 2018], RCL [Xu and Zhu, 2018] and HNET [Von Oswald et al., 2020], which are also structural extension methods, in both TIL and CIL scenarios for 10 steps CIFAR100. iCaRL [Rebuffi et al., 2017] and DER++ [Yan et al., 2021] achieve higher accuracy of 84.20% in TIL scenarios than our 77.92%, but they are inferior in CIL scenarios (51.40% and 55.30%) than our 60.47%. Moreover, the DSD-SNN compresses the network to only 37.48% after learning all tasks, further saving energy consumption. For 20 steps CIFAR100 with more tasks, our DSD-SNN achieves even higher accuracy 81.17% in TIL scenario and has excellent experimental results consistent with 10 steps. To the best of our knowledge, this is the first time that the energy-efficient deep SNNs have been used to solve CIFAR100 continual learning and achieve comparable performance with DNNs.

In summary, the DSD-SNN model significantly outperforms the SNNs-based continual learning model on the NMNIST dataset. On MNIST and CIFAR100 datasets, the proposed model achieves comparable performance with DNNsbased models and performs well on both TIL and CIL.

4.3 Effects of Efficient Continual Learning

Fig. 4 depicts the performance of the DSD-SNN model for task incremental learning on multiple datasets. The experimental results demonstrate that our SNNs-based model could improve the convergence speed and performance of new tasks during sequential continual learning, possessing the forward transfer capability. The newer tasks achieve higher performance from the beginning for MNIST and CIFAR100 datasets, indicating that the previously learned knowledge is fully utilized to help the new tasks. Also, the new tasks converge to higher performance faster, suggesting that the network has a strong memory capacity to continuously learn and

remember new tasks. Similar comparable results can be ob-

tained on the N-MNIST dataset.

4.4 Ablation Studies

Effects of each component. To verify the effectiveness of the growth and pruning components in DSD-SNN model, we compare the number of network parameters (Fig. 5a) and performance (Fig. 5b) of DSD-SNN, DSD-SNN without pruning, and DSD-SNN without reused growth during multi-task continual learning. The experimental results show that the number of parameters in the DSD-SNN model fluctuates up and finally stabilizes at 37.48% for CIFAR100, achieving superior accuracy on multi-task continual learning. In contrast, the network scale of the model without pruning rises rapidly and quickly fills up the memory capacity, leading to a dramatic drop in performance after learning six tasks. The above results reveal that the pruning process of DSD-SNN not only reduces the computational overhead but also improves the performance and memory capacity.

For the growth module of DSD-SNN, we eliminate the addition of connections from frozen neurons to verify the effectiveness of reusing acquired knowledge in improving learning for new tasks. From Fig. 5a and b, DSD-SNN without reused

growth suffers from catastrophic forgetting when there is no

additional conservation of sub-network masks. The scale of

the non-reused network is very small, and the failure to reuse

acquired knowledge significantly degrades the performance

of the model on each task. Therefore, we can conclude that

reusing and sharing acquired knowledge in our DSD-SNN

model achieves excellent forward transfer capability.

Effects of different parameters. We analyze the effects of different growing and pruning parameters (the growth scale ρ and pruning intensity ρc, ρf ). For the growth parameter ρ, the results are very close in the range of 5-15% for MNIST in Fig. 6a, as well as in the range of 7.5-15% for CIFAR100 in Fig. 6b. Only in the larger case, there is a performance degradation in the later learning task (8th task), due to the larger growth scale of the previous task resulting in insufficient space to learn new knowledge in the later tasks.

Fig. 6c and d describe the effects of pruning strength ρc, ρf on performance. The larger ρc, ρf , the more convolutional channels and fully connected neurons are pruned. We found that the accuracy is very stable at less than ρc = 0.50, ρf =

1.00 for MNIST and ρc = 0.75, ρf = 1.25 for CIFAR100, but the accuracy declines at larger ρc, ρf due to the overpruning. The DSD-SNN model is more adaptable to pruning parameters on the CIFAR100 dataset because it has a larger parameter space of SNN model. These ablation experiments demonstrate that our DSD-SNN is very robust for different growth and pruning parameters across multiple datasets.

5 Conclusion

Inspired by the brain development mechanism, we propose a DSD-SNN model based on dynamic growth and pruning to enhance efficient continual learning. Applied to both TIL and CIL scenarios based on the deep SNN, the proposed model can fully reuse the acquired knowledge to help improve the performance and learning speed of new tasks, and combine with pruning mechanism to significantly reduce the computational overhead and enhance the memory capacity. Our DSD-SNN model belongs to the very few explorations on SNNs-based continual learning. The proposed algorithm surpasses the SOTA performance achieved by SNNs-based continual learning algorithm and achieves comparable performance with DNNs-based continual learning algorithms.

Acknowledgements

This work is supported by the National Key Research and Development Program (Grant No. 2020AAA0107800), the Strategic Priority Research Program of the Chinese Academy of Sciences (Grant No. XDB32070100), the National Natural Science Foundation of China (Grant No. 62106261).

Contribution Statement

• B.H. and F.Z are equal contribution and serve as co-first authors. B.H., F.Z. and Y.Z. designed the study. B.H., F.Z. W.P. and G.S.performed the experiments and the analyses. B.H., F.Z. and Y.Z. wrote the paper.