Brain-inspired Balanced Tuning for Spiking Neural Networks

1 Introduction

Decoding brain on both structural and functional perspectives has lasted for centuries. In this procedure, many inspirations from the brain have contributed to the research of Artificial Intelligence (AI). For example, Hopfield network with recurrent connections is inspired by the Hippocampus; Hierarchical temporary memory (HTM) network with micro-column structures is inspired by the neocortex; Convolutional neural network (CNN) with hierarchical perception is inspired by the primary visual cortex; Reinforcement learning with dynamic acquisition of online rules is inspired by the basal ganglia centric pathway.

∗Tielin Zhang and Yi Zeng contribute equally to this article and should be considered as co-first authors.

Many Artificial Neural Network (ANN) models are with brain-inspirations at different level of details. And they have shown their power on various tasks, such as image caption, language translation [LeCun et al., 2015] and the game Go [Hassabis et al., 2017].

However, the tuning methods of back propagation in ANNs are facing challenges on preventing overfitting, improving transferability, and increasing convergence speed. The firerate neuron models in ANNs are also short at processing temporal information which makes them hard to be with good self-stability. The principles of neurons, synapses, and networks in biological systems are far more complex and powerful than those used in ANNs [Hassabis et al., 2017]. It has been proved that even a single biological neuron with dendritic branches needs a three-layered ANN for finer simulations [H¨ausser and Mel, 2003].

The intelligence of biological systems is based on multiscale complexities, from microscale of neurons and synapses to the macroscale of brain regions and their interactions. At the microscale, the neurons in biological systems represent or process information by discrete action potentials (or spikes). It raises an open question that how these discrete neuron activities interpret continuous functions, or from a similar point of view, how these network with non-differential neurons could be successfully tuned by biological learning principles. Understanding these mechanisms of biological systems will give us hints on the new biological-plausible tuning methods [Abbott et al., 2016].

Compared to other neural network models, Spiking Neural Networks (SNNs) are generally more solid on biological plausibility. The SNNs are considered to be the third generation of neural network models, and are powerful on the processing of both spatial and temporal information [Maass, 1997].

Neurons in SNNs communicate with each other by discontinuous spikes, which raises gaps between spikes and behaviors but also narrow down the multi-level integration challenges since the spikes could be considered as naturally interactive signals. In SNNs, the neurons will not be activated until the membrane potentials reach thresholds. This makes them energy efficient.

The diversity of neuron types (e.g. excitatory and inhibitory neurons) also enables SNNs to keep balance, which will, in turn, help SNNs on efficient learning and forming specific functions. In addition, different computing costs in neurons and synapses cause various kinds of time delays which will also contribute to the asynchronous computation of SNNs, since these kinds of delays will open up a new temporal dimension on SNN for better representation capacity. SNNs have been well applied on XOR problem [Sporea and Gr¨uning, 2013], visual pattern recognition [Diehl and Cook, 2015; Zeng et al., 2017], probabilistic inference [Soltani and Wang, 2010] and planning tasks [Rueckert et al., 2016].

Although SNNs have shown more biological plausibility than conventional ANNs, from the computational perspective, lack of efficient and biological plausible learning methods in the current SNN models limits their values to support understanding the nature of intelligence and potential applications.

With respect to this, some efforts have been made to train the networks by biological plausible principles. Long-Term Potentiation (LTP), Long-Term Depression (LTD), Short Term Plasticity (STP) which includes Short Term Facilitation (STF) and Depression (STD), Hebbian learning, Spike Timing Dependent Plasticity (STDP), lateral inhibition, synaptic scaling, synaptic redistribution, and many other braininspired learning principles from biological nervous systems are proposed and applied on the training procedure of SNNs [Abraham and Bear, 1996]. Nevertheless, there is still gaps for SNNs in specific applications when compared with ANN models. More efficient and comprehensive learning frameworks for SNNs need to be proposed and applied.

In this paper, we propose brain-inspired balanced tuning for SNNs (Balanced SNN for short), we will tune the SNNs based on three inspirations from the brain: Firstly, the biological network will be trained towards the equilibrium states for the membrane potentials. Secondly, the proportion of excitatory and inhibitory neurons need to make a balance and cooperate together for better tuning. Thirdly, the STP is used to keep the synaptic input-output balanced towards a better learning convergence. These inspirations are introduced as three balance principles, namely, the Membrane Potential (MP) based balance, the Excitatory-Inhibitory neuron type (E-I) based balance and the STP based balance. With these inspirations, we start to train SNNs with three steps: training the membrane potential of SNNs with three braininspired principles; then a weakly supervised learning is used to tune the membrane potential in the final layer; finally consolidating the learned membrane potential information into synaptic weights by STDP. The MNIST benchmark is used to test the performance of the proposed model.

2 RelatedWorks

Zenke et al. showed that the interaction of Hebbian homosynaptic plasticity with rapid non-Hebbian heterosynaptic plasticity would be sufficient for memory formation, and then memory could be recalled after a brief stimulation of a subset of assembly neurons in a spiking recurrent network model [Zenke et al., 2015].

Alireza et al. proposed a local learning rule supported by the theory of efficient, balanced neural networks (EBN) for the tuning of recurrent spiking neural networks. An additional tight excitatory and inhibitory balance is maintained for the spiking efficiency and robustness [Alemi et al., 2018].

Zeng et al. proposed seven biologically plausible rules to train multi-layer SNNs with Leaky-Integrated and Fire (LIF) neurons, which includes more local principles such as dynamic neuron allocations, synapse formation and elimination, various kinds of STDPs, and also more global learning principles such as background noise influence and the proportion of different kinds of neurons [Zeng et al., 2017]. It has been proved that the synaptic weights in first few layers of SNNs could be dynamically updated by STDP rules without any supervision, and the weights between the final two layers could be supervised and learned by Hebb's law.

Diehl et al. trained an SNN with conductance-based synapses, STDP, lateral inhibition, and adaptive spiking threshold, and used an unsupervised learning scheme to train a two-layered SNN. Finally, the accuracy reached 95% on the MNIST benchmark [Diehl and Cook, 2015].

Some other efforts get around of the direct training of SNNs by equivalent converting of learned synaptic weights from ANNs into SNNs. Diehl et al. try to convert deep ANNs into SNNs directly and keep the minimum performance loss in the conversion process, the key techniques include the limitation of rectified linear units (ReLUs) with zero bias and weight normalization into a linear range [Diehl et al., 2015]. Although this method could achieve the performance of 98.48% on 10-class hand-written digit MNIST classification task, the performance of SNN is actually contributed by ANN from backpropagation instead of pure biological plausible SNN learning.

Lee et al. argued that the discontinuities between spikes could be considered as noises, and the SNN without noises are continuous and could use backpropagation for training. A new architecture based on this idea is tested on MNIST and N-MNIST dataset, and a better performance and a faster convergence are achieved compared with conventional SNNs [Lee et al., 2016].

SpikeProp is an error-backpropagation based supervised learning algorithm for the training of spiking networks which could actually be considered as equivalent exchanges from spatial information (i.e. fire rates) in ANN to the temporal information (i.e. timing of inter-spike intervals) in SNN [Bohte et al., 2002].

In our efforts, we aim to minimize artificial rules and principles, and incorporate more biological plausible tuning mechanisms. We want to understand whether more solid biological principles could bring current brain-inspired SNN models to the next level and computationally enhance our understanding of learning in the brain, and be more practically efficient in AI applications.

3 The Architecture of Balanced SNNs

Different SNNs are with different structures (e.g. recurrent, or feed-forward connections) and learning methods. The recurrent networks are usually designed for temporal information processing and the multi-layered networks are mainly for the abstraction of spatial information.

The simplest version of feed-forward multi-layered SNNs is with two-layered architecture, which is also the first successful paradigm which could be tuned well by biologically plausible learning principles [Diehl and Cook, 2015]. A three layered SNN is constructed and trained by seven braininspired learning principles [Zeng et al., 2017], which shows unique contributions of different brain-inspired principles.

In this paper, we use similar building blocks as in [Zhang et al., 2018] which uses the LIF neuron model for temporal information processing and feedforward three-layered SNN for information integration as the basic structure.

3.1 The Basic LIF NeuronModel

The LIF neuron model is the basic building block of the SNN model in this paper. It's function is for non-linear information integration and non-differential spikes generation.

As shown in Figure 1, when pre-synaptic neuron fires, the spikes are generated and propagated into post-synaptic neurons. We use V to represent V (t) for simplicity. The dynamic function of membrane potential in LIF will be integrated with dt. The Cm is the membrane capacitance, the gL is the leaky conductance, VL is the leaky potential, and Isyn is the input stimulus (converted from spikes by synapses) from presynaptic neurons.

The gE is the excitatory conductance, gI is the inhibitory conductance, VE and VI are the reversal potentials for excitatory and inhibitory neurons respectively, and τm = Cm gL .

The value of Isyn in Equation (2) will be updated by the pre-synaptic spikes. The wj,i is the connection weight from pre-synaptic neuron j to the target neuron i, δt denotes the pre-synaptic spikes (in the next step it will be updated into non-differential potential Vj). The fsyn is the function from membrane potential (or spikes) to currents, which could be a decay factor or an STP function.

3.2 The Multi-layered Balanced SNN

As shown in Figure 2, a multi-layered feed-forward SNN (for simplicity here we use three-layered SNN) is constructed. The neurons in the first layers are non-LIF neurons which only receive the signals from inputs and output signals to the next layer directly without decay.

The neurons in the second layers are LIF neurons with both excitatory and inhibitory types. For the excitatory neurons, all of the output synapses are excitatory (with positive values). On the contrary, for the inhibitory neurons, all of the output synapses are inhibitory (with negative values). Synapses will receive spikes from pre-synaptic neurons and then send positive (or negative) spikes to postsynaptic neurons after firing. The proportion of inhibitory neurons in the second layer will be predefined.

The neurons in the final layer are all of excitatory LIF neurons which could receive both inputs from pre-synaptic neurons and also the additional teaching signals. The teaching signals will be updated synchronously with the network inputs.

4 Brain inspired Balanced Tuning

Most of the cognitive and functional neural systems tend to keep balanced states for better adaptability and stability. Here we introduce three brain-inspired balance principles: the Membrane Potential (MP) based balance, the ExcitatoryInhibitory neuron type (E-I) based balance and the STP based balance.

4.1 Membrane Potential based Balance

Here we focus on the membrane potential Vi as one of a main balanced variable for tuning. Membrane potential is a temporal dynamical variable which works for the function of information integration or abstraction. Here we define ∆Ei = dEi dt as the energy representation of temporal differential states of neurons.

As shown in Equation (3), the first term after equality sign is the current membrane potential of neuron i, and the second term is the future membrane potential of neuron i which integrates all of the inputs from pre-synaptic Vj. The network will learn dynamically towards network convergence, and with training time going by, the current states and next states of neurons will become equivalent, which means the ∆Ei will be around zero.

Considering that in our work, the membrane potential Vi has already taken the place of wj,i on network tuning (the information will be consolidated from Vi to wj,i in final steps), we update Equation (3) into Equation (4) according to the differential chain rule.

Finally Equation (5) describes the detailed update mechanisms of Vi based on the membrane potential based balance.

4.2 Excitatory-Inhibitory Neuron Type based Balance

In the biological brains, the proportion of excitatory neurons is larger compared to inhibitory neurons. These two types of neurons are integrated together in a interactive way to make the network in balanced states [Okun and Lampl, 2009].

Different with conventional ANNs, in which the weights of output synapses from a single neuron could be both positive or negative, in our model, we follow the biological system that normally, weights of synapses have to be positive for excitatory neurons while being negative for inhibitory neurons. Considering that the initial weights of neurons have already fit for the biological conditions, we separate the procedure of weights update into two situations: the first situation is when wj,i∆wj,i ≥ 0, in which the weights of synapses will not change their symbols after update (i.e. the weights of synapses will increase for the excitatory type and will decrease for the inhibitory type); the second situation is when wj,i∆wj,i < 0, where the updated weights of synapses may change their symbols.

The first situation has already fit for the biological brains. For the second situation, we make another condition as wj,i (wj,i +∆wj,i) ≥ 0 to limit the update range of synapses (i.e. wj,i+∆wj,i) which distinguish the types of neurons (excitatory or inhibitory). The equation wj,i (wj,i +∆wj,i) ≥ 0 could also be converted into the form of -∆wj,i wj,i ≤ 1 which could be considered as a condition to the learning rate of wj,i, as shown in Equation (6).

where ηw is the learning rate for each synapse, η0 is the predefined initial learning rate, η1 is the variable in the range of (0, 1) which makes the condition of -∆wj,i wj,i ≤ 1 works (here we use η1 = 12 for simplicity). Finally, we will have the rule for excitatory and inhibitory synapses update which also works as an alternative balanced principle for network tuning.

4.3 Short-TermPlasticity based Balance

To a certain extent, the firing frequencies of neurons in SNNs are kept balanced by STP. The spiking frequency will increase by STF when the frequency is low, while will decrease by STD when the frequency is high [Zucker and Regehr, 2002]. For STF, the release of Ca2+ from synapses will increase the probability of the neuron firing next time. For STD, the highfrequency firing will cost too much energy to support spike generation next time which is very near for the last spike.

As shown in Equation (7) and Equation (8), u and x are the normalized variables which represent the dynamical characteristics of STF and STD respectively. δ(t - tsp) is the input of spikes on time tsp, τf and τd are the recover time constants of STF and STD respectively.

As shown in Equation (9), A is the maximal connection weight, τs is the recover time constants for Isyn. Isyn will be updated based on the u in Equation (7) and x in Equation (8), then the Isyn will be combined with Equation (1) for the balanced tuning.

4.4 Supervised Learning in the Final Layer

As shown in Figure 2, the neurons in the final layer of the network receive inputs from both the pre-synaptic neurons and also the teaching signals. The teaching signal is a kind of very high-frequency stimulus (the frequency will be the same as the input signals of the first layer) to the neurons in the final layer.

As shown in Equation (10), N is the number of neurons in the last layer, we set the differences of Vi and δ (t - tsp) as the cost of the network. Then it could be converted into Equation (11), where the ηc is the learning rate. V SUP i will be calculated only one time in the final layer which contains the divergence of realistic prediction and supervised teaching signals.

4.5 Equivalent Conversion fromMembrane Potential to Synaptic Weights based on STDP

The tuning of Vi and its relationship with network outputs has been discussed. However, the network has to save the learned knowledge by consolidating them from membrane potential Vi into synapses wj,i. Here we use STDP-like rules [Bi and Poo, 2001; Bengio et al., 2015] to realize this function.

The Equation (12) is a integration of two different types of STDP rules, one is ∆wj,i ∝ Vj dVi dt [Bengio et al., 2015] and another is ∆wj,i ∝ dVj dt Vi [Bi and Poo, 2001]. wj,i is the synaptic weights between neuron i and neuron j, ηSTDP is the learning rate of STDP rule, V t j and V t+1 j are the different temporal states of neuron j.

4.6 The Learning Procedure of the Balanced SNN

The training and test procedure of the balanced SNN model is shown in Algorithm 1.

Algorithm 1 The Balanced SNN Learning Algorithm.

1. Convert the spatial inputs into temporary inputs with random sampling. Initialize weights wj,i with random uniform distribution, membrane potential states Vi with leaky potential VL, iteration time Iite, simulation time T, differential time ∆t, learning rates ηMP , η0, η1, ηSTDP and ηc ;
2. Start Training procedure:
3. Load training samples;
4. Update Vi by feed forward propagation with Equation (1) and Equation (2);
5. Update V MP i by the condition of membrane potential based balance with Equation (5);
6. Update I STP syn by STP based balance with Equation (7), Equation (8) and Equation (9);
7. Update V SUP i by weak supervised learning in final layer with Equation (11);
8. Equivalent conversion from membrane potential to synaptic weights based on integrated STDP, and passively update synaptic weights wj,i based on Equation (12);
9. Update wj,i by excitatory-inhibitory neuron type based balance with Equation (6);
10. Iteratively train SNNs from Step 2.2 to Step 2.7 and finally save tuned wj,i;
11. Start test procedure:
12. Load test samples;
13. Test the performance of trained balanced SNN with feed forward propagation based on saved wj,i;
14. Output test performance without cross validation;

5 Experiments

We use the standard MNIST [LeCun, 1998] to test the proposed brain-inspired Balanced SNN model. MNIST contains

10 classes of handwritten digits with 60, 000 training samples and 10, 000 test samples.

In order to understand the mechanisms and individual contributions, we will add each individual principle gradually to compare and then integrate them together for the ultimate performance.

5.1 Performance of the Membrane Potential based Balanced Tuning on SNN

We construct an SNN and train it with only LIF based feed forward (FF) architecture and the membrane potential balanced principle based on Equation (3), Equation (4) and Equation (5).

As shown in Figure 3, the number of neurons in the hidden layer are 100, and the x-axis of the figure is the iteration time, the y-axis is the accuracy of SNN on the MNIST classification task. We could conclude from the figure that the proposed membrane potential (MP) balanced tuning principle is working for the network convergence, and the SNN could form the classification ability after the MP based balanced tuning.

5.2 Performance of the Excitatory and Inhibitory Neuron Type based Balance Tuning on SNN

The function of inhibitory neurons in the biological system is a mystery. Some of them play the role on the anti postsynaptic membrane potentials (anti-E), and some of them work on blocking activities of other neurons (Block-E) [Zeng et al., 2017]. Here we test the anti-E type of inhibitory neurons and also the proportion of them on SNNs with Equation (6) based on the tuned result of membrane potential based balanced tuning.

The test accuracies of E-I based balanced tuning on SNN is shown in Figure 4, from which we could see that the SNNs with a proper proportion of inhibitory neurons could be trained for the function of classification. When the proportion is too big (e.g. bigger than 80%), the network will be failed to learn. More concretely, as shown in Figure 5, SNN will be well tuned when the proportion of inhibitory neurons is smaller than 60%. The best proportion of inhibitory neurons is 30%.

5.3 Performance of the STP based Balanced Tuning on SNN

The STP principle will keep the neurons firing towards a stable frequency. Figure 6 shows the dynamic changes of variables of u and x with the input of spikes.

For the different frequencies of spikes, the u and x will be tuned automatically towards the stable output of Isyn which will be updated by the product of u and x in Equation (9).

As shown in Figure 7, when the iteration time is 100, the test accuracy performance on pure MP balanced tuning could reach 58%, the MP with feedforward LIF neuron model could reach 89%, and the integration of MP, FF, STP, and 30% inhibitory neurons could reach 97.9%. As a conclusion, the MP, the proportion of E-I neurons and STP are contributing the balanced effects to SNNs for a better classification accuracy.

5.4 Comparative Studies

The training of SNNs is very different with DNNs which are trained by backpropagation. Here we exclude these none biological-plausible tuning methods which firstly train DNNs by backpropagation and then convert into SNNs (e.g. the Convolutional SNN in Table 1 with the accuracy of 99.1%), since these efforts are not biological plausible, and the contributions are mainly from backpropagation.

Some state of the art performance of SNNs on the MNIST benchmark with different structures is shown in Table 1. For two-layer SNNs, the accuracy of 95% is achieved with 6, 400 output neurons [Diehl and Cook, 2015]. For three-layered SNNs, VPSNN gets the accuracy of 98.52% with 4, 800 hidden neurons [Zhang et al., 2018]. While in our work (Balanced SNN), we reach the accuracy of 97.90% with only 100 hidden neurons and also reach 98.64% with 6, 400 hidden neurons. To the best of our knowledge, our result is the state of the art performance of pure biological plausible SNNs on the MNIST benchmark.

6 Conclusion

There are various learning principles from the brain which may help to design better spiking neural network models for Artificial Intelligence. However, how to integrate these brain inspirations together properly for optimal model are still with big challenges. Here we focus on the research of balanced states of SNNs and try to integrate three kinds of balanced learning principles together. They are the membrane potential based balance, the excitatory-inhibitory neuron type based balance and the STP based balance. The model analysis supports the hypothesis that the balanced state of the network is important for network training, and the experimental result also proves that, even without backpropagation, a better SNN

performance could be achieved based on a deeper integration of brain-inspired principles.

Acknowledgments

This work is supported by Beijing Natural Science Foundation (4184103) and the CETC Joint Fund (Grant No. 6141B08010103).