SCNNs: Spike-based Coupling Neural Networks for Understanding Structural-Functional Relationships in the Human Brain

1 Introduction

Analyzing the relationship between structure and function in the human brain is a current hot topic in neuroscience [Sporns, 2013]. Fundamentally, the brain's structure provides the physical basis for its complex functions, while functional changes may prompt structural adjustments in turn [Anderson and Finlay, 2014]. Most studies have indicated a close interconnection and synergy between the brain's regional structures and functions [Park and Friston, 2013; Pessoa, 2014; Petersen and Sporns, 2015]. In addition, the relationship between the local optimization function of the brain and the pattern of neuronal activity has prompted researchers to try to imitate the structure and function of the brain and develop artificial neural network models [Kanwisher et al., 2023]. Therefore, in-depth research on the brain structure-function relationship aids in understanding brain complexity and provides crucial theoretical support for artificial intelligence.

∗ Corresponding author

Structural-functional brain connectivity coupling (SCFC coupling) is a new paradigm for analyzing structuralfunctional relationships [Fotiadis et al., 2024]. Structural connectivity (SC) usually refers to the physical connection between brain regions [Zhang et al., 2024], while functional connectivity (FC) refers to the temporal co-activation between activities in different brain regions [Gao and Wu, 2016]. SC-FC coupling integrates cross-information of structural and functional brain networks and can detect abnormalities more sensitively than relying on either network alone [Ma et al., 2021]. Existing studies have shown that changes in SCFC coupling are considered potential biomarkers [Popp et al., 2024] and are closely related to the occurrence of a variety of brain diseases, such as schizophrenia, epilepsy, Alzheimer's disease, and Parkinson's Disease, etc [Zhu and Yu, 2025; Sun et al., 2017]. In addition, SC-FC coupling has been shown to be a unique feature of brain organization, which is affected by genetic factors and provides important clues for understanding changes in cognitive function at different developmental stages [Gu et al., 2021].

A variety of methods have been developed over the past decade to quantify SC-FC coupling, including statistical models, biophysical models, and communication models [Zamani Esfahlani et al., 2022; Griffa et al., 2022]. However, the complex relationship between structural and functional connectivity remains elusive, and current state-of-the-art SCFC coupling methods have not fully considered the neurobiological mechanisms between SC and FC. Specifically, many SC-FC coupling methods primarily focus on finding statistical correlations between SC and FC topology patterns, lacking principled system-level integrated to characterize the coupling mechanism of how neuronal populations communicate and generate significant brain function on top of structural connectivity [Dan et al., 2023]. Therefore, to more accurately describe the SC-FC coupling relationship, models more consistent with neurobiological principles must be employed.

To this end, we propose a biologically inspired deeplearning model called spike-based coupling neural networks (SCNNs), which uses spike neurons to characterize the dynamic interaction behavior of SC and FC to understand the SC-FC coupling relationship in the human brain. In addition, to establish the interaction mechanism between SC and FC in the brain, that is, how to accurately quantify SC-FC coupling, we focus on designing a new spike coupling pooling (SCP) module to effectively integrate the interaction between brain structure and function. The schematic diagram of our proposed method is shown in Figure 1. Specifically, we use spike neurons to capture the FC temporal characteristics of the original fMRI time series and the SC spatial characteristics of the structural brain network, and use synaptic and neuronal filter effects to simulate the coupling mechanism of SC and FC in the brain at different temporal and spatial scales. Finally, we fuse SC and FC spike trains through the SCP module to obtain spike SC-FC coupling for brain disease identification. In this work, our contributions are summarized as follows:

• We propose an end-to-end spike train-based framework for analyzing structural and functional relationships, called spike-based coupling neural networks (SCNNs). To the best of our knowledge, the proposed SCNNs are the first deep-learning model that uses spike neurons to understand the SC-FC coupling mechanism from a neurobiological perspective.
• We design a novel low-rank prior-based spike coupling pooling (SCP) module to quantify SC-FC coupling by fusing the variation characteristics of SC in the spatial dimension and the temporal variation of FC in the temporal dimension.
• We evaluate the performance of the proposed method us-

ing a real epilepsy dataset, and the results show that the proposed method outperforms existing methods.

2 Method

2.1 Subjects and Data Preprocessing

In this study, we recruited 103 frontal lobe epilepsy (FLE) patients (53 men, 50 women, mean age 24.1), 89 temporal lobe epilepsy (TLE) patients (44 men, 45 women, mean age 25.9), and 114 normal controls (NC) (58 men, 56 women, mean age 26.2). The experiment was approved by the Ethics Committee of Jinling Hospital, Nanjing University Medical School. All subjects provided written informed consent before participation. All subjects were right-handed. The data was collected by a Siemens Trio 3 T magnetic resonance scanner. We used DPARSF and spm12 toolboxes to process the fMRI data, including time correction, alignment, normalization, and detrending and then applied a bandpass filter of 0.010.08 Hz. The resulting volume had 240-time points and was divided into 90 regions of interest (ROI) using the automatic anatomical labeling (AAL) atlas. We used PANDA and FSL toolboxes to process Diffusion Tensor Imaging (DTI) data, including distortion correction, extraction of brain masks, deterministic tracking, and definition of anatomical regions based on AAL, with fiber count representing connectivity strength.

2.2 Spiking NeuronModel

The general Leaky Integrate-and-Fire (LIF) neuron [Gerstner and Kistler, 2002] only focuses on the membrane potential dynamics, ignoring synaptic dynamics and SNN filtering, it is difficult to handle the temporal and long-term dependency characteristics in SC and FC. Thus, we use infinite impulse response (IIR) filters to represent the dynamics of LIF neurons [Fang et al., 2021], using their recursive structure to capture the long-termmemory effect of the signal and handle the historical dependency and temporal feedback in SC and FC.

We denote the input spike train as a sequence of timeshifted Dirac delta functions: xi = ∑j δ(t - tji ), where tji denotes the arrival time of the jth spike from the ith input synapse. Similarly, the output spike train can be defined as O(t) = ∑ δ(t - tf ), tf ∈ {tf : v(tf ) = Vthre}. Thus, the SNN model is reformulated as a linear constant coefficient difference equation:

where t is the time, v[t] is the neuron membrane potential, Vthre is the threshold potential, ωi is the associated weight of the ith synapse, M is the total number of synapses, r[t] is the reset filter, τr controls the decay speed of the reset impulse, fi[t] is the second-order IIR filter of the ith synapse, α1 = e -1 τm + e -1 τs , α2 = -e- τm+τs τmτs , β = e -1 τm - e -1 τs , τm and τs are time constants, and Hea(·) is the Heaviside step function, if x ≥ 0, Hea(x) = 1, otherwise Hea(x) = 0.

In biological systems, axons often connect multiple target neurons, and their synapses share the same state [Luo, 2021; Shen et al., 2021]. Therefore, tracking the state of synapses with the same neuron is unnecessary, because these synapses can share the same state and computation. Furthermore, the state of neuron v[t] may also depend on past states, which enables SNN to be interpreted as IIR filter networks with nonlinear neurons:

where i and l represent the index of the neuron and layer respectively, j represents the input index, and Nl is the number of neurons in the lth layer. V l i [t] is the neuron membrane potential, Rli [t] is the reset voltage, Oli [t] is the spike function, and F l j [t] is the postsynaptic potential. η and λ are the coefficients of the neuron filter and the reset filter respectively. P and Q represent the feedback and feedforward orders, while αlj,p and βlj,q are the coefficients of the synaptic filter. We show the SNN model simulated by IIR filters in Figure 2(a). It is worth noting that better to simulate the dynamic behavior of biological neural networks, we use alpha synapses [Ye, 2021] to simulate neuron, where α1 = 2e- 1 τr , α2 = -e- 2 τr , αp = 0, p ∈ {3, 4, ..., P} and β1 = 1 τr e- 1 τr , βq = 0, q ∈ {0, 2, 3, ..., Q}.

Let Axon(·) represent the operation of transmitting a spike to the next neuron, which can be simulated by a second-order exponential IIR filter in Eq. 8. Let SN(·) represent the behavior of the neuron updating the membrane potential and firing spikes, as shown in Eq. 5 to 7. Therefore, the SNN model simulated by the IIR filter can be divided into two operations, Axon(·) and SN(·). For ease of understanding, we provide a specific example in Figure 2(b).

2.3 Spike FC Feature ExtractionModule

Based on the advantages of SNN in accurately capturing spike train and temporal patterns [Guo et al., 2023], we adopt a temporal coding method to encode the signal through precise spike timing, effectively representing FC information in the original fMRI time series. In addition, we use the SNN model simulated by IIR filters to capture the time series features in fMRI. Let Gfc ∈ RN×T represent the time series matrix of fMRI, N correspond to the set of brain regions nodes, and T correspond to the length of the time series. We use Gfc as the functional input of SCNNs and obtain the FC spike matrix through the Axon(·) and SN(·) operations:

where l is the layer index, here L = 3. When l = 1, Sl-1 = Gfc . The final output FC spike matrix is denoted as Sfc ∈ RN×T . We show the detailed process for calculating the spike FC in Figure 1(b).

2.4 Spike SC Feature ExtractionModule

Considering the graph structure of structural connections, this study uses Convolutional Neural Networks for Brain Networks (BrainNetCNN) [Kawahara et al., 2017] to obtain complex features from structural brain network data. Specifically, we use E2E (Edge-to-Edge) and E2N (Edge-to-Node) convolutions to extract features from structural brain networks hierarchically. We will introduce these two convolution operations in detail. Let Asc ∈ R|N|×|N| denote the adjacency matrix of SC and N correspond to the node set of brain regions.

E2E Convolution. E2E convolution is used to extract edge features from the original adjacency matrix. It takes advantage of the topological structure of the network and performs filtering operations by combining the weights of the edges of shared nodes. Its mathematical form is expressed as follows:

where rn ∈ R|N| and cn ∈ R|N| are the learned weights, f n represents the nonlinear activation function of the nth filter.

E2N Convolution. E2N convolution aggregates edge features into node features, which extract information from the neighboring edges of each node and generate a representation of each node. Therefore, the output of the E2N layer is defined as:

where rm,n d ∈ R|N| is the learned weight, (m, n) represents each pair of input and output feature maps of each layer.

The features of SC can be extracted hierarchically through E2E and E2N convolution operations. As shown in Figure 1(c), the SC feature map after the E2E convolution is Qsc ∈ RN×N×C1 , and the feature map after the E2N convolution is Gsc ∈ RN×C2 , where C1 and C2 are the number of filters for the two convolution operations, respectively.

We use the SC feature map Gsc after E2E and E2N convolution operations as the structural input of SCNNs and perform implicit spike encoding through internal spike neurons. Figure 1(c) shows the detailed process of calculating spike SC. We obtain the SC spike matrix through the SN(·) operation and the linear network:

where Xsc is the final output SC spike matrix, D is the feature dimension and T is the time step. It is worth noting that the choice of T is determined by the length of the fMRI time series. V sc is obtained by passing Gsc through a linear network after T repetitions. ˜ V sc is obtained by passing it through SN(·), so it is a binary spike tensor.

2.5 Spike Coupling PoolingModule

After the above operations, we obtain the FC spike matrix Sfc and SC spike matrix Xsc containing the functional and structural information of the brain network. Since Sfc and Xsc are high-dimensional binary spike signals, direct calculation may lead to increased memory cost and the risk of overfitting [Yu et al., 2017]. Thus, we propose a spike coupling pooling (SCP) module based on a low-rank prior to effectively quantify the coupling relationship between FC and SC by low-rank decomposition, thereby reducing the amount of computation and improving model efficiency.

The functional and structural inputs of SCP can be described as Sfc = (s0, ..., st, ..., sT ), st ∈ RN and Xsc = (x0, ..., xt, ..., xT ), xt ∈ RN respectively. For ease of explanation, we focus on the processing at time t (t ∈ [0, T]). Specifically, given the FC feature vector st and the SC feature vector xt at time t, the simplest bilinear transformation is defined as ht = s(tr) t Wxt ∈ R, where W ∈ RN×N is the projection matrix, (tr) represents the transpose operator. To obtain a o-dimensional output, we need to learn W ∈ RN×N×o , which results in a large number of parameters. However, W can be factorized into two low-rank matrices:

where k is the factorization rank, ⊗ is the element-wise multiplication (Hadamard) of two eigenvectors, 1 ∈ Rk is an all-ones vector, P ∈ RN×k , Q ∈ RN×k . Therefore, to obtain an output feature vector ht ∈ Ro , two 3-D tensors ˜ P = [P1, P2, ..., Po] ∈ RN×k×o and ˜Q = [Q1, Q2, ..., Qo] ∈ RN×k×o are required. Without loss of generality, we can reformulate ˜ P and ˜Q as 2-D matrices P ′ ∈ RN×ko and Q′ ∈ RN×ko respectively with simple reshape operation. The final fused vector hsc-fc t ∈ Ro is obtained by performing sum pooling on the projection vectors using P ′ and Q′ with a 1-D non-overlapping window of size k:

As shown in Figure 1(d) and Figure 3, the above operation generates a feature vector hsc-fc t that combines FC temporal information and SC spatial information. It only represents the coupling relationship between FC and SC at time t. After repeating the above operation T times, we obtain a spike matrix Hsc-fc ∈ RT×o consisting of T feature vectors, which we call spike SC-FC coupling. It not only contains temporal FC features, but also retains spatial SC features, and fully characterizes the coupling relationship between FC and SC. Finally, we can get the spike ⃗Z[t] ∈ Rm fired by each output neuron over time through the linear network and SN(·) operation. It is a vector that changes over time, where m is the number of output classes.

2.6 Implementation

Let ⃗ p ∈ Rm count the spikes of each output neuron, which can be obtained by summing ⃗Z[t] over T time steps:

The index of ⃗ p with the maximum count corresponds to the predicted class ˆ y:

We provide an example in Figure 1(e), where neuron 1 fired 6 spikes, neuron 2 fired 3 spikes, and neuron 3 fired no spikes. Compared to neurons 2 and 3, neuron 1 has the maximum spike count, and the corresponding prediction class is 1. In addition, to train the SCNNs architecture, we use the classification cross-entropy loss function Lc. For binary classification tasks, the cross-entropy loss is calculated as follows:

where y is the true label.

3 Experiments and Results

3.1 Experimental Setup

The proposed model and algorithm are implemented in PyTorch and snnTorch [Eshraghian et al., 2023], and the GPU used in training is NVIDIA RTX 4090. We first briefly introduce the architecture of BrainNetCNN. The input of the BrainNetCNNmodel is the structural brain network Asc , represented as a 90 × 90 adjacency matrix. In our experiment, the E2E layer is composed of [32, 1 × 90] and [32, 90 × 1] filters producing feature maps of size 90 × 90 × 32. Next is the E2N layer, composed of [64, 1 × 90 × 32] filters producing SC feature maps of size 1 × 90 × 64. For the design of SCNNs, we employ synapse model depicted by Eq. 8, in which τr = 0.9, Vthre = 1, η = 1, λ = e- 1 τr , T = 240. During training, we use a dropout of 0.1 and use the Adam optimizer with hyperparameters θ1 = 0.9 and θ2 = 0.999. The learning rate is 1 × 10-3 , the training epochs are 100, factorization rank k = 3.

We mainly perform two binary classification tasks: FLE vs. NC and TLE vs. NC task. The performance of our method is evaluated with the following metrics: accuracy (ACC= TP+TN TP+TN+FP+FN ), sensitivity (SEN= TP TP+FN ), specificity (SPE = TN TN+FP ), positive predictive value (PPV = TP TP+FP ), negative predictive value (NPV = TN TN+FN ), and balanced accuracy (BAC= SEN+SPE 2 ). FP, TP, FN and TN represent falsepositive, true-positive, false-negative, and true-negative classification results. It is worth noting that the selection of parameters and other experiments are performed using a fivefold cross-validation strategy.

3.2 Methods for Comparison

To verify the effectiveness of the SCNNs method, we compare SCNNs with existing SC-FC coupling analysis methods. For convenience, we divide the comparison methods into three types and briefly introduce the basic characteristics of these three types of comparison methods below.

The first type is the traditional SC-FC coupling methods based on classifiers, mainly including i) Direct statistics (Statistics) [Chiang et al., 2015], ii) Multi-kernel learning method (Multi-kernel) [Dyrba et al., 2015], iii) Multilinear principal component analysis (MPCA) [Huang et al., 2020], iv) Self-calibrated [Lei et al., 2020] and v) Coupling in Brain SC and FC (C-BSF) [Liu et al., 2024]. These methods focus on modeling and analyzing SC-FC coupling using statistics and classical machine learning algorithms, usually relying on feature extraction and selection, combined with support vector machines, random forests, and other machine learning classifiers for prediction and classification.

The second type is methods based on deep learning (DL), mainly including i) Diffusion-convolutional neural networks (DCNN) [Atwood and Towsley, 2016], ii) Multi-view graph convolutional network (MVGCN) [Zhang et al., 2018], iii) Deep neural network (BrainDNN) [Sarwar et al., 2021], iv) Supervised graph representation learning (Brain-GRL) [Li et al., 2022], v) Multicenter and multichannel pooling graph convolutional network (MMP-GCN) [Song et al., 2022], vi) Graph convolutional network (Brain-GCN) [Xia et al., 2024] and vii) SC-FC detour network (SFDN) [Wei et al., 2024a]. These methods use deep neural networks to extract complex features in brain network data, and gradually extract higherlevel features through a multi-level network structure, thereby improving the prediction accuracy of the model.

The third type is to introduce neuroscience improvement methods into DL methods, mainly including i) Deep Kuramoto model [Dan et al., 2023], ii) Neural Koopman operator [Chow et al., 2024] and iii) Neural pathway transformer (NeuroPath) [Wei et al., 2024b]. These methods combine neurobiological mechanisms and improve the neural network structure by simulating the activity and connection mode of brain neurons, so as to better capture biological characteristics and laws. This type of method has certain similarities with the SCNNs we proposed in terms of ideas, and both try to improve the performance of DL models by drawing inspiration from neuroscience.

3.3 Classification Performance

The experimental performance of each method in the FLE vs. NC and TLE vs. NC tasks is shown in Table 1. Specifically, the ACC of the SCNNs method proposed by us for epilepsy identification reaches 95.16% and 94.17% in two tasks, respectively. In comparison, only the third type of method (Based on DL and neuroscience) has an ACC of more than 90%, while the first type of method (Based on classifier) and the second type of method (Based on DL) do not reach 90%. By introducing neuroscience and biological mechanisms, the disease prediction accuracy of the model is significantly improved, and SCNNs perform best among these methods.

From the perspective of other evaluation indicators, in the FLE vs. NC task, the SEN of the SCNNs method is 3.05% higher than that of the best NeuroPath method among other methods. Similarly, in the three indicators of PPV, NPV, and BAC, the SCNNs method shows improvements of 0.68% to 2.08%. In addition, the SPE of the Kuramoto method is slightly higher than that of SCNNs, with a difference of 0.16%, while the SPE values of other methods are lower than that of SCNNs. In the TLE vs. NC task, although the NeuroPath method outperforms SCNNs in the SPE and NPV indicators, the SCNNs method performs best in SPE, PPV, and BAC. Overall, the comprehensive performance of our proposed SCNNs method in identifying epilepsy is significantly better than that of other methods.

3.4 Spike SC-FCCoupling Statistical Analysis

To demonstrate the effectiveness of the SC-FC coupling features obtained by our method, we performed statistical analysis. Specifically, in the experiment, we count the total number of spike SC-FC couplings fired in 90 brain regions of FLE, TLE patients, and normal controls in the test sets of the two tasks. For ease of observation, we show in Figure 4 the total number of spike SC-FC couplings fired in 90 brain regions in the test set in the FLE vs. NC and TLE vs. NC tasks, as well as the visualization results of the spike SC-FC couplings generated by a single subject. It is clear from Figure 4 that, within 240 time points, almost every brain region of normal controls fires a large number of spikes, while some brain regions of FLE and TLE patients do not fire spikes, and other brain regions only fire a small number of spikes. We speculate that epilepsy disrupts information transmission and cognitive circuits within the brain, interfering with normal communication between brain regions. Abnormal electrical activity may also cause damage to neurons, thereby affecting the normal performance of spike SC-FC coupling.

To further verify the reliability of the results, we perform a statistical analysis on the total number of spike SC-FC couplings fired by patients and normal controls across 90 brain regions in the two tasks of FLE vs. NC and TLE vs. NC. The results show that the difference between patients and normal controls is statistically significant. We present the five brain regions with the most significant differences and their box plots in Figure 5. It is clearly visible from Figure 5 that the number of spike SC-FC couplings fired by normal controls in these five brain regions is large and densely distributed, while the spike SC-FC couplings fired by FLE and TLE patients in these regions are significantly fewer and more sparsely distributed. Some previous studies have also confirmed similar findings, although these studies mainly focus on traditional SC-FC coupling rather than SC-FC coupling in the form of spike trains. For example, [Sinha et al., 2023] and [Zhou et al., 2024] report that, compared with normal controls, the SCFC coupling in certain brain regions of epileptic patients exhibits weaker responses or inactive states. [Shah et al., 2019] and [Wang et al., 2015] point out that SC-FC coupling in patients with epilepsy is significantly reduced compared with normal controls, particularly during epileptic seizures, where the reduction trend becomes even more pronounced. These findings are consistent with our current results, which show that the reduction in SC-FC coupling corresponds to a decrease in the number of spike SC-FC couplings fired. Additionally, the brain regions with significant differences (such as PreCG.L, SMA.L, and ACG.L in the FLE vs. NC task, as well as HIP.L and PCG.L in the TLE vs. NC task) are also shown to be associated with frontal lobe [O'Muircheartaigh and Richardson, 2012] and temporal lobe epilepsy [Pitk¨anen and Sutula, 2002]. These results further demonstrate that epileptic seizures are closely linked to impaired connectivity and abnormal electrical activity patterns between brain regions, thereby confirming the reliability of our findings.

3.5 Spike Coupling PoolingModule Validity

To verify the effectiveness of the spike coupling pooling (SCP) module in SCNNs, we conduct ablation experiments on the same dataset. Specifically, we design the following three multimodal fusion and two single-modal comparison methods: i) SC-FC: retain the SCP module without any changes, ii) Hadamard: remove the low-rank decomposition and pooling operations in the SCP module, and only use the Hadamard product to calculate the spike SC-FC coupling, iii) SC+FC: concatenate and fuse the spike FC and SC and input them into the classification layer to obtain accuracy, iv) FC: train SCNNs using only single-modal FC (original fMRI time series), v) SC: train SCNNs using only single-modal SC.

The results are shown in the radar chart of Figure 6, we find that in both tasks, the multimodal fusion method consistently outperforms the single-modal method, and the SCFC method with the SCP module retained achieves the best performance among the multimodal fusion methods. Specifically, in the FLE vs. NC task, the ACC of the single-modal methods, FC and SC, is 71.54% and 66.47%, respectively. In the multimodal methods, the ACC of Hadamard and SC+FC is 81.48% and 76.54%, respectively, both significantly better than the single-modal methods. However, the ACC of the SCFC method reaches 95.16%, outperforming both Hadamard and SC+FC. Similarly, in the TLE vs. NC task, the SC-FC method with the SCP module retained also demonstrates a clear advantage. In addition, we find that the performance of single-modal FC is slightly better than that of single-modal SC, which may be because the original fMRI time series contains more dynamic information related to epilepsy, making FC more prominent in extracting effective features. These results indicate that both FC and SC are indispensable, and that the spike SC-FC coupling generated by the fusion of the two comprehensively considers structural and functional information and can more accurately capture the dynamic characteristics of brain networks.

4 Conclusion

This study proposes spike-based coupling neural networks (SCNNs) for understanding the SC-FC coupling relationship in the human brain, which simulates the interaction of SC and FC in the brain through dense interconnection of spiking neurons and synaptic weights, and quantifies SC-FC coupling using spike timing and network activity characteristics. Our experiments on a real epilepsy dataset demonstrate the superior performance of SCNNs in brain disease diagnosis and provide a new perspective for SC-FC coupling analysis.

Acknowledgments

This study was supported in part by the National Natural Science Foundation of China (No. 62471259, 62472228, 62406153, 62376123), and the Natural Science Foundation of Jiangsu Higher Education Institutions of China (No. 24KJB520032, 23KJB520031).

Contribution Statement

Shaolong Wei and Shu Jiang contributed equally to this work.