Towards Recognizing Spatial-temporal Collaboration of EEG Phase Brain Networks for Emotion Understanding

1 Introduction

Decoding emotional states from brain activity is a fundamental challenge at the intersection of neuroscience and artificial intelligence [Pessoa and Adolphs, 2010; Huang et al., 2017; Kragel and LaBar, 2016]. Non-invasive neuroimaging techniques such as EEG, fMRI, and MEG have enabled fine-grained analysis of brain dynamics associated with emotional and cognitive processes [Meer et al., 2020; He and Zhang, 2024; Flournoy et al., 2024]. Among these, EEG stands out due to its high temporal resolution, which is crucial for capturing the rapid neural fluctuations underlying affective states. Nevertheless, accurately recognizing emotions from EEG signals remains a formidable task, as it requires advanced models capable of capturing complex spatial-temporal interactions within brain networks.

∗ Corresponding author.

Current EEG-based emotion recognition methods typically employ Region-of-Interest (ROI) strategies, which partition the brain into predefined regions and frequency bands. While widely used, such approaches often fail to capture the dynamic and collaborative nature of emotional neural processes [Pessoa, 2017; Pessoa, 2018; Zheng et al., 2023]. Although state-of-the-art time-series models have shown strong capability in capturing local temporal dynamics [Ju et al., 2024; Si et al., 2024], they are inherently limited in modeling inter-regional synchronization-an essential feature for decoding emotion-related brain activity. This limitation aligns with our core motivation: going beyond sequential modeling to explicitly account for spatial-temporal dependencies across brain regions. As a result, existing approaches still struggle to fully leverage the structure of brain networks, substantially restricting their performance in emotion decoding.

Brain network-based methods have shown strong potential in cognitive prediction and neurological diagnosis [Alarcao and Fonseca, 2017; Zhu et al., 2021; Shi et al., 2023]. However, applying them to EEG-based emotion recognition presents two key challenges. First, constructing effective brain networks requires capturing the dynamic, nonstationary nature of EEG signals. Most existing methods rely on static frequency bands, which constrain electrode interactions to fixed ranges [Miao et al., 2023] and overlook individual spatial-temporal variability (Fig.1-(a)). Adaptive frequency selection, guided by physiological priors, offers a promising alternative[Liu et al., 2024a]. Second, critical spatial-temporal dependencies often degrade during node aggregation, especially when temporal asynchronies between brain regions-vital for emotional processing-are not well preserved (Fig.1-(b)). Addressing these challenges calls for advanced techniques that can jointly model spatial-temporal interactions[Liu et al., 2024b].

To address these challenges, we propose the SpatialTemporal Electroencephalograph Collaboration (Stella) framework. Specifically, Stella introduces Adaptive Bands Selection (ABS) to dynamically partition EEG signals into optimal frequency bands, effectively constructing Phase Brain Networks (PBNs) to capture inter-region synchronization. Additionally, we design a dual-path architecture integrating a Fourier Graph Operator (FGO) to mitigate feature degradation, and a Spatial-Temporal Encoder (STE) to strengthen spatial-temporal representations. Comprehensive experiments on benchmark datasets (DEAP and FACED) demonstrate that Stella achieves state-of-the-art performance. Our key contributions are:

• Adaptive Bands Selection (ABS): A data-driven approach that dynamically divides EEG signals into base and harmonic frequency bands, enabling robust Phase Brain Network (PBN) construction through phase synchronization.
• Stella Framework: A dual-path architecture integrating the Fourier Graph Operator (FGO) to reduce feature degradation and the Spatial-Temporal Encoder (STE) to strengthen spatial-temporal representations.
• Comprehensive Evaluation: Extensive experiments on public EEG datasets show that Stella achieves state-ofthe-art emotion recognition, supported by detailed ablation and sensitivity analyses.

2 RelatedWork

In this section, we review key research areas relevant to our approach, including frequency band division, graph learning from brain networks, and spatial-temporal learning from EEG signals. These methods have been widely explored to enhance the analysis of EEG signals, with each contributing to the understanding of brain activity and emotion recognition.

Frequency band division. Existing methods are mainly categorized into filter-based methods driven by physiological priors and data-driven filter methods. Filter-based methods driven by physiological priors rely on well-defined physiological frequency bands, such as Delta (0.5-4Hz), Theta (48Hz), Alpha (8-13Hz), Beta (13-30Hz), and Gamma (above 30Hz) [Jenke et al., 2014]. Common implementations include bandpass filters [Li et al., 2023], wavelet transforms [Akin, 2002], and Fourier transforms [Lin et al., 2010], which effectively extract the corresponding features through predefined frequency bands. Although static frequency band divisions align with physiological paradigms, they may not fully account for individual differences, limiting the model's adaptability. On the other hand, data-driven methods, while capable of handling non-stationary signals and capturing dynamic variations, do not rely on physiological plausibility, which can affect their interpretability and alignment with known brain functions.

Graph learning from EEG brain networks. Graph Convolutional Network (GCN) extend traditional Convolutional Neural Network (CNN) by leveraging spectral graph theory, enabling the direct processing of non-Euclidean data, such as graphs [Kipf and Welling, 2016]. Despite the impressive successes of GCN in emotion recognition, the static nature of their adjacency matrices limits their ability to adapt dynamically. Dynamic Graph Convolutional Neural Networks (DGCN) [Song et al., 2018] is a GCN-based method that models multichannel EEG features using a dynamic adjacency matrix, enabling the dynamic learning of intrinsic relationships between EEG channels for enhanced emotion recognition. [Wang et al., 2018] integrate the strengths of dynamic adjacency matrix and Broad Learning Systems to design a novel architecture. [Zhang et al., 2021] impose sparsity constraints on graph to enhance emotion recognition. [Gu et al., 2023] use a Generative Adversarial Network to generate latent representations of EEG signals while combining a GCN and a Long Short-Term Memory (LSTM) network to recognize emotions from EEG signals. [Li et al., 2023] combine single-channel Differential Entropy (DE) features with cross-channel functional connectivity features to simultaneously extract temporal variations and spatial topology information from EEG data. However, existing approaches tend to ignore the feature degradation problem and underestimate the effect of channel heterogeneity in feature engineering, reducing the efficiency of graph learning.

Spatial-Temporal Learning from EEG Signals. Many EEG-based emotion recognition methods adopt hybrid architectures (e.g., GCN + LSTM, CNN + RNN) to model spatialtemporal patterns [Li et al., 2016]. Yang et al. [Yang et al., 2018] and Ma et al. [Ma et al., 2019] integrate CNNs and RNNs for joint feature learning, with Ma further introducing residual shortcuts. Tao et al. [Tao et al., 2020] enhance spatial focus via channel attention, while Yin et al. [Yin et al., 2021] and Liu et al. [Liu et al., 2024a] fuse GCNs with RNNs to better capture spatial-temporal dependencies. However, these composite designs often apply temporal modeling post hoc to graph-based features, limiting direct temporal reasoning at the raw signal level. Recent work such as DeepCN [Yi et al., 2024] emphasizes joint modeling of intra- and interseries couplings, offering deeper spatial-temporal insights. Moreover, research beyond EEG, including OptIForest [Xi-

Algorithm 1 Adaptive Bands Selection

• Input: Original EEG signal x[n] Output: Selected bands features D 1: Initialize: window function w = Hamming, window length L, number of windows S, harmonic order h 2: Apply STFT to EEG signal x[n] to obtain X(f) 3: for each channel i do 4: for each window l ∈ L do 5: fbli ← argmaxf |Xi(f)l | 6: end for 7: fbsi ← 1L ∑Ll=1 fbli , fhsi ← (fbsi )h 8: µbi , σbi ←mean and std of fbsi 9: µhi , σhi ←mean and std of fhsi 10: Compute frequency boundaries using µ and σ: f blow,i , f bhigh,i , f hlow,i , f hhigh,i 11: if fhlow,i < fbhigh,i then 12: Adjust boundaries: fhlow,i ← fbhigh,i if necessary 13: end if 14: end for 15: return D for the selected bands across all channels

ang et al., 2023] and federated isolation forests [Xiang et al., 2024], highlights the importance of capturing multi-source, distributed, and cross-domain dependencies in time series data. These perspectives inspire more integrated and systemlevel approaches to EEG-based spatial-temporal modeling.

3 The Stella Framework

3.1 Overall Architecture

As illustrated in Fig. 2, we introduce the Stella framework to effectively model spatial-temporal features in EEG signals. The proposed method comprises two key components: Adaptive Bands Selection (ABS) and a dual-path architecture integrating the Fourier Graph Operator (FGO) and SpatialTemporal Encoder (STE). Specifically, ABS adaptively divides EEG signals into optimal Base and Harmonic frequency bands, addressing the limitation of static band definitions. The Base Band is utilized by FGO to construct robust Phase Brain Networks (PBNs), capturing stable inter-regional connectivity, while the Harmonic Band is processed by STE to extract detailed spatial-temporal dynamics, effectively mitigating feature degradation.

3.2 Adaptive Bands Selection

Frequency Filter Method. While traditional approaches often rely on static frequency bands such as the γ band for emotion recognition, emerging evidence indicates that dynamic, individualized frequency bands are critical for accurately capturing neural dynamics, highlighting the limitations of a fixed-band strategy.

The Adaptive Bands Selection (ABS) module leverages signal decomposition theory to adaptively identify frequency bands across EEG channels and individuals. Using the ShortTime Fourier Transform (STFT), the raw EEG signal is decomposed into the frequency domain. For each electrode, we extract the dominant frequency (base frequency) from each windowed segment and identify its harmonics as integer multiples. This decomposition enables personalized frequency band selection that captures individual-specific spectral features. The formal process is defined in Eq. 1.

where x[n] denotes the raw EEG signal, W(·) is a window function centered at time m, and f ∈ {0, . . . ,N - 1} is the frequency bin. X l i (f) is the STFT of electrode i at window l, and fbli denotes its dominant frequency. The final base frequency fbsi is obtained by averaging across all Lwindows.

As detailed in Alg. 1, due to inter-channel variability in EEG signals, base and harmonic frequencies are computed on a per-channel basis. For each channel i, the average base frequency is calculated as µbi = 1S ∑Ss=1 fbsi , with standard deviation σbi = 1S √∑Ss=1 (fbsi -µbi )2 . The base band boundaries are defined as fblow,i = |µbi - σbi | and fbhigh,i = |µbi + σbi |. Similarly, the harmonic frequency is computed as fhsi = (fbsi )h , with harmonic band boundaries fhlow,i = |µhi - σhi | and fhhigh,i = |µhi + σhi |. This adaptive, channel-specific filtering ensures personalized and robust frequency band extraction.

Phase Brain Networks. Given the non-periodic nature of EEG signals, we construct brain networks using a sliding window strategy. To capture the topological relationships between brain regions, we compute the Phase Locking Value (PLV), which quantifies the functional connectivity between electrode pairs. Phase synchronization has been widely recognized in neuroscience as a robust indicator of functional brain connectivity, particularly in the context of cognitive and emotional coordination.

The phase synchronization matrix (PSM) G serves as the adjacency matrix of a weighted undirected graph, where stronger phase synchronization corresponds to higher edge weights. In accordance with common practice in EEG-based analysis, we use Differential Entropy (DE) features as a compact and effective representation of EEG signals in each frequency band. DE quantifies the information content of a continuous-valued signal and has been widely adopted as a standard pre-processing step in EEG emotion recognition due to its superior discriminability [Zheng and Lu, 2015]. Let Dp(X) and Dq(X) denote the DE features extracted from channels p and q, respectively. Within time window s, the instantaneous phase of each DE time series is computed via the Hilbert transform, denoted as ϕp(s,Dp) and ϕq(s,Dq). The phase synchronization between these two electrodes in window s is then computed as:

Here, ∠(X) denotes the phase of X, ejθ represents a complex exponential with phase θ, and ℑ(X) denotes the imaginary part of X. To prevent numerical instability, we apply logarithmic normalization on the PLV with a small constant α, as expressed in Eq.3.

The base frequency, as the dominant and more stable component of EEG signals, is less affected by noise and is thus used to construct the phase synchronization matrix G to reflect functional connectivity between brain regions. Harmonic components, being more noise-sensitive, are excluded from this construction. Analysis on the DEAP dataset shows that the extracted base frequency band ranges from [7, 16] Hz, covering traditional θ, α, and β bands-while harmonics span [41, 58] Hz, corresponding to γ activity. These bands are associated with distinct cognitive and emotional states: θ with relaxation and light sleep, α with calm wakefulness, β with active thinking, and γ with high-level cognition and sensory processing. The ABS module naturally aligns with this five-band theory, providing a principled basis for adaptive frequency band selection in EEG analysis.

3.3 Model Design

As illustrated in Fig. 2, we design a dual-path deep learning model that integrates Fourier Graph Operator (FGO) with a Spatial-Temporal encoder (STE) to produce the stable and dense representations.

Fourier Graph Operator. In the temporal graph model, we use the constructed temporal phase brain networks {G0, G1, G2, ..., Gn} as input and approximate the Laplacian matrix using Chebyshev polynomials for graph convolution. The recursive definition of Chebyshev polynomials Tk(x) is defined as Eq. 4. We adopt Chebyshev-approximated spectral GCNs rather than GATs for both theoretical and practical reasons. First, our PLV-based brain graphs are inherently defined in the frequency domain, making spectral methods like GCNs more aligned with the underlying graph construction. In contrast, GATs operate in the spatial domain and rely on learned attention weights across node pairs, which can be less interpretable and harder to constrain physiologically in brain network analysis. Second, Chebyshev polynomial approximation allows efficient localized filtering, avoids eigendecomposition, and enhances scalability.

The Chebyshev polynomial approximation of the graph Laplacian matrix is given by Eq.5.

where θ represents the coefficients of the Chebyshev polynomials, and K denotes the order of the polynomial. In this paper, a 5th-order Chebyshev polynomial is used to approximate the Laplacian matrix. The graph convolution operation is expressed as Eq.6.

After graph convolution, the resulting temporal features, ˆX1, ˆX2, ..., ˆXt, suffer from feature degradation, a typical issue with standard graph convolution due to the heterogeneity of EEG signals. To address this, we apply Short-Time Fourier Transform (STFT) to process these signals in the complex frequency domain, mitigating the degradation problem. The STFT operation is given by Eq.7.

where ˆXn is the time series signal at time n, ω(·) is the window function, τ is the central time point in this window and f is the frequency variable.

To perform frequency domain analysis, we apply a STFT to transition the temporal features to the frequency domain, yielding real and imaginary components. The complex convolution is then applied to these components, with the input signal x = xreal + jximag and the convolution kernel h = hreal + jhimag. The convolution operation is y = (xreal+jximag)∗(hreal+jhimag), where real and imaginary parts are convolved independently using real-valued convolutions, with kernels hreal and himag. Then, a batch norm is applied followed by a linear projection W to produce the graph representation, as shown in Eq.8.

Spatial-Temporal Encoder. The STE processes the EEG signals to capture both spatial and temporal dependencies. Channel-wise Multi-Head Attention is applied to model spatial relationships in the harmonic EEG signals. Each channel is treated independently to focus on the inter-channel relationships, allowing the model to capture connectivity between brain regions. To further enhance spatial feature extraction, convolution is applied to these channel-wise representations, capturing multi-scale spatial relationships across the EEG channels. The operations are as follows: for the input matrix H ∈ RB×D×C , where B is the batch size, C is the number of channels, and D is the feature dimension, and W is the linear projection, then we compute the attention scores for each channel using Qi = XW iQ, Ki = XW iK , Vi = XW iV . The attention scores for each channel are then computed as Eq.9.

and the output for each attention head is headi = eiVi. The outputs of all heads are concatenated and passed through a linear transformation as Eq.10.

As expressed in Eq.11, the attention output is then processed by a convolution operation to further capture the spatial features, providing the output of the spatial encoding.

The temporal dependencies are captured mainly using a 3layer RNN, which models the sequential nature of the EEG signal over time. The RNN is implemented using an off-theshelf implementation in PyTorch, and details are omitted for brevity. The output of temporal encoding is given by Eq.12.

Finally, the spatial and temporal features are concatenated, and a linear layer is applied to fuse them into a single representation, as shown in Eq.13.

Above all, by combining outputs of FGO and STE modules, we obtain features enriched with brain networks and spatial-temporal information from the EEG signals. As shown in Eq.14, fusing these features results in a stable, dense representation that significantly enhances emotion recognition efficiency.

3.4 Loss Function

EEG data is inherently noisy and often limited in quantity, making it prone to overfitting under standard cross-entropy loss, which relies on hard labels. To improve generalization, we adopt a label smoothing strategy that incorporates a smoothing factor ϵ ∈ [0, 1], transforming hard labels into soft ones. The overall loss consists of two components: the standard cross-entropy loss L1, capturing model's confidence in the correct class, and a smoothing term L2, promoting uniformity across all classes. The total loss is defined as:

where n is batch size, C is number of classes (e.g., C = 2 for valence and arousal classification), ˆ yi,j is the predicted probability for class j, and yi is the true label of sample i.

4 Experiments

4.1 Datasets

This section presents extensive experiments on two widely used EEG emotion recognition datasets:

• DEAP [Koelstra et al., 2011]: EEG (40 channels, 128Hz) from 32 participants watching 40 music videos, with ratings on arousal, valence, dominance, liking, and familiarity, plus facial and physiological data.
• FACED [Chen et al., 2023]: EEG (32 channels, 1000Hz) from 123 participants watching 28 emotion-

eliciting videos, covering nine emotion categories, with ratings on arousal, valence, familiarity, and liking.

4.2 Baselines

We compare the performance of Stella against the following EEG-based emotion recognition baselines:

• Graph models: ECLGCN [Yin et al., 2021], STGCLSTM [Feng et al., 2022], STFCGAT [Li et al., 2023], DGGN [Gu et al., 2023], and BF-GCN [Li, 2024], which use Graph-based models to capture spatialtemporal EEG features.
• Spatial-temporal models: PCRNN [Yang et al., 2018], ACRNN [Tao et al., 2020], MM-ResLSTM [Ma et al., 2019], and BiDCNN [Huang et al., 2021], which leverage RNN or CNN architectures, with attention, multimodal integration, or hemispheric modeling.

4.3 Main Experimental Results

We compare Stella against strong baselines on the DEAP and FACED datasets for valence and arousal classification. As shown in Table 1, Stella consistently outperforms the best baseline, DGGN, improving valence accuracy by 0.46% and arousal accuracy by 0.91% on DEAP, and by 1.85% and 1.96% on FACED. It also achieves notable gains in F1 scores and reduces standard deviations across metrics. These improvements are statistically significant (p < 0.05 on DEAP, p < 0.01 on FACED), demonstrating both superior performance and stability. While some baselines show lower variance, their average accuracy is markedly lower, underscoring Stella's robustness. Furthermore, the smaller performance gap between DEAP and FACED indicates strong generalization across datasets, effectively handling FACED's higher heterogeneity.

4.4 Ablation Study

We conducted an ablation study to evaluate the contributions of the Fourier Graph Operator (FGO) and the Spatial-Temporal Encoder (STE). As shown in Table 3, removing FGO leads to substantial performance degradation-on DEAP, valence and arousal accuracy drop from 97.44%/98.11% to 85.75%/93.56%; on FACED, from

96.98%/95.83% to 94.65%/92.90%. This confirms the importance of modeling inter-regional brain synchronization. Removing STE also reduces performance, though less severely (e.g., DEAP valence down to 75.39%, FACED arousal to 84.53%). Notably, FGO alone performs better than STE alone across all metrics, suggesting that low-level brain coordination is more critical than local temporal encoding. However, the best performance is achieved only when both modules are combined, demonstrating their complementarity and the necessity of joint spatial-temporal modeling for accurate emotion decoding.

4.5 Parameter Sensitivity Analysis

To assess the framework's robustness, we analyze its sensitivity to key hyperparameters, such as the historical time period T (DE feature sequence length) and learning rate lr. Experiments on FACED (T = 11, 13, 15, 17, 19) and DEAP (T = 30, 32, 34, 36, 38) with lr = 1e-5, 3e-5, 5e-5 demonstrate stable performance across settings (Fig. 4), confirming its robustness. Additionally, as illustrated in Fig. 5, we compare the standard and smoothed cross-entropy loss, with the latter showing faster convergence, higher accuracy, and improved training stability.

4.6 Bands Selection Analysis

To assess the effectiveness of ABS, we compare it with the classic 5-band approach by using a single frequency band (e.g., Gα) to construct the phase brain network for FGO, while the remaining bands' DE features are input to STE, with all other settings identical to Stella. As shown in Table 4, STE consistently outperforms the 5-band method across all metrics-for instance, improving DEAP valence and arousal accuracy by 2.14% and 1.93%, and FACED by 3.50% and 1.85% over the δ band. These results demonstrate that accounting for channel differences provides superior performance, supporting the value of adaptive frequency selection in emotion recognition.

5 Conclusion

We present Stella, a framework that combines ABS to differentiate base and harmonic band signals for phase brain networks construction, FGO to address the feature degradation problem in graph convolution, and STE to enhance spatialtemporal information with stable and dense EEG signal representations. Stella effectively builds brain networks, extracts deep spatial-temporal features, and stabilizes the learning process, achieving state-of-the-art emotion recognition performance. It also offers valuable insights for applying graph learning to non-stationary signals like EEG, from feature construction to model design. Beyond emotion recognition, the proposed framework is scalable and potentially applicable to other EEG-based tasks such as cognitive load prediction and neurological disorder detection.

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant 62406036, the National Key Research and Development Program of China under Grant 2024YFC3308500, and the Key Laboratory of Target Cognition and Application Technology under Grant 2023-CXPT-LC-005. It is also sponsored by the SMPZhipu.AI Large Model Cross-Disciplinary Fund under Grant ZPCG20241029322, and supported by the State Key Laboratory of Networking and Switching Technology under Grant NST20250110.