A Correlation Manifold Self-Attention Network for EEG Decoding

1 Introduction

Deep neural networks (DNNs) have significantly progressed across a broad range of applications [Simonyan and Zisserman, 2015; He et al., 2016; Vaswani et al., 2017; Zeng et al., 2024; Tang et al., 2024]. However, most existing methods assume that the data adheres to a vector space structure, whereas many of them emerge from latent spaces governed by non-Euclidean geometries, such as Riemannian geometries. Building on this insight, researchers have made notable strides in generalizing different types of DNNs to manifolds, known as Riemannian neural networks [Huang and Van Gool, 2017; Gulcehre et al., 2018; Chen et al., 2023; Wang et al., 2024b; Wang et al., 2024a; Chen et al., 2024b; Chen et al., 2024d; Chen et al., 2024a; Chen et al., 2024c; Wang et al., 2025; Chen et al., 2025a; Chen et al., 2025b].

Drawing inspiration from the effectiveness of the attention mechanism in capturing correlations between different feature regions [Vaswani et al., 2017; Hu et al., 2018; Dosovitskiy, 2020], the investigation of the Riemannian attention mechanism has gained increasing interest. Notably, the hyperbolic attention network [Gulcehre et al., 2018] represents a pioneering effort in this area, designed based on the Hyperboloid and Klein models. Building on this, [Pan et al., 2022] extended the attention mechanism to Symmetric Positive Definite (SPD) manifolds, implemented under the Log-Euclidean geometry. Subsequently, [Wang et al., 2024a] adapted this approach to Grassmannian manifolds, utilizing an extrinsic mean within the Projection Metric.

∗Corresponding author: Rui Wang

The correlation matrix, which is scale-invariant [David and Gu, 2019], serves as a compact and normalized alternative to the covariance matrix for data representation. A slice of research fields in artificial intelligence, such as Diffusion Tensor Imaging (DTI) [Pennec et al., 2006], Brain-Computer Interfaces (BCI) [Jalili and Knyazeva, 2011], and Gaussian graphical models [Epskamp and Fried, 2018] have particularly benefited from the utilization of correlation matrices in place of covariance matrices. The basic reason is that eliminating the influence of variable scales is particularly effective for the handled problems where the scales are irrelevant [Thanwerdas, 2024]. In particular, non-invasive BCI systems rely heavily on effectively decoding EEG signals to enable direct communication between the brain and external devices. EEG records neural activity with high temporal resolution by measuring electrical potentials on the scalp [Subha et al., 2010], but the resulting signals are often noisy and lack specificity [Hine et al., 2017]. To address these challenges, correlation matrices have emerged as a suitable representation for EEG analysis, as they emphasize statistical dependencies over absolute magnitudes. This is particularly advantageous since strong inter-channel correlations could remain stable despite substantial variations in electrode signal strengths.

Recently, several Riemannian metrics have been proposed for the manifolds of full-rank correlation matrices, including the Off-Log Metric (OLM) and Log-Scaled Metric (LSM) [Thanwerdas, 2024]. The Riemannian operators associated with them, such as geodesics, Fr´echet Means, and exponential & logarithmic maps, are not only permutation-invariant but also computationally efficient. This provides a theoretical possibility for further exploration of attention mechanisms on full-rank correlation matrices.

Designing the attention mechanism for full-rank correlation matrices presents a unique challenge, primarily due to the absence of corresponding transformation layers. The main difficulties stem from the following two aspects. On the one hand, the designed transformation function should preserve the characteristics of full-rank correlation matrices, making the traditional linear layers or their manifold counterparts, e.g., bilinear mapping (BiMap) function [Huang and Van Gool, 2017], for covariance matrices unsuitable. On the other hand, to the best of our knowledge, there is no prior knowledge for constructing neural networks on the manifolds of full-rank correlation matrices, preventing the generation of manifoldvalued queries, keys, and values. Another important problem to be solved is the lack of classification layers defined on the Correlation manifolds. To address these challenges, we introduce two novel transformation layers based on the Lie group homomorphisms, explicitly tailored for the OLM and LSM within the Riemannian geometry of Correlation manifolds. Moreover, building upon [Thanwerdas, 2024], we derive the Weighted Fr´echet Mean (WFM) [Karcher, 1977], a more general Fr´echet Mean, for feature aggregation under OLM and LSM. Additionally, we harness the Riemannian logarithm function to develop two tangent mapping layers under the framework of OLM and LSM to enable the classification of the Correlation manifolds. With these preparations, we propose a Correlation Attention Network (CorAtt) for learning effective spatiotemporal statistical information of EEG signals. In summary, our key contributions are as follows:

• Two novel transformation layers based on Lie group homomorphisms. We design two transformation layers explicitly tailored to preserve the geometric structure of full-rank correlation matrices under the OLM and LSM.
• Two attention models are established on the Correlation manifolds. This article proposes two attention models for the full-rank correlation matrices based on the permutation-invariant and computationally efficient OLM and LSM, respectively.
• Two tangent mapping layers are proposed under the Correlation geometry. Two tangent mapping layers are induced by LSM and OLM to project the full-rank correlation matrices into a flat space for classification.
• Empirical validations in three EEG decoding tasks. Experimental results achieved on three benchmarking EEG datasets validate the effectiveness of our proposed CorAtt and each of the designed components.

2 Preliminary

This section briefly reviews the Lie group and the geometry of full-rank correlation matrices. For more in-depth discussions, please refer to [Do Carmo and Flaherty Francis, 1992; Tu, 2011; David and Gu, 2019; Thanwerdas, 2024].

Definition 2.1 (Lie Groups). A smooth manifold is a Lie group if it is endowed with a group operation ⊙ such that both mappings, m(x, y) ↦→ x ⊙ y and i(x) ↦→ x-1 ⊙ , are smooth. Here, x-1 ⊙ denotes the group inverse.

A Lie group is both a group and a manifold, which motivates the study of smooth maps that preserve these structures. Definition 2.2 (Lie Homomorphisms). Let {M,⊙M} and {N,⊙N} be two Lie groups. A smooth map f(·) : {M, ⊙M} → {N, ⊙N} forms a Lie group homomorphism if it preserves the group structure:

Next, we briefly review the manifolds of full-rank correlation matrices. Any correlation matrix is derived by normalizing the covariance matrix with its variances. Let X be a random variable with an invertible covariance matrix P = (Cov(Xi,Xj))1≤i,j≤n , the corresponding correlation matrix C is defined as:

where Diag(P) is the diagonal matrix of P. The set of all full-rank correlation matrices is denoted as C n++.

Recent studies have discovered that C n++ has a smooth structure and developed several different Riemannian metrics [David and Gu, 2019; Thanwerdas, 2024] over it. This paper focuses on two permutation-invariant and simple metrics, which are the OLM and LSM [Thanwerdas, 2024]. Here, the permutation-invariant property ensures that the analysis is unaffected by arbitrary choices in ordering. In the following, we first introduce four key maps (diffeomorphisms) used to construct these metrics. Wherein, the OLM is associated with the two maps given below:

where mlog(·) and mexp(·) denote the matrix logarithm and exponential, Hol(n) = {X ∈ Rn×n | X = X⊤, Diag(X) = 0}, while Off(X) denotes the off-diagonal part of X. As demonstrated by [Archakov and Hansen, 2021][Sec. 3.3], Do (S) is a diagonal matrix satisfying

This can be solved via the fixed-point iteration.

The following two maps are associated with LSM:

Here, Row0(n) = {X ∈ Rn×n | X = X⊤,X1 = 0} and 1 ∈ Rn is a vector of all ones. The diagonal matrix D⋆ (C) is

where x represents a positive vector and 1x = ( 1 x1 , . . . , 1 xn ). Eq. (6) can be solved via the damped Newton's method

[Thanwerdas, 2024][Sec. 3.5].

Actually, the OLM is induced from Hol(n) via the map Logo (·), while LSM is induced from Row0(n) via the map Log⋆ (·). Additionally, as demonstrated by [Thanwerdas, 2024], both correlation manifolds form Lie groups under OLM and LSM. The geodesic distances and group operations for these two metrics are summarized in Tab. 1.

3 ProposedMethod

In this section, we provide the technical details of the proposed attention mechanism for full-rank correlation matrices. To be specific, we first introduce the main framework of the suggested correlation attention mechanism in Sec. 3.1. This is followed by the specific implementations under OLM and LSM in Secs. 3.2 and 3.3, respectively. Finally, the classification method defined on the Correlation manifolds is detailed in Sec. 3.4.

3.1 Correlation AttentionMechanism

This section showcases how to leverage the correlation matrix geometry to generalize the core operations of transformation mapping, attention computation, and feature aggregation.

Correlation Transformations. In the Euclidean attention mechanism, the linear map Linear(·) : Rn → Rm is commonly employed to generate qi, ki, and vi. This transformation preserves the vector space structure, as shown by the following property:

This indicates that the transformation is a homomorphism over vector spaces. Since correlation matrices lie in a manifold with non-Euclidean geometry, directly applying linear transformations will compromise their inherent geometric properties. However, the Lie group homomorphism (Def. 2.2) generalizes this concept from vector spaces to Lie groups. Therefore, it appears to be a possible and natural choice to define transformation layer on the Correlation manifold using Lie homomorphism hom(·).

Correlation Attention. Let Xi,Qi,Ki, Vi,Ri be the input correlation matrices, queries, keys, values, and output data, respectively. One of the key ideas in attention is to compute the similarity-based score between Qi and Kj for each pair of {Vi, Vj}. In contrast to the commonly used dot product for vectors, the most natural way to compute the similarity between Correlation manifold-valued points is the utilization of geodesic distance. However, a function is needed to map the computed geodesic distance into a valid form, as higher similarity corresponds to smaller distance. Specifically, given that both Qi and Kj reside on the Correlation manifolds, the attention weight is computed by:

where d(Qi,Kj) represents the geodesic distance between the correlation matrices as shown in Tab. 1.

Correlation Aggregation. Corresponding to the weighted average in Euclidean space, the WFM is a principled mathematical tool for feature aggregation in manifolds [Karcher,

Input : A set of correlation matrices {X1...N} Output : A set of correlation matrices {R1...N} for i ← 1 to N do Queries: Qi = hom(Xi) Keys: Ki = hom(Xi) Values: Vi = hom(Xi) end for i ← 1 to N do for j ← 1 to N do Sij = (1 + log(1 + d(Qi, Kj )))-1 end Attention weight: Aij = Softmax(Sij) Aggregation: Ri = WFM({Aij}Nj=1 , {Vj}Nj=1 ) end

1977; Ginestet et al., 2012]. It minimizes the weighted sum of squared geodesic distances. Given the geodesic distance d(·, ·) , a set of points Pi...N ∈M, and the corresponding weights {w1...N} that satisfy the convexity constraint, i.e.,∀i, wi > 0 and ∑i wi = 1, the WFM is defined as:

With the computed attention matrix A and a set of values {Vi...N ∈ C n++}, the i-th aggregated output Ri ∈ C n++ in the built correlation attention model is formulated as:

With these basic components in place, we summarize the forward pass of the proposed attention mechanism on the Correlation manifolds in Alg. 1.

3.2 Correlation Attention Based on OLM

This section details the implementation of Alg. 1 under OLM. While the geodesic distance for similarity computation is summarized in Tab. 1, we focus here on deriving the expressions for the Lie group homomorphism and the WFM. By defining the group operation ⊙ol for OLM, we present the expression for the Lie group homomorphism over the Correlation manifolds as follows:

Theorem 3.1 (OLM Lie Homomorphism). For any C ∈ {C n++,⊙ol}, and M ∈ Rn×m. The transformation mapping homol (·) : {C n++,⊙ol} → {Cm ++, ⊙ol} is defined as:

It can be proved that homol (·) is a Lie group homomorphism.

As discussed in [Thanwerdas, 2024], the Correlation manifold enjoys closed-form expressions of Fr´echet mean under OLM. For attention computation, we present a more general version, the WFM under OLM.

Theorem 3.2 (The WFM under OLM). For C1...N ∈ C n++, w1...N > 0 satisfying ∑i wi = 1, the expression of WFM has a closed form shown below:

It is evident that G corresponds to the OLM-based Fr´echet mean, when wi = 1N for all i in Eq. (12).

3.3 Correlation Attention Based on LSM

Similarly, we derive the expressions for the Lie group homomorphism and the WFM over the Correlation manifolds under LSM.

Theorem 3.3 (LSM Lie Homomorphism). For any C ∈ {C n++,⊙ls}, and M ∈ Rn×m, the transformation mapping homls (·) : {C n++,⊙ls} → {Cm ++,⊙ls} is formulated as:

where ⊙ls denotes the LSM-based group operation and ϕ(X) is expressed as:

Wherein, diag(·) creates a diagonal matrix from a vector. We can prove that homls (·) is a Lie group homomorphism.

Under LSM, the WFM has a closed-form expression. Theorem 3.4 (The WFM under LSM). For C1...N ∈ C n++, w1...N > 0 satisfying ∑i wi = 1, the WFM under LSM can be described as:

When ∀i,wi = 1N in Eq. (15), G corresponds to the LSMbased Fr´echet mean, as indicated by [Thanwerdas, 2024].

3.4 Classification

This section presents the design of the classification layer on the Correlation manifolds. Since the underlying space of the correlation matrices is a non-Euclidean manifold, a manifoldto-Euclidean embedding mapping is required to convert the learned manifold data into the corresponding Euclidean representation. To this end, the tangent mapping layer is designed to project the refined correlation matrices onto the tangent space of the Correlation manifold at the identity matrix using the Riemannian logarithm function. With the following two propositions, the tangent mapping operations under OLM and LSM can be defined.

Proposition 3.5. For any C ∈ C n++, the OLM-based Riemannian logarithm at the identity matrix Logol In (C) can be formulated as:

Proposition 3.6. For any C ∈ C n++, the LSM-based Riemannian logarithm at the identity matrix Logls In (C) can be expressed as:

Now, the tangent mapping operations under OLM and LSM are defined by Eqs. (16) and (17), respectively.

Since each output of the tangent mapping layer is a symmetric matrix with all zeros on the main diagonal, we extract its strictly lower triangular part, vectorize it, and concatenate all vectors w.r.t the i-th input data. The obtained data points are then passed through a fully connected (FC) layer, followed by a Softmax function for the final classification.

4 Experiments

This section tests the proposed CorAtt in two specific forms, called CorAtt-OLM and CorAtt-LSM. To ensure a comprehensive assessment, we apply the two models to three typical BCI tasks, which are the Mental Imagery (MI) decoding on the BCIC-IV-2a dataset [Brunner et al., 2008], Steady-State Visual Evoked Potential (SSVEP) decoding on the MAMEMSSVEP-II dataset [Nikolopoulos, 2016], and Error-Related Negativity (ERN) decoding on the BCI-ERN dataset [Margaux et al., 2012]. For comparison, the following state-ofthe-art (SOTA) deep learning methods are included: ShallowConvNet [Schirrmeister et al., 2017], EEGNet [Lawhern et al., 2018], SCCNet [Wei et al., 2019], MBEEGSE [Altuwaijri et al., 2022], TCNet-Fusion [Musallam et al., 2021], and FBCNet [Mane et al., 2021]. In addition, we incorporate several representative geometric deep learning models, such as SPDNet [Huang and Van Gool, 2017], SPDNetBN [Brooks et al., 2019], MAtt [Pan et al., 2022], and GDLNet [Wang et al., 2024a], to provide a more convincing comparison. All experiments were conducted on an i9-14900 CPU with 64GB RAM and two NVIDIA RTX4080 Super GPUs.

Motor Imagery. The BCIC-IV-2a dataset [Brunner et al., 2008] is a widely recognized public EEG resource, containing signals from 9 subjects performing a four-class motor imagery task. Each subject completed two sessions, with each trial involving four seconds of imagined movement (right hand, left hand, feet, or tongue). Following the protocol in [Pan et al., 2022], the first session of BCIC-IV-2a is used for training, reserving one-eighth of it for validation. Besides, the performance indicator is based on classification accuracy.

SSVEP. The MAMEM-SSVEP-II dataset [Nikolopoulos, 2016] includes EEG data from 11 subjects, each contributing five sessions. In each session, subjects focused on a 5-second visual stimulus oscillating at one of five frequencies: 6.66, 7.50, 8.57, 10.00, or 12.00 Hz. Each subject completed five trials, one for each frequency, yielding 100 trials per session. Each trial lasted between 1 to 5 seconds after the prompt, divided into four one-second segments. Following the protocol in [Pan et al., 2022], we train on the first four sessions, with session 4 used for validation, and tested on the fifth session.

ERN. The BCI-ERN dataset [Margaux et al., 2012] originates from a Kaggle BCI Challenge and contains recordings from 26 subjects who participated in a P300-based spelling task. ERN was measured in response to mistakes made by the BCI speller, leading to a binary, imbalanced classification problem, as correct inputs significantly outnumber erroneous ones. Following the criterion in [Pan et al., 2022; Wang et al., 2024a], we adopt the same dataset partitioning as in the MAMEM-SSVEP-II dataset and employ the Area Under the Curve (AUC) to measure the model performance.

4.1 Proposed Network

Network Architecture. As shown in Fig. 1, the architecture of the proposed CorAtt consists of four main components: a Feature Extraction Module (FEM), a Manifold Modeling Module (MMM), a Correlation Attention Module, and a classification module. We follow [Wei et al., 2019] to make the FEM contain two convolutional layers: one for applying spatial filtering to the multi-channel EEG signals and the other for extracting spatiotemporal features. The MMM is applied to split and transform data points onto the Correlation manifold. We impose segmentation on the output data of FEM, generating s non-overlapping subparts. Then, a correlation matrix is computed for each subpart using Eq. (2).

This is followed by utilizing the correlation attention block, as shown in Alg. 1, to capture the long-range dependencies between different features on the Correlation manifold. whereafter, the classification layer (introduced in Sec. 3.4), incorporated with FC & Softmax, to realize EEG classification.

ImplementationDetails. Considering that orthogonal constraint can serve as an implicit regularization to improve the network's generalization [Lezcano-Casado and MartınezRubio, 2019], we impose orthogonality on M in both homol (·) and homls (·). As orthogonal matrices lie in special orthogonal groups, their optimization requires a Riemannian optimizer, which we implement by generating a parameter A ∈ Rn×n and computing its skew-symmetric matrix as S = A - A⊤. Under this parameterization, the orthogonal matrix O can be obtained by:

This approach optimizes all parameters within Euclidean spaces. For the BCIC-IV-2a dataset, the number of subparts, the size of the transformation matrix in CorAtt, the learning rate, and the batch size are respectively set to 3, 25×25, 5e-4 , and 128, while those for the MAMEM-SSVEP-II dataset are configured as 7, 15×15, 5e-3 , and 64, respectively. In comparison, these values are respectively set to 3, 14×14, 1e-3 , and 32 on the BCI-ERN dataset. For convenience, we summarize the specific network configurations of CorAtt across all the used datasets in Tab. 2.

4.2 Performance Comparison

Tab. 3 lists the experimental results of CorAtts and the selected competitors on the three used EEG datasets. Here, CorAtt-OLM and -LSM represent a unified metric for both the Attention Block and Tangent Mapping layers (OLM and LSM, respectively), whereas CorAtt-MIX adopts different metrics, specifically OLM for attention and LSM for classification. Note that with identical parameter settings for CorAttLSM, -OLM, and -MIX. Overall, standard deep-learning O2

PO8

O2

PO8

O2

PO8

O2

PO8

O2

PO8

0.0 0.25 0.50 0.75 1.00 0.0 0.25 0.50 0.75 1.00 0.0 0.25 0.50 0.75 1.00 0.0 0.25 0.50 0.75 1.00 0.0 0.25 0.50 0.75 1.00

Time Time Time Time Time

Figure 2: Heatmaps of CorAtt-OLM (a) and CorAtt-LSM (b) for the S11 subject across five different frequencies on the MAMEM-SSVEP-II

dataset. The x-axis and y-axis represent time and EEG channels, respectively.

models (EEGNet, ShallowCNet) exhibit relatively lower accuracy than geometry-based approaches (MAtt, GDLNet). CorAtt-MIX achieves the highest performance across all three tasks. This suggests that employing different metrics between attention and classification layers can more effectively adapt the geometric properties of correlation matrices, further demonstrating the flexibility and effectiveness of the proposed approach. Meanwhile, CorAtt-OLM and CorAttLSM consistently outperform MAtt, showing gains of approximately 0.30% and 0.24% on the MI dataset, 2.20% and 2.55% on SSVEP, and 3.10% and 2.95% on ERN. We attribute this performance gap between CorAtt and MAtt to two key factors: (1) CorAtt focuses on correlation matrix modeling, inherently addressing scale-invariant features of signals; (2) CorAtt transformation layer preserves the Lie group structure of Correlation manifolds, providing a more natural extension of attention mechanisms to non-Euclidean geometries.

4.3 Ablations

Ablations of the main components. As shown in Tab. 4, removing any module from the proposed CorAtts significantly drops the classification accuracy, confirming that all the components are essential. The fourth row in Tab. 4 reports the performance of FEM combined with a Euclidean selfattention (ESA) module. The comparison between CorAtts and FEM+ESA highlights the necessity of incorporating Riemannian computations into manifold attention design.

Ablations of the tangent mapping layers. In this subsection, we investigate the impact of the tangent mapping layer on the classification performance of the proposed CorAtt. From Tab. 5, it is evident that when the tangent mapping layer is omitted, the learning ability of CorAtt significantly drops. For example, the accuracy of CorAtt-LSM decreased by 9.94%, 5.25%, and 8.00% on the MI, SSVEP, and ERN

0.2

datasets, respectively. Furthermore, when the attention block is equipped with OLM and the classification layer is realized by LSM, CorAtt consistently achieves the highest accuracy on all the used datasets. This suggests that selecting appropriate Riemannian metrics for different layers is beneficial to enhancing performance, further revealing the flexibility and adaptability of our method.

Ablations for the number of attention blocks. We investigate the impact of using one, two, or three correlation attention blocks under both OLM and LSM. As shown in Tab. 6, two blocks occasionally offer slight gains (e.g., OLM for MI and ERN), but three blocks generally degrade performance (e.g., OLM for MI and LSM for all tasks). Notably, three blocks yield a marginal improvement in the SSVEP task under OLM yet produce an overall decline in accuracy for MI and ERN. This suggests that a single correlation attention block is sufficient for low-dimensional EEG data.

4.4 EEGModel Interpretation

For the MAMEM-SSVEP-II dataset, as shown in Figs. 2 and 3, across five stimulus frequencies, both CorAtt-OLM and CorAtt-LSM primarily exhibit heightened gradient responses around the Oz electrode. These responses appear most prominently between 0.25 and 0.75 seconds, indicating the crucial role of Oz in the visual cortex. Such findings are highly consistent with existing literature on the correlation between SSVEP and Oz in EEG recordings [Herrmann, 2001; Han et al., 2018], likely due to the electrode's central location in the primary visual cortex, resulting in more pronounced induced potentials and an improved signal-to-noise ratio.

As shown in Figs. 4 and 5, for the BCI-ERN dataset, gradient responses for distinguishing 'correct' versus 'error' trials predominantly centre around the FCz. This observation aligns with substantial empirical evidence that the anterior cingulate cortex, a central medial prefrontal cortex region connected to limbic and frontal areas, underlies ERN generation. The consistent gradient responses for both feedback types around the FCz electrode should be noted, particularly in the 0.1 to 0.4-second interval. These findings strongly corroborate the differences in ERP waveforms between correct and incorrect stimuli reported by [Hajcak, 2012]. For the BCIC-IV-2a dataset, please refer to our App. A.

5 Conclusion

This paper proposes the correlation attention mechanism, which generalizes the Euclidean paradigm to the context of Correlation manifolds. Besides, we define the tangent mapping operations for classification over the Correlation manifolds under two Riemannian metrics. Extensive experimental results achieved on three EEG datasets certify the effectiveness and versatility of the proposed CorAtt. In summary, this is the first work to design a deep learning model (attention model in this article) on the Correlation manifolds to the best of our knowledge. The exploration of CorAtt is expected to help the emergence of more geometric deep learning methods for the correlation matrices in the future.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (62306127, 62020106012, 62332008, 62172228), the Natural Science Foundation of Jiangsu Province (BK20231040, BK20221535), the Fundamental Research Funds for the Central Universities (JUSRP124015), the Postgraduate Research & Practice Innovation Program of Jiangsu Province (SJCX25 1319), the Key Project of Wuxi Municipal Health Commission (Z202318), and the National Key R&D Program of China (2023YFF1105102, 2023YFF1105105).

Contribution Statement

Rui Wang and Ziheng Chen contributed equally to the supervision of this work.