A Grassmannian Manifold Self-Attention Network for Signal Classification

1 Introduction

In recent years, the emerging field of computer vision and pattern recognition (CV&PR) has witnessed remarkable advancements, especially in the realm of sequential data (video clip, Electroencephalograph (EEG) signal, image set, etc) analysis [Nguyen, 2021; Chen et al., 2021; Ingolfsson et al., 2020]. Compared with the existing Euclidean feature learning methods, geometry-aware approaches that leverage manifold structures have gained prominence, mainly because they can capture appropriate statistical representations [Huang and Van Gool, 2017; Huang et al., 2018; Nguyen et al., 2019; Chakraborty et al., 2020; Nguyen, 2021; Chen et al., 2023]. One such fundamental latent space is the Grassmannian manifold [Edelman et al., 1998a], the space of linear subspaces. The Riemannian geometry of linear subspaces provides a solid foundation for the characterization and analysis of sequential data, offering a powerful paradigm for capturing spatiotemporal relationships. In the field of medical imaging, linear subspace finds application in the classification of time-series data for BrainComputer Interfaces (BCI) [Gao et al., 2022] and in the analysis of magnetic resonance imaging [Chakraborty et al., 2020]. For visual classification, its effectiveness has been well-substantiated across a spectrum of practical scenarios, such as dynamic scene classification [Wang et al., 2021; Wei et al., 2022b], facial emotion recognition [Huang et al., 2018; Wang et al., 2022b], face recognition [Wang and Wu, 2020; Wei et al., 2022a], and action recognition [Nguyen and Yang, 2023; Chen et al., 2024b].

∗Corresponding author

While linear subspaces can accommodate the influence of data variations and demonstrate comparatively higher computational efficiency, their intrinsic Riemannian geometry hinders the direct generalization of Euclidean methods to the Grassmannian manifolds. Fortunately, the exploitation of projection operator [Huang et al., 2015; Harandi et al., 2013] to represent each Grassmannian element can bridge the gap mentioned above. In this scenario, the associated measurement on the Grassmannian manifold is known as the Projection Metric (PM) [Edelman et al., 1998b]. Based on the PM, some methods utilize the kernel functions [Hamm and Lee, 2008; Harandi et al., 2013] to achieve discriminative transformation of manifold data points to Euclidean representations, while others directly learn an embedding mapping between two Grassmasnnian manifolds [Huang et al., 2015; Wang et al., 2022b]. Although the latter could yield more discriminative features, there exists an inherent shortcoming that weakens its representational capacity, i.e., feature learning on the nonlinear manifolds utilizing a linear embedding function.

Convolutional neural networks (ConvNets) [He et al., 2016; Simonyan and Zisserman, 2014] are widely acknowledged for their superior performance compared to traditional shallow learning architectures in acquiring potent features. This stems not only from their ability to conduct multi-stage nonlinear computations but also from the effectiveness and scalability of the gradient-descent training procedure. Building upon this insight, certain researchers have undertaken efforts to generalize the ConvNets paradigm to the scenario of Riemannian manifolds [Huang and Van Gool, 2017; Huang et al., 2018; Wang et al., 2022a; Chakraborty et al., 2020; Chen et al., 2024b; Chen et al., 2024a], injecting new dynamism into the realms of data modeling, learning, and classification. GrNet [Huang et al., 2018] is a Riemannian neural network designed with an end-to-end architecture, introducing a deep and nonlinear learning mechanism tailored for linear subspaces. It consists of two fundamental trainable blocks, a Projection block and a Pooling block, for the implementation of data compression, nonlinear activation, and Grassmannian feature pooling.

This innovative architecture guarantees the preservation of Grassmannian properties for the input data at each layer. Subsequently, an Output block is designed to project the learned manifold representations into an Euclidean space for classification. This design has laid the groundwork for further developments in refining and tailoring the existing building blocks [Wang and Wu, 2020] to suit a variety of CV&PR tasks better.

The utilization of deep learning techniques in the field of Grassmannian manifolds is promising, but it remains in its infancy. The existing Grassmannian neural networks (GrasNets) encounter two major limitations: 1) the network inputs, i.e., orthonormal basis matrices, are the global spatiotemporal representation of the raw sequential data, which may be underinformed; 2) there is no explicit statistical correlation established between channel dimensions. Both of these problems will have a potential negative impact on the learning ability of GrasNets. Given the great success of the self-attention mechanism defined in the Euclidean space in characterizing the long-range relationships between features [Dosovitskiy et al., 2020; Huang et al., 2022; Li et al., 2022], we propose a self-attention mechanism on the Grassmannian manifold (GMSA) by referring to some geometric operators, including Riemannian metric, Riemannian mean, and Riemannian optimization. Based on GMSA, a geometric deep learning network, referred to as GDLNet, is proposed to address the aforementioned issues. Specifically, GDLNet first stacks several convolutional layers as an extractor of spatiotemporal representations w.r.t the raw data. Then, a manifold modeling module is attached, which maps the resulting features onto the Grassmannian manifold in two steps. Firstly, each input tensor data is grouped into m sections in the channel dimension, where m stands for the number of divisions. Secondly, we model each section as a Grassmannian element using eigenvalue decomposition. This is followed by a Riemannian self-attention module with the purpose of mining statistical complementarity between different channels.

In contrast to regular self-attention that operates on vectors, our approach proposes query, key, and value for the Grassmannianmanifold. In such a case, to effectivelymine and aggregate the geometrical dependencies between different channels, the Riemannian metric (PM in this paper) instead of the commonly used dot-product is exploited to measure the similarity between query and key. Based on the attention matrix computed, the PM-based weighted Fr´echet mean (wFM) is naturally utilized to obtain the final outputs. The main reasons for using the PM-based wFM are threefold: 1) it is faithful to the Riemannian geometry of Grassmannian manifolds; 2) the Fr´echet mean has shown theoretical and practical advantages in Riemannian data analysis [Chakraborty et al.,

2020]; 3) the PM has exhibited success in many applications [Huang et al., 2018; Wang et al., 2021]. Our main contributions can be summarized into the following three aspects:

• A self-attention mechanism is proposed on the Grassmannian manifold.
• A lightweight geometric deep learning network is designed for learning vibrant spatiotemporal representations of sequential data across Euclidean and Riemannian spaces.
• Extensive empirical validations of our model on three benchmarking datasets certify its effectiveness.

2 Preliminary

This section provides a brief review of the Grassmannian geometry. The Grassmannian manifold G(q, d) is comprised of a set of q-dimensional linear subspaces of the Rd . Each linear subspace can be naturally represented by its orthonormal basis Y of size d × q (YTY = Iq and Iq is an identity matrix of size q × q). Therefore, the matrix representation of Grassmannian is constituted by the equivalence class of orthonormal basis

This definition is known as the Orthonormal Basis (ONB) perspective [Bendokat et al., 2024]. By abuse of notation, we use [Y] or Y interchangeably.

As shown in [Bendokat et al., 2024], each Grassmannian point can also be represented as an idempotent symmetric matrix of rank q by Φ(Y) = YYT, which is known as the projector perspective. This representation indicates that the Grassmannian is a submanifold of the Euclidean space of symmetric matrices. Therefore, an extrinsic distance can be induced by the ambient Euclidean space, which is also known as the Projection Metric (PM) [Hamm and Lee, 2008]:

where ∥·∥F is the Frobenius norm. As demonstrated in [Harandi et al., 2013], the distance computed by the PM deviates from the true geodesic distance on the Grassmannian manifold up to a scale of √2, thus making it a widely used Grassmannian metric.

3 ProposedMethod

As shown in Fig. 1, our GDLNet is composed of three components: a feature extraction module, a manifold modeling module, and a Grassmannian self-attention module. This section will give a detailed introduction to each of them.

3.1 Feature ExtractionModule (FEM)

In this module, a stack of convolutional layers is tailored to different classification tasks to extract task-specific information from raw sequential data. For the RADAR dataset, the FEM is composed of a convolutional layer meant for extracting spatiotemporal information. For the EEG datasets, we follow [Wei et al., 2019] to make the FEM contain two convolutional layers, one of which is used to impose spatial filtering to the multi-channel EEG signals, while the other extracts spatiotemporal features.

3.2 ManifoldModelingModule (MMM)

Let Ei ∈ Rc×l be the i-th feature matrix generated by FEM w.r.t the i-th input data sequence. Here, c represents the number of channels, while l indicates the dimensionality of a channel. Since each point of G(q, d) represents a qdimentional linear subspace of the d-dimentional vector space Rd (see Section 2), the Grassmannian manifold G(q, d) thus becomes a reasonable and efficient tool for parametrizing the q-dimentional real vector subspace embedded in Ei [Turaga et al., 2011; Harandi et al., 2011]. To capture complementary statistical information embodied in different channel features, as illustrated in Fig. 2, we first partition each Ei into m sections along the channel dimension, denoted as ˜Ei1, ˜Ei2, · · · , ˜Eim. Then, a similarity matrix is computed for each ˜Eir, followed by the SVD operation to obtain a qdimentional linear subspace spanned by an orthonormal matrix Yir ∈ Rd×q , s.t. ˜Eir ˜ETir ≃ YirΣirYTir . Wherein, Yir and Σir are two matrices consisting of q leading eigenvalues and the corresponding eigenvectors, respectively. For simplicity, we abbreviate Yir as Yr in the following. Now, the resulting Grassmannian representation of Ei is denoted as Υ = [Y1,Y2, · · · ,Ym].

3.3 GrassmannianManifold Self-Attention Module (GMSA)

An overview of the designed GMSA is illustrated in Fig. 1. It can be seen that the input to this module is a series of orthonormal matrices, i.e., Υ = [Y1,Y2, · · · ,Ym]. To capture the geometric relationship within Υ, we extend the Euclidean multihead attention mechanism [Vaswani et al., 2017; Dosovitskiy et al., 2020] to the Grassmannian manifolds, establishing query, key, and value based on the Grassmannian geometry. To intuitively articulate the forward propagation of each orthonormal matrix involved in GMSA, we design seven auxiliary layers as follows.

GMT layer. In contrast to the traditional linear transformation of vectors in Euclidean space, the generation of query, key, and value in our approach relies on the orthonormal matrices. Inspired by [Huang et al., 2018], we design a Grassmannian transformation (GMT) layer to produce Qr, Kr, and Vr fromYr via the mapping function fgmt:

whereWq,Wk,Wv ∈ Rdt×d (dt < d) are three projection matrices of row full rank. Since the underlying space of these weight matrices is a non-compact Stiefel manifold and the geodesic distance has no upper bound [Huang et al., 2018; Edelman et al., 1998b], it is infeasible to directly optimize them on such manifold. To tackle this issue, we follow [Huang et al., 2018] to impose orthogonality constraint on each weight matrix Wb (b ∈ {q, k, v}), such that the weight space Rdt×d ∗ becomes a compact Stiefel manifold St(dt, d) [Absil et al., 2009].

ORM layer. Inspired by [Huang et al., 2018], we design the orthonormal maintaining (ORM) layer to impose orthonormality, preventing the matrices from degeneracy. Taking Kr as an example, since it is not an orthonormal basis matrix, a typical solution, i.e., QR decomposition, is adopted to make the space spanned by the output feature matrices of the previous GMT layer be a valid Grassmannian manifold. Specifically, the QR decomposition w.r.t Kr is given below:

where Ωr ∈ Rdt×q and Rr ∈ Rq×q are the orthonormal and invertible upper-triangular matrices, respectively. Therefore, Kr can be normalized into an orthonormal basis matrix via the mapping form:

Similarly, we can obtain Q′r and V′r .

PM layer. In Euclidean space, the inner product is commonly used to measure the similarity between the query and key vectors. In contrast, the query, key, and value in our method are orthonormal matrices residing on the Grassmannian manifold. Therefore, in the designed Projection Metric (PM) layer (denoted as fpm), we utilize Eq. (2) to compute the distance between Qr and Kj , given below:

SIM layer. However, the distances computed in the previous PM layer cannot be directly used as attention weights, because an increased similarity between any two samples invariably results in a decrease in their corresponding distance. To this end, we design the similarity measurement (SIM) layer to convert Drj to be a valid form using the transformation function fsim:

It can be seen that fsim is a decreasing function w.r.t dPM(, ). Now, we denote the attention matrix as A = [D′rj ]m×m.

SMX layer. Considering that the values in each row of A do not necessarily satisfy convexity constraint, we use the Softmax (SMX) function, denoted as fsmx, to compress its value range along the row direction in the designed SMX layer:

Now, we use ˆ A = [D′′ rj ]m×m to represent the final attention probability matrix.

WAE layer. As stated in Section 1, this article employs the weighted Fr´echet mean (wFM) to realize the weighted average (WAE) operation involved in GMSA. Before introducing the designed WAE layer, we first give the definition of wFM on the Grassmannian manifolds.

Given a batch of Grassmannian activations {Yr}mr=1 , their weighted Fr´echet mean (P ∗ ) can be defined as:

where dG is a distance on the Grassmannian manifold, and wr is the weight assigned to each Yr, satisfying wr > 0 for all r ∈ {1, 2, · · · ,m} and ∑r wr = 1.

As noted in [Helmke et al., 2007], G(q, d) is isometric to the d×d idempotent symmetric matrices of rank q, denoted as IS(q, d), which is a submanifold of the Euclidean space S d of d × d symmetric matrices [Bendokat et al., 2024]. The famous PM is the distance from the ambient space S d . Inspired by this, the PM is utilized to compute the wFM on IS(q, d):

where ˘Yr =YrYTr and ˘ P = PP T. However, Eq. (12) is a non-convex problem, as ˘ P ∈ IS(q, d) implies a non-convex constraint, i.e., ˘ P 2 = ˘ P. Considering that the essential purpose of designing the WAE layer is to achieve feature fusion on the Grassmannian manifold, we remove the idempotent constraint from Eq. (12). In this way, the above formula is reduced to the following form:

where S d denotes a set of real symmetric matrices. Then, the solution to Eq. (13) can be derived directly through the Euclidean weighted average:

Replacing wr and Yr with D′′ rj and V′j respectively, the WAE layer can be expressed as:

where V = {V′1 , · · · ,V′m} is produced by the ORM layer.

REO layer. Finally, we design the reorthonormalization (REO) layer, denoted as freo, to map each V′′ r ∈ Rdt×dt (symmetric matrix) output by the WAE layer back onto the Grassmannian manifold:

where Eq. (16) represents the SVD operation, and Z1:q is a matrix composed by the eigenvectors corresponding to the first q largest eigenvalues.

Now, the embedding mapping of GMSA can be expressed as: Υ′ = ϕGMSA(Υ), where Υ′ = {Y′1 ,Y′2 , · · · ,Y′m}. The forward pass of the proposed GMSA is summarized in Algorithm 1.

Remark 1. Generally speaking, the computed Riemannian mean should be a valid point on the Grassmannian manifold, that is, it meets the following condition:

where f(Yr) = YrYTr , and f is a bijection from G(q, d) to IS(q, d). Easy computation shows that f -1 is defined by Eqs. (16-17). However, the learned P ∗ ∈ S d in the WAE layer (Eq. (14)) is not idempotent. Nevertheless, we treat the designed REO function (freo) as an approximation to f -1 . Although this approach may not be able to render the optimal Riemannian mean, the composition of the functions of WAE and REO provides a feasible pathway for the realization of the weighted average on the Grassmannian manifolds. Besides, when wr = 1m for all r in Eq. (13), f -1 (P ∗ ) is the extrinsic mean [Srivastava and Klassen, 2004].

Algorithm 1 Grassmannian Manifold Self-Attention Module

Input: A sequence of Grassmannian data {Yr}mr=1

Output: A sequence of Grassmannian data {Y′r }mr=1

• 1: for r ← 1 to m do

• 3: K′r = form(Kr) = form(WkYr)

• 5: end for

• 7: ˆ A := Softmax(A) = [D′′ rj ]m×m
• 8: for r ← 1 to m do

• 10: Y′r = freo(V′′ r ) = Z1:q
• 11: end for

3.4 Backward Propagation

Due to space limitation, this section just provides the computed gradients in the layers of PM, WAE, and REO. For details on other layers, please refer to our supp.. To facilitate expression, we use the symbol k to represent any layer in the designed attention module.

REO layer. As Eq. (16) involves SVD operation, we refer to [Ionescu et al., 2015] to first compute the partial derivative of L(k) w.r.t V′′ r in this layer, given below:

where Λz = ∂L(k′ ) ∂Z , Λs = ∂L(k′ ) ∂S , L(k) = ℓ◦f (K) ◦ ... ◦f (k) (ℓ is the cross-entropy loss) represents the loss function of the k-th layer, k′ denotes a virtual transition layer, regarding V′′ r as its input and outputs Z and S, and G is defined by:

where σr signifies the r-th eigenvalue in S. According to the invariance of the first-order differential, which is also the basic criterion for deducing Eq. (19), the following two partial derivatives can be obtained:

WAE layer. According to Eq. (15), the partial derivatives of L(k) w.r.t D′′ rj and Vj can be computed by:

PM layer. On the basis of Eq. (8), the partial derivatives of L(k) w.r.t Q′r and K′j are shown below:

Based on the aforementioned components, the training and inference of the designed network can be unclogged.

4 Experiments

In this section, we assess the performance of the proposed GDLNet in two distinct classification tasks: drone recognition using the RADAR dataset [Brooks et al., 2019] and EEG signal classification employing the MAMEM-SSVEPII dataset [Pan et al., 2022] and the BCI-ERN dataset [Margaux et al., 2012], respectively. In this article, we execute the publicly accessible source codes of all the involved comparative methods and report their best results across all the used datasets. The number of GMSAs is configured as one in the designed GDLNet, which determines that the following six layers are required for the construction of GMSA: fgmt → form → fpm → fsim → fsmx → fwae. Besides, our model is trained on a PC equipped with an i7-13700H CPU and 32GB of RAM.

4.1 Drone Recognition

The RADAR dataset encompasses 3,000 synthetic radar signals, distributed among three distinct categories. Each radar signal is segmented into windows of length 20, resulting in an orthonormal matrix of size 23 × 10 for the representation of one sequential data. For a fair comparison, we follow [Brooks et al., 2019] to designate 50%, 25%, and 25% of the obtained 3,000 orthonormal matrices to the training set, validation set, and test set, respectively. On this dataset, the dimensionality q of the generated linear subspaces and the size of the projection matrices of the GMT layer are set to 10 and 18 × 23, respectively. Besides, the learning rate of GDLNet is configured as 5e-3 , and the batch size is fixed to 50.

The 10-fold experimental results obtained by different Riemannian neural networks (RiemNets) on this dataset are listed in Table 1. It can be seen that the classification score of GDLNet surpasses those of GrNet, SPDNet, and SPDNetBN by 4.57%, 4.79%, and 0.30% respectively, while the lowest standard deviation (SD) additionally provides the initial evidence for the robustness of the designed model. Following this, we reduced the volume of training data by 95% for a more comprehensive evaluation of the robustness of these four networks. As presented in the last column of Table 1, all the competitors lag behind GDLNet in terms of SD. Moreover, our method still stands out in terms of classification ability compared with GrNet and SPDNet. At the same time, Fig. 3 illustrates that compared with the baseline model (GrNet), the convergence performance of the proposed GDLNet is also quite good. All in all, these experimental results not only attest to the efficacy of GDLNet in capturing useful spatiotemporal statistics of sequential data, but also demonstrate its robustness in the scenario of data scarcity.

4.2 EEG Signal Classification

EEG is a sophisticated neuromonitoring technique that can precisely measure the electric fields generated by cortical neurons, enabling the non-invasive capture of various rhythmic activities occurring within the brain. Several modalities of BCI are based on EEG signals, such as the steady-state visual evoked potential (SSVEP) paradigm and the error-related negativity (ERN) paradigm. The SSVEP paradigm entails external stimuli leading to changes in brain potentials, and the ERN paradigm captures event-related potential (ERP) in the brain's electrical activity when individuals make errors.

However, several special characteristics of EEG data, such as non-linearity, non-stationarity, and a high susceptibility to external interference, make meaningful feature extraction a substantial challenge within this domain. In this part, we apply the designed GDLNet to the task of EEG decoding to further validate its effectiveness, selecting the MAMEMSSVEP-II and BCI-ERN datasets as two typical examples.

SSVEP. This Dataset (MAMEM-SSVEP-II) was collected using the EGI 300 Geodesic EEG System, which features 256 channels and a data sampling rate of 250 Hz. It comprises the data garnered from 11 participants, who partook in five identical yet independent sessions. In each session, the participants were instructed to concentrate on a visual stimulus enduring for 5 seconds, each oscillating at different frequencies: 6.66, 7.50, 8.57, 10.00, and 12.00 Hz. Each subject performed five trials, corresponding to each of the five stimulus frequencies, as prompted. Moreover, each session involved 100 trials, with each trial segment crafted within 1-5 seconds post-prompt, and further split evenly into four one-second segments.

ERN. This dataset (BCI-ERN) utilizes a P300-based BCI, with a total of 26 participants partaking in a spell-check task. The incorrect inputs from the participants are monitored and exploited for measuring the ERP. The primary objective of this challenge is to determine and detect the types of signal disturbances elicited by BCI spelling error feedback to form a judgment regarding its robustness. Since the quantity of correct inputs detected by the BCI speller greatly surpasses that of incorrect inputs, this task is established as an unbalanced binary classification task.

We comply with the established standards in [Mane et al., 2021] for data preprocessing and performance evaluation. To be specific, the initial four sessions of each subject serve as the training set, in which one out of four (i.e., session 4) is used for validation, and the remaining session 5 is allotted for testing. The maximum number of training epochs of the proposed GDLNet is set to 150 and 130 on the SSVEP and ERN datasets, respectively. Besides, the model obtaining the lowest validation loss during training is selected for testing on the 5-th session of the same participant. Here, we take the mean accuracy of ten repetitions per subject as the performance indicator of GDLNet on the SSVEP dataset. Due to data imbalance, we follow the criterion of [Lawhern et al., 2018] to utilize the area under the curve (AUC) to estimate the performance of our model on the ERN dataset. Besides, on the SSVEP dataset, the size of each transformation matrix in the GMT layer, learning rate of GDLNet, and batch size are respectively configured as 19 × 21, 5e-3 , and 64, while those on the ERN dataset are set to 12 × 15, 4e-2 , and 30, respectively. By grouping the feature maps output by FEM in the channel dimension, a number of 3 orthonormal matrices of size 21 × q can be generated in MMM for the characterization of each EEG sample on the SSVEP dataset. Here, the value of q varies for each participant, due to the variability of participants. Similarly, the number and size of the produced orthonormal matrices in MMM are 3 and 15 × q on the ERN dataset, respectively.

It can be seen from Table. 2 that the classification ability (CA) of RiemNets is superior to most of the Euclidean deep learning (EuDL)-based EEG models. This provides a good demonstration of the effectiveness of Riemannian geometry in encoding the nonlinear structure of sequential signals. More importantly, the proposed GDLNet is the best performer on the SSVEP and ERN datasets, further certifying its efficacy in learning useful spatiotemporal representations. It is noteworthy that although MAtt yields similar results to GDLNet on the SSVEP dataset, the training time of GDLNet averages at 1.23 seconds per epoch, which is 0.64 seconds faster than MAtt. This discrepancy arises from the fact that MAtt, a SPD manifold self-attention method, involves a greater number of eigenvalue operations than our GDLNet.

4.3 Ablation Study

In this part, we take experiments to further study the significance of each primary component involved in GDLNet.

1. From Table. 3, it is evident that excluding any module from the proposed GDLNet results in a significant decrease in classification accuracy, implying that there are no surplus components included in our framework. Since MMM acted as the input part of GMSA, it is omitted from Table 3.
2. The fourth line in Table 3 gives the accuracy of FEM+EuSA on the three used datasets, where EuSA represents the Euclidean self-attention module and is computed by: Softmax (QKT/√dk ) V [Dosovitskiy et al., 2020]. The comparison between GDLNet and FEM+EuSA demonstrates the necessity and effectiveness of Riemannian computations in the design of a manifold attention module.

(3) Please refer to our supp. for other ablation studies.

4.4 EEGModel Interpretation

Through comprehensive analysis of GDLNet, fundamental features captured from EEG data can be elucidated. On the SSVEP dataset and as illustrated in Fig. 4, across five stimulus frequencies, the predominant gradient responses are located on the Oz, exhibiting a focused presence between 0.4

and 0.7 seconds. An abundance of gradient responses underscores the pivotal role of Oz in the visual cortex. These findings are resolutely in line with existing research on the correlation between SSVEP and Oz in EEG recordings [Herrmann, 2001; Han et al., 2018]. This is attributable to Oz location at the heart of the primary visual cortex, possessing more robust induced potential amplitudes and a superior signal-to-noise ratio.

On the ERN dataset, the strong gradient responses in the classification of 'correct' and 'error' predominantly center around the FCz, as demonstrated in Fig. 5 and Fig. 6. This is consistent with a large body of empirical evidence that the ERN is generated in the anterior cingulate cortex. This area, being a part of the medial prefrontal cortex, boasts of rich connections to the limbic and frontal brain regions. Additionally, the FCz at the frontal-central midline effectively captures the ERP of ERN. Special attention must be given to the consistent gradient responses exhibited by both feedback types at the FCz location, particularly around the 0.1 and 0.4 seconds timeframe. Notably, these findings strongly conforming to the differences in ERP waveforms observed between correct and incorrect stimuli as indicated by [Hajcak, 2012].

Please refer to our supp. for other experimental results.

5 Conclusion

In this article, we propose a novel geometric deep learning network for more effective signal representation. By Riemannian computations, the proposed Grassmannian selfattention module is qualified to encode and learn useful manifold-valued spatiotemporal patterns of input sequential features, within a lightweight, end-to-end architecture. The experimental results achieved on three benchmarking datasets demonstrate the superiority of GDLNet over some leading comparative methods in both RDL- and EuDL-based signal classification. In addition, the ablation studies confirm the effectiveness of each ingredient in GDLNet. In summary, the proposed network, especially our GMSA block, is a competitive candidate in the modeling, learning, and classification of sequential data across Euclidean and Riemannian spaces. Besides, GMSA provides a feasible and effective path for explicitly mining statistical correlations between Grassmannian features, which brings a new ideology into Grassmannian approaches for signal classification.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (62306127, 62020106012, 62332008, U1836218, 62336004), the Natural Science Foundation of Jiangsu Province (BK20231040, BK20221535), the Fundamental Research Funds for the Central Universities (JUSRP124015), the Key Project of Wuxi Municipal Health Commission (Z202318), the National Social Science Foundation of China (21&ZD166), and the National Key R&D Program of China (2023YFF1105102, 2023YFF1105105).