ListenNet: A Lightweight Spatio-Temporal Enhancement Nested Network for Auditory Attention Detection

1 Introduction

In multi-speaker environments, humans with normal hearing have the ability to focus on a specific speaker while ignoring interference from other sound sources, the phenomenon known as the cocktail party effect [Cherry, 1953]. The mechanism behind it is commonly referred to as selective auditory attention. This inherent ability plays a crucial role in human communication and has attracted growing interest in auditory attention detection (AAD), which aims to localize the attended speaker using brain signals [Dai et al., 2018]. AAD could potentially enhance the design of human-centered intelligent interaction systems, such as hearing aids.

∗ Corresponding Author

Neuroscientific studies have demonstrated a nonlinear relationship between auditory attention and brain activity [Choi et al., 2013; Mesgarani and Chang, 2012], which involves higher cognitive processing in the cerebral cortex. Electroencephalography (EEG) signals are widely used due to their non-invasive nature, ease of acquisition, and high temporal resolution [De Taillez et al., 2020; Fan et al., 2024a]. Spatio-temporal patterns of EEG reveal attentional regulation during selective listening [Tune et al., 2021]. Findings of the inter-subject correlation (ISC) suggest that EEG signals are synchronized across subjects during perception of the same naturalistic visual and narrative speech stimuli [Dmochowski et al., 2014; Shen et al., 2022]. Taking this perspective, EEG signals exhibit temporal correlations, spatial correlations across channels, and spatio-temporal dependencies, which could provide valuable information for discriminating different attention states and advancing robust AAD methods.

Despite significant progress made by existing EEG-based AAD methods, three major challenges still limit their performance and practical application. Firstly, many existing methods have made substantial strides in spatio-temporal modeling, effectively capturing dynamic spatial patterns, leading to improved detection performance. These methods typically treat space and time separately, as shown in Figure 1 (a) and (b). Spatial dependencies are captured independently, and temporal dependencies are subsequently extracted. However, these methods overlook the temporal context under dynamic time conditions, as well as the spatio-temporal dependencies across different channels during auditory stimulus processing. Secondly, the individual differences and the non-stationary characteristics of EEG signals lead to significant performance degradation when applying AAD methods across subjects. [Cai et al., 2024; Fan et al., 2024b] effectively leverage individual-specific features to demonstrate strong performance in the subject-dependent setting, but they lack good generalization ability, which makes it difficult to develop subject-independent robust methods. Lastly, the pursuit of accuracy in current methods [Jiang et al., 2022; Ni et al., 2024] leads to large model sizes and high computational complexity, which are often attributed to complex feature extraction methods and transformer attention mechanisms, making them impractical for low-power devices.

To address these issues, this paper proposes a Lightweight Spatio-Temporal Enhancement Nested Network (ListenNet) with low parameter count and computational complexity. As shown in Figure 1 (c), it captures multi-channel spatiotemporal dependencies and multi-scale dynamic temporal patterns, ensuring high accuracy and strong generalization. Specifically, the proposed ListenNet consists of three components: (1) Spatio-temporal Dependency Encoder (STDE) captures consecutive time steps and multi-channel features, differing from previous studies that first focus on channel features. It expands the input EEG signals within each channel to capture temporal dependencies and extracts spatial features both within and across channels, enhancing spatio-temporal representation capacity. (2) Multi-scale Temporal Enhancement (MSTE) captures temporal dependencies at multiple time scales, adding dynamic temporal context to build robust temporal embeddings. (3) Cross-Nested Attention (CNA) groups spatio-temporal features in parallel, extracts sub-feature context, and recalibrates weights by encoding global information, enhancing deep spatio-temporal correlations. Finally, the effective features are passed to a classifier to predict the subject's attended speaker. The major contributions of this work are summarized as follows:

• The proposed ListenNet overcomes the performance and efficiency limitations of existing methods for AAD by efficiently capturing spatio-temporal dependencies in both subject-dependent and subject-independent settings.
• A novel MSTE module is designed to efficiently extract multi-channel dependencies across multiple scales and time steps to integrate multi-level features, enhancing and complementing robust temporal representations.
• Experimental results show that ListenNet achieves outstanding accuracy while reducing the trainable parameter count by approximately 7 times. Specifically, it surpasses the best baseline by 6.1% on the DTU dataset under the subject-dependent setting and by 8.2% on the KUL dataset under the subject-independent setting, all within a 1-second decision window.

2 RelatedWorks

For spatial dependency modeling, existing methods are divided into physical and dynamic dependencies. [Cai et al., 2021; Jiang et al., 2022] project differential entropy (DE) features in the frequency domain onto 2D topological maps using the known electrode positions to calculate spatial dependency based on physical distance and achieve good performance. Although physical dependency conforms to prior physiological paradigms, the electrode positions relations between channels cannot be directly equated to their functional connections [Liu et al., 2024]. Currently, some researchers autonomously learn spatial dependency relationships during training. [Fan et al., 2024b] extracts DE features as nodes to construct graph neural networks (GNN) and utilize an updated parameter matrix to represent spatial dependency. [Su et al., 2021; Cai et al., 2023; Cai et al., 2024] design channel-wise attention mechanisms that learn to assign distinct weights to capture spatial patterns. [Ni et al., 2024] utilizes a dual-branch approach to extract features from the temporal and frequency domains in parallel. For the frequency branch, it projects DE onto 2D maps and uses their topological patterns. For the temporal branch, the transformer encoder embeds a single cross-channel time step as an input token to autonomously learn features. The current state-of-the-art (SOTA) study [Yan et al., 2024] employs spatial convolution operations across all channels to effectively capture spatial dependencies, resulting in competitive AAD performance.

For temporal dependency modeling, existing methods typically capture temporal dependencies using convolutional neural networks (CNN) andattentionmechanisms. [Monesi et al., 2020] independently uses long short-term memory (LSTM) networks to capture dependencies within EEG signals and achieve decent decoding performance. [Vandecappelle et al., 2021] applies a simple one layer CNN model to directly process EEG data, where the time series are reduced to a single value. [Su et al., 2022] sequentially processes temporal information after spatial attention, multiplying attention maps with EEG signals for adaptive feature refinement. [Wang et al., 2023] utilizes a temporal attention mechanism after GNN that assigns varying weights to a sequence of EEG signals, enabling the capture of the complex temporal dynamics and enhancing the detection of even subtle changes in attentional states over time. Recently, [EskandariNasab et al., 2024] employs gated recurrent units (GRU) and CNN to consider both historical and new temporal information when calculating the current state value, thereby inferring the temporal dependencies between time steps.

The methods mentioned above often focus separately on spatial and temporal features, or adopt a two-step processing strategy in which spatial dependencies are captured, followed by the modeling of temporal dependencies. However, these approaches tend to overlook the rich temporal contextual information under dynamic time conditions, as well as the spatio-temporal distribution characteristics of different brain regions during the reception, processing, and response to auditory stimuli. As a result, the failure to capture critical spatio-temporal dependencies significantly limits model performance.

3 The Proposed ListenNet Method

The proposed ListenNet is designed to comprehensively integrate spatio-temporal dependencies in EEG signals, addressing the limitations of existing methods by modeling dependencies across both multiple channels and time scales. Figure 2 illustrates the overall structure of ListenNet. The method will be specified in the following subsections.

Given the EEG data split by a moving window, a series of decision windows is obtained, each containing a short time segment of EEG signals. Consider the original EEG data of a decision window represented by 𝑋 = [𝑥1, ..., 𝑥𝑖, ..., 𝑥𝑇] ∈ R 𝐶×𝑇 , where 𝐶 is the number of EEG channels and 𝑇 is the length of the decision window. Here, 𝑥𝑖 ∈ R 𝐶×1 is the EEG data at the 𝑖-th window of 𝑋. We aim to learn a representation 𝐹(·), which maps 𝑥 to the corresponding label 𝑦 = 𝐹(𝑥). Here, 𝑦 denotes the locus (i.e., left or right) of auditory attention. Before inputting the EEG data into ListenNet, a Euclidean alignment (EA) method [Miao et al., 2022] is employed, which standardizes the EEG data by calculating the average covariance matrix to extract shared features from the data across different brain states. ˜𝑋 ∈ R 𝐶×𝑇 is obtained by normalizing and aligning 𝑋.

3.1 Spatio-temporal Dependency Encoder (STDE)

EEG signals are derived from different brain regions and exhibit dynamic changes in connectivity patterns between brain regions over time. Previous studies neglect the spatiotemporal characteristics of EEG signals. Meanwhile, as networks become increasingly complex [Zhang et al., 2023; Chen et al., 2023; Niu et al., 2024], the limited size of EEG data makes these networks prone to overfitting. CNN-based networks have demonstrated sufficient feature extraction capabilities in brain-computer interface (BCI) tasks [Lawhern et al., 2018; Miao et al., 2023]. Considering these characteristics, we design a spatio-temporal dependency encoder to extract robust dynamic patterns using depthwise separable convolutions, which consists of the temporal feature component (STDE-T) and the spatial feature component (STDE-S), as shown in Figure 2 (a).

Firstly, STDE-T extracts dynamic features from EEG signals through temporal convolution layers, capturing temporal dependencies and constructing the temporal patterns 𝐸𝑡 . This can be expressed as:

where 𝐸𝑡 ∈ R 𝑑depth×𝐶×𝑇 ′ , 𝐶𝑜𝑛𝑣(·) represents convolutional filters with a 1 × 1 kernel size to perform spatio-temporal reshaping on the input signals. 𝐷𝑒𝑝𝑡ℎ𝑤𝑖𝑠𝑒𝐶𝑜𝑛𝑣(·) performs convolution independently on each input channel along the time dimension with a kernel size 1 × 𝑘0 and a group size 𝑑depth, followed by the 𝐺𝐸𝐿𝑈(·) activation function.

Subsequently, STDE-S encodes the spatial distribution information across all channels through spatial convolution layers, capturing the spatial distribution features 𝐸𝑠 from EEG signals, which facilitate a comprehensive understanding of the brain's activity patterns in response to various auditory stimuli. This can be expressed as:

where 𝐸𝑠 ∈ R 𝑑depth×1×𝑇 ′ , 𝐶𝑜𝑛𝑣(·) represents convolutional filters with a 1 × 1 kernel size for initial channel mapping and achieving channel-wise feature fusion. 𝐷𝑒𝑝𝑡ℎ𝑤𝑖𝑠𝑒𝐶𝑜𝑛𝑣(·) performs convolution to capture inter-channel dependencies with a 𝐶 × 1 and a group size 𝑑depth, with the 𝐺𝐸𝐿𝑈(·) activation function. We integrate the spatial distribution features with the temporal patterns to form a comprehensive spatiotemporal embedding 𝐸𝑠.

3.2 Multi-scale Temporal Enhancement (MSTE)

The auditory system is sensitive to the temporal patterns [Puffay et al., 2022]. Inspired by the concept of multi-scale modeling [Wu et al., 2020; Fan et al., 2024c], we propose a novel MSTE module. As shown in Figure 2 (b), the module captures dynamic brain activity across multiple time scales, offering a comprehensive representation of temporal patterns.

MSTE integrates dilated convolutions with the inception strategy to capture temporal features across multiple scales, thereby enabling a more comprehensive representation of multi-level temporal dependencies and enhancing the modeling of complex temporal patterns. The dilated convolution filters use four different kernel sizes to capture patterns at different time scales, with same dilation factor progressively expanding the effective receptive field. This enables the module to more efficiently capture both fine-grained and long-term temporal dependencies without increasing the number of parameters. Formally, the Inception strategy is combined with dilated convolutions to capture multi-scale temporal features. Given the input from the temporal convolution layers, the module applies four convolutional filters, each with a fixed dilation factor, to extract multi-scale temporal features. The outputs are truncated to match the size of the largest kernel, concatenated along the channel dimension, and normalized using batch normalization, ultimately generating the multiscale feature map. The above process can be formulated as:

where 𝑈 ∈ R 𝑑depth×𝐶×𝑇 𝑘 min , and 𝑇 𝑘 min represents the minimum time dimension among the outputs. [·] represents concatenation operation, and DilatedConv1×𝑘 (·) is implemented as a set of dilated convolutions with 𝑘 ∈ {𝑘1, 𝑘2, 𝑘3, 𝑘4}. For each kernel size 𝑘, a convolution is applied along the temporal dimension with a fixed dilation factor 𝑑.

The skip connection is implemented using a depthwise convolution with a kernel size of 𝐶 × 1 and a group size 𝑑depth. These transform spatial information while preserving channel structure and standardize the sequence length for consistent transmission to the output module. The features are resized via bilinear interpolation to match the dimensions required by the subsequent layer, resulting in 𝑆 ∈ R 𝑑depth×1×𝑇 ′ , which is then added to 𝐸𝑠, producing a robust representation of spatiotemporal dynamics 𝐸 ′ 𝑠 ∈ R 𝑑depth×1×𝑇 ′ .

3.3 Cross-Nested Attention (CNA)

The multi-head attention mechanism in transformer models achieves significant results but incurs high computational cost. Inspired by the parallel strategy for cross-dimensional spatial information aggregation [Wang et al., 2020; Ouyang et al., 2023], we propose a novel cross-nested attention module that efficiently integrates hierarchical spatio-temporal features and reduces computational cost.

CNA employs dual-branch decomposition and interactive enhancement, extracting deep spatio-temporal features through attention weighting. Prior to processing, the input temporal feature 𝐸𝑡 is depth-aligned with 𝐸 ′ 𝑠 to produce

𝐸 ′ 𝑡 ∈ R 𝑑depth×𝑑depth×𝑇 ′ . As shown in Figure 2(c), both 𝐸 ′ 𝑡 and 𝐸 ′ 𝑠 are divided into 𝐺 groups along the depth dimension, where 𝐺 = � 𝑑depth/2 � , and ⌊·⌋ denotes the floor operation. The dimension-adjusted features are denoted as 𝐹𝑡 and 𝐹𝑠, respectively. Then, a dual-branch spatio-temporal module is applied to decompose and capture global information in both directions, producing two enhanced features, 𝐹1 and 𝐹2, as formulated below:

where, 𝐺𝑁(·) denotes the group normalization operation, 𝐴𝐴𝑃𝑆 (·) denotes the spatial adaptive average pooling operation, 𝐴𝐴𝑃𝑇 (·) denotes the temporal adaptive average pooling operation, 𝜎(·) denotes the sigmoid activation function, and ⊙ denotes the element-wise multiplication operation.

To capture long-range dependencies and global context, global average pooling and softmax are applied to each input branch to produce attention vectors. These are reshaped and used to compute cross-attention maps with features from the opposite branch via matrix multiplication. The resulting maps are concatenated and passed through a shared 1×1 convolution for feature fusion and dimensionality reduction, yielding the final attention weights 𝑊 ∈ R (𝐵×𝐺)×1×1×𝑇 ′ , with 𝐵 denoting the batch size. Finally, the output deep spatio-temporal features 𝐸 ∈ R 𝑑depth×1×𝑇 ′ are obtained by applying element-wise multiplication between 𝐹𝑠 and the sigmoid-activated 𝑊.

3.4 Classifier

The classifier is designed to provide the final auditory attention results. Global average pooling is applied to reduce the dimensions of the features output by the CNA module. Then, the normalized feature maps are flattened into a 1D vector and fed into a fully connected layer to produce the final result. In the training stage, we apply the binary cross-entropy function to update the parameters.

where 𝑦𝑖 means the ground-truth label of 𝑖-th decision window, 𝑁 means the number of samples, and 𝑄𝑖 is the corresponding possibility of predicted direction label with softmax function processing.

4 Experiment

4.1 Datasets

We evaluate ListenNet on three publicly available datasets: KUL [Das et al., 2019; Das et al., 2016], DTU [Fuglsang et al., 2018; Fuglsang et al., 2017], and AVED [ZHANG et al., 2024]. We summarize the details of the above datasets in Table 1.

1) KUL: This dataset consists of 16 normal-hearing subjects, with 64-channel EEG data recorded. Each subject was instructed to attend to one of two competing voices from either the 90 ◦ left or right. Each subject completed 8 trials, each lasting 6 minutes.

2) DTU: This dataset consists of 18 normal-hearing subjects, with 64-channel EEG data recorded. Each subject was instructed to perform a target speaker tracking task in an environment with reverberation and dynamic background noise interference, attending to one of two competing voices from speakers positioned at a 60 ◦ relative to the subject. Each subject completed 60 trials, each lasting 50 seconds.

3) AVED: This dataset consists of 20 normal-hearing subjects, with 32-channel EEG data recorded. Subjects were evenly divided into two experimental conditions: audio-only and audio-visual, with 10 subjects in each condition. Each subject was instructed to attend to one of two competing voices from either the 90 ◦ left or right. In the audio-visual condition, subjects not only listened to the stories but also watched the video of the narrator they were instructed to focus on. Each subject completed 16 trials, each lasting 152 seconds.

4.2 Data Processing

To eliminate artifact noise and obtain cleaner EEG signals, specific preprocessing steps are applied to the three datasets to ensure consistency and comparability across experiments. For the KUL dataset, EEG signals are band-pass filtered (0.150 Hz) to remove irrelevant frequencies and downsampled to 128 Hz. For the DTU dataset, 50 Hz line noise and power line interference are filtered out, followed by downsampling to 128 Hz and high-pass filtering at 0.1 Hz. Eye artifacts are removed using joint decorrelation, and data are re-referenced to the average EEG channel response. For the AVED dataset, 50 Hz power line interference is removed, and the signals are band-pass filtered (0.1-50 Hz) and downsampled to 128 Hz. Subsequently, ocular and muscle artifacts are eliminated using independent component analysis (ICA). Finally, all EEG channels were re-referenced.

To evaluate ListenNet, we compare it with other SOTA AAD methods under both subject-dependent and the more challenging subject-independent settings. Specifically, four open-source models are selected as baselines: SSF-CNN [Cai et al., 2021], MBSSFCC [Jiang et al., 2022], DBPNet [Ni et al., 2024], and DARNet [Yan et al., 2024].

4.3 Implementation Details

We evaluate the performance of ListenNet on KUL, DTU, and AVED datasets under both subject-dependent and subjectindependent settings. For the subject-dependent condition, each subject's data is split into training, validation, and test sets in an 8:1:1 ratio. The batch size is set to 32, the maximum number of epochs to 100, and an early stopping strategy is employed. Moreover, the model is trained using an Adam optimizer with a learning rate of 5e-4 and weight decay of

3e-4. For the subject-independent condition, the leave-onesubject-out (LOSO) cross-validation strategy is used. Namely, one subject's EEG data constituted the testing data, and the remaining subjects' EEG data constituted the training data. Here, the batch size is set to 128, with a maximum of 100 epochs. An Adam optimizer is also used with a learning rate of 1e-3 and a weight decay of 3e-4.

The following describes the implementation details, including the training settings and network configuration. The hyperparameters of ListenNet are consistently fixed across the three datasets to ensure a fair comparison of its generalizability. For STDE, the kernel size 𝑘0 is set to 8, and the group size 𝑑depth is set to 16. For MSTE, the kernel sizes used in the 2D dilated convolutional filter are 𝑘 ∈ {1, 2, 3, 5}, and the dilation factor 𝑑 is set to 1. Consequently, the number of groups 𝐺 in CNA is configured as 8. All experiments are conducted using PyTorch on an RTX 4090 GPU.

5 Results

5.1 Comparison with Prior Art

In this work, we maintain the same subject-dependent setup as most existing models and evaluate our model in a more challenging subject-independent setup to better align with realworld applications, as detailed in Table 2.

Performance of Subject-Dependent

The comparison of subject dependence AAD performance between the ListenNet model and other baselines on the KUL, DTU and AVED datasets is presented in Tables 2. Our method significantly outperforms the current SOTA on both the KUL and DTU datasets. Specifically, on the KUL dataset, ListenNet demonstrates higher accuracies by 3.3%, 2.1%, and 1.8% for the 0.1-second, 1-second, and 2-second decision windows, respectively. Similarly, on the DTU dataset, it achieves improvements of 4.8%, 6.1%, and 5.4% in the same decision windows. On the AVED dataset, ListenNet performs slightly worse than DARNet in the 1-second and 2-second decision windows, but still achieves optimal performance in the very short 0.1-second window. One possible explanation is that DARNet's transformer attention outperforming by capturing long-range cross-modal dependencies in the AVED dataset.

We observe that ListenNet's decoding accuracy increases with the enlargement of decision windows, due to longer decision windows providing more information. The proposed ListenNet exhibits satisfactory performance at a temporal resolution of 1-second, which is approximately close to the time lag necessary for humans to switch attention. Moreover, our advantages are further enhanced under the highly challenging short 0.1-second decision window length, thereby contributing to the subsequent realization of real-time decoding of auditory attention.

Performance of Subject-Independent

Apart from excellent results in the subject-dependent setup, the proposed ListenNet also demonstrates comprehensive leading classification performance in the more challenging subject-independent setup across three datasets for the commonly used two detection window sizes. ListenNet benefits from better results by more comprehensively and effectively integrating dynamic temporal patterns and spatio-temporal dependencies, enabling the model to flexibly utilize subjectinvariant representations. The results further confirm this capability. Especially on the KUL dataset, ListenNet achieves notable performance, demonstrating accuracy increases of 8.2% and 7.7% over the current SOTA model for the 1-second and 2-second decision windows, respectively. Furthermore, ListenNet outperforms baselines for DTU and AVED as well.

Compared to the widely-used KUL dataset, the DTU and AVED datasets pose a more challenging AAD task. Specifically, DTU presents speech at a narrower angle, and its recording environment includes reverberation and background noise, whereas AVED introduces complex multi-modal stimulus materials. The results show that ListenNet outperforms the baseline methods across diverse datasets, with lower variability in its results, further highlighting the stability and reliability of our approach across different decision windows. It learns the common pattern of feature distribution from subjects, thereby more effectively simulating real-world scenarios. These results highlight the robustness and generalization capabilities of the proposed model, emphasizing its potential superiority in EEG-based applications.

5.2 Ablation Analysis

Ablation studies are conducted on three datasets using a 1second window setting, which most closely aligns with human attention switching [Jiang et al., 2022; Fan et al., 2025]. ListenNet constructs robust spatio-temporal representations.

This enables the model to capture the full spatiotemporal information in EEG signals, thereby improving the interpretation of brain activity. Table 3 presents a comparison between the full ListenNet model and these four variants across the three datasets.

STDE-T and STDE-S are each removed to disrupt the integrity of STDE, thereby assessing the critical role of these components in the model's performance. Removing the STDE-T module for spatio-temporal dependency encoding has the most significant impact on the model's performance. The effectiveness of STDE-T can be attributed to the fact that EEG signals, as high temporal-resolution time series, exhibit strong temporal dependencies. Prioritizing the modeling of temporal continuity allows for the extraction of more effective and accurate spatio-temporal feature embeddings. Removing the STDE-S module results in accuracy decline, as full-channel spatial convolution captures inter-channel dependencies and establishes a robust spatio-temporal feature framework.

The removal of the MSTE module results in the loss of multi-scale temporal information and disrupts potential dependencies between temporal segments, thereby increasing the risk of overlooking critical temporal features essential for accurate recognition. Similarly, eliminating the CNA module diminishes the model's ability to dynamically assign feature weights and enhance spatio-temporal representations, weakening the extraction and integration of multi-level spatiotemporal features and ultimately reducing overall accuracy.

5.3 Computational Cost

Table 4 compares the parameter counts and MACs of ListenNet with those of MBSSFCC, DBPNet, and DARNet on the KUL dataset. With only 0.01 M trainable parameters, ListenNet achieves remarkable parameter efficiency, requiring approximately 8390 times fewer parameters than MBSSFCC, 90 times fewer parameters than DBPNet, 7 times fewer parameters than DARNet. Additionally, ListenNet's computational demand is also markedly reduced, with its MACs only 12.16 M, approximately 86% lower than MBSSFCC, 87% lower than DBPNet and 26% lower than DARNet. These substantial reductions in both parameter count and computational complexity highlight ListenNet's enhanced efficiency, making it especially suitable for deployment on devices with limited computational resources.

5.4 Visualization Analysis

To assess the effect of extracting subject-invariant features, we randomly select 30 samples from each subject in the KUL dataset and visualize them using t-SNE [Van der Maaten and Hinton, 2008]. The resulting plots are shown in Figure 3. Different colors represent subjects, with circles and squares indicating attention to the left or right speaker, respectively. In Figure 3 (a), the raw features are scattered with significant overlap between subjects and labels, lacking clear structure and separability. In Figure 3 (b), preprocessing improves feature quality to some extent, but notable overlap and insufficient separability still remain. In Figure 3 (c), features extracted using STDE form clearer attention-related subgroups. By capturing spatio-temporal cross dependencies, the STDE module learns dynamic patterns and enhances feature separability, though some class boundaries remain indistinct. In Figure 3 (d), features extracted by ListenNet exhibit more distinct clustering for attention labels across subjects, and the distributions become more organized. This demonstrates that ListenNet learns subject-invariant features while maintaining clear boundaries between attention categories. These results further confirm the effectiveness of our method in enhancing the model's ability to decode attention states accurately while improving generalization across different subjects.

6 Conclusion

This paper introduces ListenNet, a lightweight, highly accurate, and generalizable network for AAD. By combining spatio-temporal convolution operations across time steps and all channels, it effectively utilizes spatial information embedded in temporal EEG signals. Additionally, it captures temporal patterns at multiple scales, previously overlooked, by usingmulti-scale dilated convolutions. It integrates hierarchical spatio-temporal features through cross-nested attention mechanisms. Subject-dependent and subject-independent experiments are conducted on three AAD datasets. Experimental results show that our ListenNet exhibits competitive accuracy, especially in the very short 0.1-second decision window and across subjects. Furthermore, the compact size of our model and the reduced computational costs open new possibilities for deployment on low-power devices. For future work, we intend to extend ListenNet to streaming architectures, integrating incremental learning for real-time adaptation to AAD scenarios.

Acknowledgements

This work is supported by the STI 2030-Major Projects (No.2021ZD0201500), the National Natural Science Foundation of China (NSFC) (No.62201002, 6247077204), Excellent Youth Foundation of Anhui Scientific Committee (No. 2408085Y034), Distinguished Youth Foundation of Anhui Scientific Committee (No.2208085J05), Special Fund for Key Program of Science and Technology of Anhui Province (No.202203a07020008), Cloud Ginger XR-1.