ID-RemovalNet: Identity Removal Network for EEG Privacy Protection with Enhancing Decoding Tasks

1 Introduction

Electroencephalography (EEG) as the primary input method for Brain-Computer Interface (BCI), is convenient and costeffective, but it may leak personal privacy and medical information [Gu et al., 2021]. Studies have shown that BCI has potential in authentication systems [Neupane et al., 2019], but EEG signals collected by consumer-grade BCI devices can expose users' private information, such as credit card details, PIN codes, etc [Martinovic et al., 2012; Choi et al., 2018].

∗Corresponding author: kjlz@ahu.edu.cn

Additionally, EEG signals collected by different BCI devices may be related to each other [Kong et al., 2018]. These findings highlight the urgency of protecting user privacy in BCI applications, especially in preventing EEG data from being stolen or manipulated [Zhang and Wu, 2019]. Therefore, developing techniques that can extract EEG task features without personal identity information is crucial for the further application of BCI.

Initially, methods for the overall protection of EEG signals are proposed; however, traditional anonymization cannot fully prevent the leakage of EEG privacy data. As a result, researchers have explored more complex EEG data protection methods, primarily including data encryption and privacypreserving machine learning. Data encryption is a relatively traditional protection approach, including homomorphic encryption (HE) [Robinson and Varghese, 2016], secure multiparty computation (SMC) [Agarwal and others, 2018], and secure processors. However, encryption algorithms may disrupt the task-specific features of EEG and still carry the risk of being cracked. Privacy-preserving machine learning methods, including federated learning [Jia et al., 2024] and sourcefree transfer learning [Xia et al., 2022; Wu et al., 2024; Zhang et al., 2022; Gu et al., 2022], store the original EEG data locally and only transmit model parameters or use API interfaces to execute EEG decoding tasks. However, since the model itself may contain sensitive information, attackers can still reconstruct privacy-related information from the original data through reverse engineering or model theft.

Based on this, there has been a growing emphasis on the safeguarding of identity features within EEG signals. Research on brainwave identity privacy protection is relatively limited, with two main approaches: one is the use of generative networks to synthesize EEG data [Pascual et al., 2021; Singh et al., 2023], where the generated data resembles real data but cannot be used to identify individual identities; the second approach involves adding small perturbations to the brainwave signals, which disrupt the identity-related features in the signals, thus protecting the user's identity information [Meng et al., 2023; Chen et al., 2024]. However, both generative and perturbation methods also impair task classification features, leading to a certain degree of reduction in task classification accuracy. For example, in [Meng et al., 2023], while the identity recognition rate of the EEG signals decreased by 49.51%, the task decoding rate also dropped by

2.43%.

To address these issues, this paper proposes a brainwave privacy protection method called ID-RemovalNet. The core of this method lies in constructing a brainwave signal decomposition framework. Unlike general brainwave protection methods, the proposed method decomposes the brainwave signal into task-related features, identity privacy features, and noise components, with a focus on protecting identity privacy features. On the one hand, the proposed method uses the multi-Domain multi-Level fusion feature extraction (MDML) module to extract high-quality and comprehensive feature information from brainwave signals. On the other hand, the proposed method designs the identity decorrelation separation (IDS) module. Existing generative and subtle perturbation methods not only protect identity privacy but also interfere with task classification features. Our method directly removes the identity privacy features that are decorrelated from the task, ensuring no impact on subsequent task. Finally, to balance both brainwave privacy protection and decoding enhancement, the proposed method introduces a task feature enhancement module. By using the attention-based adversarial feature selection (AAFS) module to stimulate the selection of dominant features, the proposed method enhances decoding accuracy and cross-domain generalization ability. Additionally, a loss-guided identity-level task feature re-fusion (LITFR) module is designed to further compensate for the loss of task features after the removal of identity information, ensuring the accuracy of task decoding. The major contributions of our paper are outlined as follows:

• The proposed ID-RemovalNet designs the Identity Decorrelation Separation (IDS) module to remove the identity information to protect the EEG identity privacy, and the Feature Enhancement (AAFS&LITFR) module, LITFR effectively compensates for the partial loss of the EEG task features during the removal process, and AAFS stimulates the selection of the dominant features of the EEG task to realize the EEG task decoding enhancement and generalization.
• The proposed ID-RemovalNet designs a multi-domain multilevel fusion feature extraction (MDML) module, which extracts rich EEG features by designing the fusion of global and local features in the time-frequency domain to realize the enhancement of EEG task decoding task.
• ID-RemovalNet reduces identity information to 0.43% across four EEG datasets with two different paradigms, significantly improving brainwave task decoding accuracy by 3.28%, and achieves state-of-the-art recognition performance in cross-subject EEG experiments.

2 Our Proposed ID-RemoveNet Method

As shown in Figure 1, the designed method ID-removalNet works sequentially through three key modules: (1) EEG Data Align effectively reduces subject differences between sessions and improves EEG decoding accuracy; (2) Identity Decorrelation Separation (IDS) Module extracts high-quality task-relevant features and identity features using multidomain multi-level Fusion Feature Extraction (MDML) Module, and through the decorrelation items (Dec) and feature subtraction (Sub), separates and removes the EEG identity information, and clean task features are obtained; (3) Feature Enhancement Module utilizes the loss-guided identity-level task feature fusion (LITFR) module to compensate for the loss of task features, and the attention-based adversarial feature selection (AAFS) module to capture cross-session stable task features.

2.1 EEGData Align

Given an EEG dataset D = {xi, yi, ui}Ni=1 , where is the sample of the xi ∈ X ⊂ Rc×t is the i-th EEG trial,c denotes the sample channel, t denotes the sample time, yi ∈ Y = {1, . . . ,K} is the label corresponding to the task, ui ∈ U = {1, . . . , U} is the user label of the i-th trial, and N is the number of EEG trials.

Inspired by [He and Wu, 2020], we found that alignment can effectively reduce subject differences across sessions, where EA methods are efficient, completely unsupervised, and show excellent performance in multiple BCI paradigms. As shown in Figure1 1 (A), where for N EEG trials in a given domain, the EA first calculates the Euclidean arithmetic mean R of all N spatial covariance matrices:

Then, it performs the alignment by:

Thus, EEG data distributions from different domains become more consistent.

2.2 Identity Decorrelation Separation (IDS) Module

EEG signals, as a unique biological feature, contain a wealth of personal information. During cognitive tasks, the extracted task features often contain a large amount of identity information, which not only leads to privacy leakage but also affects the accuracy of task decoding due to the presence of subject variability. As shown in Figure1 1 (B), we propose an identity de-correlation separation module to realize the removal of task-irrelevant identity information, thus obtaining task features containing only a small amount of identity information for EEG decoding and privacy protection. The method focuses on achieving the removal of identity information by decomposing the EEG feature F into three key components, as shown in Equation 3:

where Ftask denotes task-related features, Fid denotes identity characteristics of the subject, and Fnoise denotes noise features in EEG. After MDML extraction of features only a small number of noise features are retained and Eq. 3 is further simplified to:F = Ftask + Fid (4)

Multi-domain Multi-level Fusion Feature Extraction (MDML) To obtain Ftask, Fid, we use a unified backbone network to extract identity features and task features.The designed multi-domain multi-level fusion feature extraction module is shown in Figure1 1: (1) A branch-time scale CNN for converting EEG signals into time-domain discriminative SST representations; (2) A branch-frequency scale learnable continuous wavelet for converting signals into frequencydomain discriminative SST representations; (3) Multi-level fusion of local and global time-frequency features.

Based on [Jia et al., 2021], dynamic adjustment of attention is used to analyze signals at different time scales. "Block T" designs five parallel temporal convolutional layers to extract time features at different scales. After undergoing Squeeze and Excitation (SE) [Hu et al., 2018] operations, the output local discriminative SST F 1 and global discriminative SST F 2 represent different levels of temporal dependencies. "Block F", based on the research from [Liu et al., 2023], adopts five parallel learnable continuous wavelet transformations [Li and al., 2022] to extract features corresponding to specific frequency bands, also outputting local discriminative SST F 1 f and global discriminative SST F 2 f through SE.

Finally, unlike fusion methods that only focus on different domain features, the fusion strategy we use combines both temporal and frequency domain features, integrating both local and global features, effectively capturing multi-level and multi-domain feature information. As shown in equation 5:

Decorrelation Items (Dec) Due to the complexity of the EEG features, there is a coupled correlation between the task features extracted by MDML and the identity features, so we introduce a decorrelation regularisation term to compute the correlation rho between the two, and by minimising this coefficient. As expressed in Eq. 6, linear dimensionality reduction is performed on task features and identity features to obtain vectors htask and hid , where [htask, hid] ⊆ R1×1

where f(htask) and f(hid) denote doing mean processing on the vectors to make them centred. g(htask) and g(htask) denote calculating the vector variance.

And the introduction of Gradient Reversal Layer (GRL) [Ganin and Lempitsky, 2015] for domain adversarial training further enhances feature separation. As shown in Equation 7, the GRL reverses the direction of the gradient, prompting the network to optimise the features during the training process so that the task-related features are as distinct as possible from the identity-related features. Where the output of the task classifier is f(x) and the loss of the task classifier is Ld.

Feature Subtraction (Sub) At this point,the task features obtained are highly decorrelated with the identity features. Finally, by subtracting the features as shown in Eq. 8, the task features containing only minimal identity information can be

2.3 Feature Enhancement and Classifiers

Loss-guided Identity-level Task Feature Re-fusion (LITFR) Since task features and identity features are not perfectly linearly related, simple removal of identity information may result in the loss of task decoding features. As shown in Figure1 1 (C), the loss-guided identity-level task feature re-fusion module calculates the loss of identity features to the task and dynamically adjusts the feature fusion weights to balance the removal of identity information with the integrity of task decoding, as shown in Equation 9:

where fid(x) is the identity-level task feature and Ftask is the fused task characteristic, ω denotes the feature fusion weight. As shown in Equation 10, τ is the temperature coefficient that controls the weight smoothing degree.

Attention-based Adversarial Feature Selection (AAFS) During the training of a neural network, the model usually activates the main features associated with the labels. However, these features may not work when faced with unseen test data, leading to performance degradation. To address this problem, inspired by the self-challenge mechanism [Huang et al., 2020], the design of an attentional confrontation-based feature selection module reinforces the impact of key features, forcing the model to better mine the features that are most useful for decoding in the task, and at the same time enhances the robustness of the model, thus improving its ability to generalise over unseen data. The attention weights are calculated as shown in Equation 11, where αi represents the output weight.

where N represents the feature dimension, Gumbel noise is used to introduce randomness, and the temperature of Gumbel-Softmax controls the smoothness of Softmax. We further optimise the model by calculating the dominant feature FTD and the disadvantageous feature FTI .

where the weights A are learnable parameters that feed FTD and FTI into the the primary classifier C and the secondary classifier C1 for training, respectively.

As shown in Eq. 14-Eq. 18, under the supervision of cross entropy loss, we train the main feature generator and classifier to predict the correct labels, here ℓCE cross entropy loss, y is the task label and s is the user label. The loss terms are as follows:

Ultimately, we formulate the learning of the model as the following optimisation problem,Here λ is the trade-off coefficient:

3 Experiments

3.1 Datasets

The following four public databases are used in this experiment. See Table 1:

(1)Four-class motor imagery dataset (MI4) [Tangermann et al., 2012]: Derived from BCI Competition IV dataset 2a. data were sampled at 250 Hz for 22 EEG channels. Data were extracted within 0-4 seconds after each imagery cue and bandpass filtered at 8-32 Hz.

(2)P300 evoked potentials (P300) [Hoffmann et al., 2008]: Data were recorded on 32 channels at a frequency of 2048 Hz. subsequently downsampled to 128 Hz. the duration of each EEG signal epoch was 0-1 seconds.

(3)Two-class motor imagery dataset (MI2C) [Abelson et al., 1985]: Derived from the BCI Competition IV dataset 2b.The first two sessions without visual feedback were used in this paper, sampled at 250 Hz and containing 3 EEG channels. Data within 0-4 seconds were extracted after each imagery cue and band-pass filtered at 1-40 Hz.

(4)Feedback Error-Related Negativity (ERN) [Leeb et al., 2008]: Sourced from a competition at the 2015 IEEE Neuroengineering Conference, this paper uses a training set from 16 users. Data was recorded on 56 channels at 200 Hz, post downsampling to 128 Hz. epochs of EEG signals from 0-1.3 seconds were extracted.

3.2 Baseline Methods

We use the following three CNN models as baseline feature extractors and retain the last fully-connected layer of each model as a task classifier, while using two fully-connected layers as identifiers:

(1) EEGNet [Lawhern et al., 2018]: A compact CNN architecture designed for EEG classification tasks with deeply separable convolution.

(2) ShallowCNN [Schirrmeister et al., 2017]:A shallow version of DeepCNN with only one convolutional block and with a larger kernel and different pooling.

(3) DeepCNN [Schirrmeister et al., 2017]: contains four convolutional blocks, the first one is designed for EEG inputs and the remaining three are standard convolutional blocks.

3.3 Hyperparameterization

In our experiments, we used a batch size of 128, an initial learning rate of 0.01 and adjusted the learning rate to 0.001 after 50 rounds, model training was performed a total of 100 times and the best model was selected for testing. We evaluate the performance of the task classifier using Balanced Classification Accuracy (BCA) and the performance of the identity classifier using Identity Accuracy (UIA). For each database, a leave-one-session cross-validation is performed, and the mean value of the three experiments is reported.For each loss function of joint multi-task learning, we test the hyperparameters in the range of 0.01-1, and use a stepwise tuning approach, i.e., adjusting each hyperparameter from smallest to largest while keeping the other hyperparameters unchanged until the optimal value is found. After each hyperparameter is determined to be optimal, the value is fixed to ensure that each hyperparameter is optimized. In addition, based on experience, we designed generic parameter settings for each database: 1.00, 0.05, 1.00, 0.05, 0.01, 1.00.

4 Results

4.1 Identity Protection and Task Decoding

To evaluate the performance of the proposed model in EEG identity privacy protection and task decoding, we conducted experiments using four models on four datasets. The specific data are shown in Table 2. The data display the EEG decoding rate and user recognition rate under the original data, as well as the EEG decoding rate and user recognition rate of task features after identity-related features have been removed.

A higher UIA value in raw EEG indicates that the raw data contains a higher degree of identity information, meaning that it is relatively easy to identify the source of a segment of EEG features

Experiments have proved that ID-RemovalNet performs well in identity protection, and the UIA value of Decorrelated EEG decreases by up to 92.12% for MI4C, 73.62% for MI2C, 80.04% for ERN, and 97.01% for P300.

Experiments have demonstrated that ID-RemovalNet enhances the decoding of EEG tasks, and the BCA value of Decorrelated EEG increased up to 7.79% for MI4C, 4.27% for MI2C, 3.63% for ERN and 2.82% for P300;

4.2 Ablation Experiment

We performed an ablation experiment by adding each component of the method step-by-step, and the results of the ablation experiment are listed in Table 3.

The baseline data are the results without any added manipulation, raw EEG decoded by the task as well as user recognition.

We added the alignment treatment and BCA values were substantially increased and UIA values showed a decrease in the motor imagery MI4C and MI2C databases. This suggests that alignment is more favorable for task decoding of EEG data.

We added the IDS component, where "Dec" represents the feature decorrelation component and "Sub" represents the feature subtraction component. The data from the four databases combined shows that after adding "Dec", the taskrelated information and identity-related information in the EEG signals were separated, resulting in a significant decrease in user identification performance, dropping to 5%- 20%. But the BCA impact was minimal. After adding "Sub", Models P300(%)

ERN(%)

EEGNet ShallowCNN DeepCNN MDML

EEGNet ShallowCNN DeepCNN MDML

BCA UIA BCA UIA BCA UIA BCA UIA BCA UIA BCA UIA BCA UIA BCA UIA

Baseline 50.86 41.72 60.46 84.93 62.37 68.83 65.32 93.34 63.92 66.76 64.07 68.23 63.57 64.09 64.41 59.47

+Align

67.89 91.70 65.14 97.21 66.60 96.08 68.69 91.09 64.29 59.90 66.03 77.82 67.51 65.14 67.75 80.28

+Dec.

65.64 10.79 66.75 14.34 67.31 16.44 68.40 13.49 64.83 6.86 67.11 7.65 67.68 6.70 69.10 6.08

+Sub.

65.64 0.18 66.75 0.13 67.30 0.13 68.40 0.06 64.83 1.05 67.11 0.24 67.69 0.25 68.74 0.57

+AAFS. 68.59 0.03 67.45 0.06 67.86 0.02 69.44 0.03 66.32 0.40 67.51 0.32 68.61 0.37 70.64 0.13

All w/LITFR. 69.47 0.55 67.64 0.20 69.42 0.20 70.85 0.06 67.21 0.50 67.57 0.54 68.64 0.38 71.38 0.24

Table 4: In the ablation studies on MI4C, MI2C, ERN, and P300, the task and user recognition accuracies achieved by MDMLNet are shown. "TD" refers to the time-domain branch, "FD" to the frequency-domain branch, "GC" to global fusion, and "LC" to local fusion.

the BCA remained almost unchanged, but the UIA once again dropped significantly to below 2.5%.

We continue to add feature enhancement components, including attention-based adversarial feature selection (AAFS) and Loss-guided Identity-level Task Feature Re-fusion (LITFR). The data from the four databases combined shows that after adding only the AAFS component, the UIA value experiences a slight fluctuation while maintaining a low value, but the BCA value sees a significant increase. Meanwhile, after adding the LITFR component, there is a slight increase across different databases.

4.3 Ablation Experiment of MDML

To verify the effectiveness of our proposed MDML in extracting high-quality EEG task features and identity features, we conducted an ablation experiment by adding each branch (TD&FD) and fusion method (FC&GC) in the method step by step, and the results of the ablation experiment are listed in Table 4.

As shown in Table 4, the multi-scale frequency domain convolution outperforms the multi-scale time domain convolution, except for P300, where globally fused features are better for recognition on motion imagery MI4C, MI2C, but local features perform better on event-related potentials ERN, P300.

The experiments show that MDML with complete structure exhibits optimal results for BCA on all four databases, its optimal BCA is about 0.77% higher on average than the rest of the results, and it exhibits the lowest UIA values on the ERN and P300 databases.

In addition we can get clean features for the task with any network model either in the training phase or in the testing phase. This makes our method highly applicable and can achieve the desired purpose with any database and feature extractor.

5 Discussion

5.1 Cross-subject Comparison Experiments

Cross-subject comparison experiments in EEG are a significant challenge in Brain-Computer Interface (BCI) research. However, due to physiological differences between individuals, there is considerable variability in EEG data across subjects, which introduces many challenges for cross-subject decoding. ID-RemovalNet has been shown to effectively separate identity features, protecting privacy while reducing interference with task-related features. We reproduced the code from existing papers and conducted cross-subject experiments on the BCI 2a, BCI 2b, ERN, and P300 databases. To ensure experimental fairness, the "cross-subject comparison experiment" was conducted by applying the model trained on data from one session to the remaining sessions, using test data features for cross-subject experiments. DRDA [Zhao et al., 2021] and BDAN-SPD [Wei et al., 2024] used the data from the original papers.

Baseline Model with ID-RemovalNet Table 5 shows that we selected the base models EEGNet, ShallowCNN, and DeepCNN for the experiments, where each subject becomes the target domain once, and the experiments show that the crosssubject performance of EEGNet, ShallowCNN, and DeepCNN with ID-RemovalNet outperforms the original data, specifically, for BCI2a , EEGNet improved from 69.8% to 77.0%, ShallowCNN from 71.3% to 76.4%, and DeepCNN

from 65.6% to 71.8%; per-subject improvement was also achieved, and in addition, Table 6 further demonstrates the applicability of ID-RemovalNet on the ERN, P300, and BCI 2b databases, the results of all baseline models using IDRemovalNet still outperform the original data.

ID-RemovalNet and Others In addition, the proposed IDRemovalNet achieves a high accuracy of 82.3%, 76.5%, 79.9% and 70.8% on BCI2a, ERN, P300, and BCI2b, respectively, a result that outperforms all the other experimental results we reproduced, which demonstrates that our model extracts the task features that effectively removes the interference of the identity information and the feature enhancement module superiority, and also demonstrates that MDML extracts discriminative features that further enhance the classification performance.

6 Conclusion

In this paper, we propose a new privacy-preserving framework for EEG signals, ID-RemovalNet, which aims to address the challenge of existing privacy-preserving methods to maintain decoding performance while protecting the privacy of EEG data. The framework efficiently extracts highquality EEG features through a multidomain multilevel fusion feature extraction module and strips identity features using an identity de-correlation separation module, which protects individual privacy and reduces the interference of identity information on task features. In addition, ID-RemovalNet further optimises the task feature selection through feature enhancement to compensate for feature loss and effectively enhance the decoding accuracy. Experiments show that IDRemovalNet removes identity information to 0.43% on four EEG datasets in two different paradigms, while significantly improving the EEG task decoding accuracy by 3.28% and reaching the optimum in cross-subject experiments. Meanwhile, in future research, we will focus on the remaining privacy information in EEG and use multi-feature removal networks to further protect EEG.

Acknowledgments

This work was supported by the Natural Science Foundation for the Higher Education Institutions of Anhui Province (Grant No. 2022AH050091), the Science and Technology Innovation Strategy and Soft Science Research Project of Anhui Province (Grant NO.202306f01050020)