Self-Promoted Clustering-based Contrastive Learning for Brain Networks Pretraining

1 Introduction

The rapid development of neuroimaging techniques, such as magnetic resonance imaging (MRI), has considerably facilitated the acquisition of the brain's structural (i.e., white matter tracts) and functional (i.e., blood oxygen level-dependent signal) connections, which paves the way for more comprehensive studies into brain cognition and behavior [Hermessi et al., 2021; Salama et al., 2018; Zhang et al., 2020]. One of the essential tools to conduct these investigations is the brain network, which represents the underlying topological organization of the brain as a graph at the macroscale [Ahmedt-Aristizabal et al., 2021; Cui et al., 2022; Hu et al., 2021; Ma et al., 2022]. In the graph, nodes correspond to anatomical brain regions, while edges illustrate the structural or functional connections between them. Graph theories and methods can then be applied to these brain networks, which allows researchers to mathematically quantify and explore the organization and interaction of the brain regions [Bullmore and Sporns, 2009; Cao et al., 2023; Gordon et al., 2016]. Consequently, brain network analysis has become an emerging research field with numerous downstream applications, such as brain disorder diagnosis and disease progression forecasting [Huang et al., 2020; Zhu et al., 2022].

∗Corresponding author

Over the past few years, Graph Neural Networks (GNN) have been proven to be a promising tool for brain network analysis, which integrates graph theories with deep learning techniques, creating a paradigm that is capable of handling the complex relationships inherent in graph data [Li et al., 2022; Peng et al., 2022b; Zhou et al., 2020]. The versatility and scalability allow GNNs to handle the complexity of brain networks, making them a powerful tool for exploring the brain's structure and function, including early diagnosis and prognosis of brain disorders, understanding the progression of various diseases, and personalizing treatments based on individual brain network structures [Ma et al., 2020; Peng et al., 2022a; Tanveer et al., 2020; Young et al., 2018].

Despite the tremendous potential, several challenges persist in applying GNNs to brain network analysis [Zhang et al., 2024; Lin et al., 2024]. One key challenge is that the GNNs usually demand large amounts of labeled data to train effectively [Cao et al., 2023; Cui et al., 2022]. However, as MRI scans are often costly, brain network data is naturally scarce, especially in the context of disease-related studies. To address this challenge, Graph Contrastive Learning (GCL) has surfaced as a promising solution [Tong et al., 2021;

You et al., 2020; Zhao et al., 2023]. GCL is a representation learning technique that learns to differentiate between similar and dissimilar data instances through contrastive pairs [Lee et al., 2022; Lin et al., 2022; Zhang et al., 2020]. The GCL model is trained to maximize the agreement, or similarity, between positive pairs while simultaneously minimizing the agreement between negative pairs. Usually, these contrastive pairs are created either from data augmentation or extracted from the training set. This pairwise organization of training data provides a valuable mechanism to alleviate the issue of data scarcity. By maximizing the utility of each data point, GCL enhances the training signal and allows for more effective learning from limited datasets.

Although GCL has achieved significant success in many fields, it still has a notable limitation when dealing with brain network data. One essential step in GCL is creating contrastive pairs, and most of the GCL methods focus on designing graph augmentation schemes [Lin et al., 2022; Liu et al., 2022a], which involve perturbing the graph structures, such as randomly adding or deleting nodes or edges of the original graph. As shown in Figure 1, this kind of graph augmentation scheme may not be appropriate for brain networks because the brain networks are derived fromMRI images, which neurologically reflect the physical brain wiring (structural brain network) or functional brain region correspondence (functional brain network) [Bullmore and Sporns, 2009]. As such, any arbitrary change or perturbation of the brain network risks creating misleading graph representations. Learning from distorted contrastive pairs cannot help GCL models explore the brain mechanisms and could negatively impact the models' performance.

Additionally, it is important to note that brain network data is typically accompanied by experts' annotations, which is due to the fact that MRI is both a technique and cost-intensive procedure requiring the oversight of medical professionals [Chu et al., 2018]. Furthermore, it is obvious that acquiring brain network data from healthy subjects is relatively easier than disorder ones in disease-related studies, as healthy individuals are the majority of the population. Thus, utilizing the extra brain network data from healthy individuals to pre-train the model is a promising way to partially alleviate the scarce issue.

In response to the aforementioned challenges and unique characteristics of brain network data, a Self-Promoted Clustering-based Contrastive Learning (SPCCL) method is proposed, aiming to help the model effectively learn instinctive graph representations from the brain network data. Specifically, extra healthy subjects are added to the training set, creating a data pool. The initial contrastive pairs are extracted from this pool. The pairs from the same group (marked by the data label) are positive pairs, whereas the pairs from different groups are negative pairs. After initializing the contrastive pairs, a Siamese Graph Convolutional Network (GCN) framework is utilized to extract graph-level representations from them [Liu et al., 2022b]. Then, a selfsupervised readout mechanism is proposed to refine the graph representations, ensuring they accurately reflect the underlying structural and functional characteristics of the brain networks. After that, a supervised clustering-based contrastive learning loss is adapted and integrated to ensure the clustering consistency with the data labels. Then, a self-promoted contrastive pairs creation scheme is proposed, which utilizes the cluster centroids as the prototype of each group. The samples that are far from their corresponding prototype are promoted to create contrastive pairs for the next round of training. So, in the final pre-trained contrastive representation space, the samples from the same group are tied closely together, whereas the distance between the prototypes is far away from each other. After the pre-training, the additional healthy subjects' data are removed from the training set, and the classification is performed inside this pre-trained contrastive representation space.

To test the effectiveness of the proposed SPCCL method, we conduct comprehensive experiments on an open-access schizophrenic dataset [Vohryzek et al., 2020], which contains two modalities named structural and functional connectomes from 27 schizophrenic patients and 27 healthy adults. An additional dataset with 70 healthy adults is included in the pre-training phrase [Griffa et al., 2019]. The details of the dataset can be found in the Experiments section.

In summary, the contributions of this paper are:

1. A Self-Promoted Clustering-based Contrastive Learning (SPCCL) method is proposed for brain network pre-training, which addresses the challenges and unique characteristics of brain network data.
2. A self-supervised readout mechanism is proposed to refine the graph representations, ensuring they accurately reflect the underlying structural and functional characteristics of the brain networks.
3. A self-promoted contrastive pairs creation scheme is proposed, which utilizes the cluster centroids as the prototype and promotes the samples far from their corresponding prototype as the candidates to form contrastive pairs.
4. Comprehensive experiments are conducted on the real schizophrenic dataset, demonstrating the effectiveness of the proposed method.

2 RelatedWorks

2.1 Graph Contrastive Learning

Graph Contrastive Learning (GCL) is a powerful technique that leverages Graph Neural Networks (GNNs) to distinguish between similar and dissimilar instances in complex graph data [Lu et al., 2021]. Over the past few years, significant advancements have been made in this field, focusing on data augmentation, pretext task designs, and contrastive objectives [Liu et al., 2022a].

One of the notable works in this area is Deep Graph Infomax (DGI), which maximizes the mutual information between local and global graph features, adapting the idea from Deep InfoMax to graphs for node representation learning [Veliˇckovi´c et al., 2018]. Similarly, InfoGraph extends this concept to learn graph-level representations by maximizing agreements between the representations of entire graphs and their various scaled substructures. Contrastive Multi-View Representation Learning on Graphs (MVGRL) is another significant contribution that maximizes mutual information from different structural views of graphs [Xu et al., 2022]. Directed Graph Contrastive Learning (DiGCL) extends the contrastive paradigm to directed graphs, aiming at learning from abundant views while retaining the original structure information [Tong et al., 2021]. Graph Contrastive Learning with Augmentation presents a new perspective on graph contrastive learning by introducing a novel augmentation strategy, which leverages both node-level and graph-level augmentations to enhance the performance of graph contrastive learning [You et al., 2020].

Last year, AFGRL, an augmentation-free self-supervised learning framework for graphs, was proposed to generate an alternative view of a graph by discovering nodes that share the local structural information and the global semantics with the graph [Lee et al., 2022]. This method has shown superiority in various node-level tasks, including node classification, clustering, and similarity search on various real-world datasets. However, since it focuses on node-level representation, AFGRL cannot be extended to graph-level contrastive learning for brain network analysis.

These advancements in GCL have significantly expanded our understanding and capabilities in handling complex graph data, paving the way for more sophisticated applications. However, it's important to note that these methods often require careful consideration of augmentation techniques and negative sampling strategies to ensure the preservation of the underlying semantics of the graph data, which may distort the anatomical brain structure and be extremely risky when doing brain network analysis.

2.2 Clustering-based Contrastive Learning

Clustering-based Contrastive Learning (CCL) has emerged as a contrastive learning method that leverages clustering techniques to structure the learning process [Deng et al., 2018; Li et al., 2021; Sharma et al., 2020]. In the learned representation space by CCL, the samples from the same group are pulled together, while different groups are pushed away from each other [Cheng et al., 2014].

In the past few years, various CCL works have been proposed. The Supervised Contrastive (SupCon) method uses label information to pull together clusters of points belonging to the same class and push apart clusters of samples from different classes [Khosla et al., 2020]. This method has shown robustness to natural corruptions and stability to hyperparameter settings, demonstrating its potential in various datasets. Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method that combines contrastive learning with clustering to encode semantic structures into the learned embedding space [Li et al., 2020]. PCL uses prototypes as latent variables to find the maximum-likelihood estimation of network parameters in an Expectation-Maximization framework.

For the graph data, the Graph Contrastive Clustering (GCC) method is proposed for image clustering tasks, which in-corporates category information to perform contrastive learning at both the instance and cluster levels, resulting in more discriminative and clustering-friendly features and more compact clustering assignments [Zhong et al., 2021; Zhang and He, 2023]. The Multilayer Graph Contrastive Clustering Network (MGCCN) method integrates a clustering objective with graph representation learning to better capture the graph's semantic structures from multilayer graphs [Liu et al., 2022a]. To address the issue of imbalanced prototype assignments that can arise from directly using K-means, the Prototypical Graph Contrastive Learning (PGCL) approach was proposed, which adds a constraint that prototype assignments must be partitioned into equally sized subsets and formulates it as an optimal transport problem [Lin et al., 2022]. It also tackles the sampling bias by sampling negatives from clusters differing from the query cluster and reweighting negative samples according to their prototype distances.

Although these methods achieved great success, they still rely on graph data augmentations that are inappropriate for brain network analysis. Furthermore, because of the technique and cost-intensive nature of the brain network data, brain networks are always accompanied by expert annotations. Thus, it is not necessary to follow the unsupervised or semi-supervised setting.

3 The ProposedMethod

In this paper, the brain networks are presented as graphs, with G = {Gi,Xi, yi}N , where N is the total number of the brain networks, Gi is the graph topology, Xi is the nodes' features, and yi is the corresponding label of the brain network. yi = 1, if the brain network is from the healthy group, whereas yi = 0, if it is from the group with disease. Each graph is a set of nodes and edges, Gi = (Vi,Ei), where Vi = {vi1 , ..., viP } is the nodes set and Ei = {ei1 , ..., eiQ} is the edge set. Note that, the number of nodes is the same across all the brain networks, as the nodes represent the anatomical brain regions, and everyone should have the same anatomical brain structure. Whereas, the number of edges may vary from individual to individual, as everyone has their own brain connections that are formed by genes and/or personal experiences.

The overall goal is to train a model to classify the brain networks into the correct group corresponding to their labels. However, due to the scarce issue and the characteristics of the brain network data, we will pre-train the model with the proposed Self-Promoted Clustering-based Contrastive Learning (SPCCL) method without distorting the brain networks. As shown in Figure 2, the proposed SPCCL has four modules: 1) A Siamese GCN module to process the contrastive pairs; 2) A self-supervised readout module to create graph-level representations from the node embeddings; 3) A contrastive clustering module to do the supervised contrastive-based clustering, which will form the; 4) A clustering-based contrastive pair creation module to promote the contrastive pairs for the next round of training. We will introduce them one by one in the rest of this section.

3.1 Siamese GCNModule

As mentioned in the Introduction Section, the contrastive pairs are initialized from a pool incorporating the training data and additional healthy subjects. The Siamese Graph Convolutional Network (GCN) module aims to extract graphlevel representations from these contrastive graph pairs. It contains two identical subnetworks with sharing weights. This is a common setting in contrastive learning. The GCN here is a standard Graph Convolutional Network with the Chebyshev polynomial [Kipf and Welling, 2016]. In this work, we use two GCN layers to process the graph data. The topology of the graph data Gi is derived from the structural MRI images, and the node features Xi for each node are extracted from the functional MRI images. Through this, two modalities are integrated into the Siamese GCN to extract node embeddings. The output node embeddings from this module are a mix of brain network structures and brain functional signals. Thus, we will need a readout module to convert the node embeddings into graph embedding [Buterez et al.,

2022]. This is done by our proposed self-supervised readout module in the next section.

3.2 Self-Supervised Readout Module

The proposed Self-supervised readout module exploits a Bidirectional Long Short Term Memory (LSTM) model to process the output node embeddings from the Siamese GCN module. In this case, the output node embeddings are considered as a sequence of all the nodes. Each extracted node embedding is processed step by step through the bi-directional LSTM. The Bi-directional LSTM used here is to partially eliminate the bias from the node order in the sequence. The last hidden states of the Bi-directional LSTM can be considered as a latent representation encrypting all the nodes' features and topology, which will be used as the Query of the attention mechanism, while the keys and values are the hidden states of each step. The attention score is calculated by the standard attention equation:

where Q,K, V are Qurey, Keys and Values, and WQ,WK,WV are the corresponding learnable weight matrices

The output of the attention is the weighted sum of all the hidden states. Through the attention mechanism, the output readout vector is refined by the context of node embeddings, which will comprehensively explore the underlying structural and functional brain networks and emphasize the most discriminative parts of the node embeddings into the readout vector zi of the graph.

3.3 Contrastive ClusteringModule

The readout vector zi of each brain network will first be fed into the Supervised Contrastive Loss, Losssup, to get the loss value of each contrastive pair [Khosla et al., 2020]. The Losssup aims to effectively leverage label information from the contrastive pairs. Two samples coming from the same label group are called positive pairs, whereas two samples coming from different groups are called negative pairs.

where P is the positive pair set and A is the negative pair set, i is the contrastive pair from the contrastive pair set.

In this way, the label information is implicitly exploited into the contrastive loss, where the positive pairs are pulled together, and negative pairs are pushed away from each other.

After each epoch, all the readout vectors are put together to do the contrastive clustering. The number of clusters is predefined by the data labels. In our case, there are two clusters: the healthy group vs. the disorder group. The consistency loss is equipped to ensure the clustering consistency corresponding to the sample labels [Lin et al., 2022].

where l(pi, qi′) = ∑y∈Y q(y|zi′) log p(y|zi), i is the contrastive pair from the contrastive pair set, zi and zi′ are two readout vectors from one contrastive pair, p(y|zi) = softmax(zi) illustrates the probability of zi belonging to group y. Similarly, q(y|zi′) illustrates the probability of zi′ belonging to group y.

The final training loss can be written as:

where λ is a hyperparameter balancing the two terms.

3.4 Self-Promoted Contrastive Pairs Creation Module

As shown in Figure 3, the prototype of each group is chosen as the centroid of each cluster. Then, the samples of each group will be sorted by the distance to their prototype. We select the top 50% of the samples from each group to form the candidate set. The new contrastive pairs for the next training epoch will be randomly sampled from this candidate set. The total number of contrastive pairs can be predefined as a fixed number, or all the possible combinations of the candidate set. The pairs that have the same labels are considered positive pairs, whereas the pairs that have different labels are considered negative pairs.

In this way, the difficult samples of each label group are promoted into the candidate set. Our model will then be trained on the contrastive pairs created from this difficult sample set in the next epoch. For each epoch, the candidate set is updated by the clustering results. With the help of Losssup and Lossconsist, the learned readout vector will be updated accordingly, which will make the learned graph-level representations more discriminative.

Algorithm 1 SPCCL pre-training

Data pool: Mix training set and extra healthy subjects. Input: Random select M contrastive pairs. Output: Graph representations

• 1: Let t = 0.
• 2: while t < T or not converge do
• 3: for pairs in M do
• 4: do Siamese GCN
• 5: do Self-supervised readout
• 6: end for
• 7: do Contrastive clustering
• 8: backpropagate with Loss = Losssup +λLossconsist
• 9: promote M contrastive pairs with the centroids.
• 10: end while
• 11: return Graph representations of data

3.5 Classification

The pre-training process can be found in Algorithm 1. It will loop through the 4 modules until it converges or reaches a given number of epochs. After pre-tained by the proposed SPCCL model, we remove all the auxiliary modules and only use the readout vectors as the graph-level representation of brain networks. These representations then train a simple multilayer perceptron network to do the final classification. The detailed experimental settings and results are reported in the next section.

4 Experiments

4.1 Dataset

The schizophrenic dataset utilized in this study is from an open-access dataset that includes two types of brain network modalities: structural and functional connectomes. It consists of MRI data acquired from 27 schizophrenic patients, and 27 matched healthy adults [Vohryzek et al., 2020]. An additional 70 healthy adults' MRI data are used to do the pretraining [Griffa et al., 2019], which reflects the characteristic of the brain network data that it is relatively easier to acquire healthy samples than the samples with diseases. All the data used in this study was collected at the Service of General Psychiatry at the Lausanne University Hospital. The diagnosis of schizoaffective disorders for the patients was established based on their fulfillment of the DSM-IV criteria. The control subjects (healthy adults), on the other hand, had no previous history of neurological disorders, ensuring a comparison between healthy individuals and those with schizophrenia. All subjects underwent scanning using a 3 Tesla Siemens Trio scanner equipped with a 32-channel head coil. The data collection protocol involved three distinct imaging modalities. Firstly, a magnetization-prepared rapid acquisition gradient echo (MPRAGE) sequence was employed to capture white/gray matter contrast, with a high resolution of 1-mm in-plane resolution and 1.2-mm slice thickness. Secondly, a Diffusion Spectrum Imaging (DSI) sequence was utilized, comprising 128 diffusion-weighted volumes and a single b0 volume. This sequence utilized a maximum b-value of 8,000 s/mm² and had a voxel size of 2.2x2.2x3.0 mm. Finally, a gradient echo EPI sequence sensitive to BOLD contrast was employed, resulting in 280 images per participant. This sequence had an in-plane resolution and slice thickness of 3.3mm with a 0.3-mm gap, TE 30 ms, and TR 1,920 ms. For the structural connectivity analysis, deterministic streamline tractography was performed on the reconstructed DTI data. The tractography pipeline initiated 32 streamline propagations per diffusion direction per white matter voxel. Fiber density, representing the number of streamlines normalized by the average streamline length and surface area of the regions, was computed to measure the structural connectivity between pairs of regions of interest (ROIs). To estimate functional connectomes, the blood oxygen level-dependent (BOLD) time series from fMRI data was utilized. Specifically, the absolute value of the Pearson correlation was computed between the time courses of individual brain regions, enabling the assessment of functional connectivity across the brain.

4.2 Experiment Setup

To ensure unbiased performance evaluation, we employ a 6-fold cross-validation strategy during the training process. This involves randomly dividing the dataset into three equal parts, where one-third of the samples from each class are selected as the testing set, while the remaining two-thirds serve as the training set. To evaluate the classification performance, we employ six distinct metrics: accuracy (ACC), sensitivity (SENS), specificity (SPEC), positive predictive values (PPV), negative predictive values (NPV), and F1 score.

By employing these metrics, we gain comprehensive insights into the performance of our classification method. Furthermore, we compare the classification performance of our approach with three commonly used learning-based methods: K-nearest neighbor (KNN), logistic regression, and K-means clustering. By conducting these comparisons, we can gauge the effectiveness and superiority of our proposed method. Lastly, we benchmark our method against the state-of-theart approach for schizophrenia classification reported in the literature [Ghosh et al., 2023], called Graph Convolution on Brain Network (GCBN), which utilized the Graph Convolutional Networks to do the schizophrenia classification task. In GCBN, a data augmentation process is proposed to create new artificial data by inducing artificial perturbation to the available data set, which may downgrade the model performance as the perturbation may distort the brain's anatomical structures. These experimental settings enable us to ascertain the advancements and contributions of our proposed method in relation to existing methodologies.

4.3 Discussion

In order to evaluate the performance of our proposed SPCCL method, we compared it against baseline and competing approaches using six distinct quality metrics, with higher values signifying superior classification performance. The summary of our experimental results can be seen in Table 1, where we report the mean and standard deviation for each metric over the 6-fold cross-validation. The results unequivocally demonstrate that our proposed Self-Promoted Clusteringbased Contrastive Learning (SPCCL) method outperforms the other methods in terms of classification accuracy. Notably, a higher F1 score is achieved by our method, indicating its superior capabilities. The improvements were especially marked when compared to the GCBN method tailored for schizophrenia analysis, underscoring the value of the SPCCL approach for brain network analysis. As shown in Figure 4, the brain regions that have the highest attention scores throughout the patients' data are identified, such as the temporal gyrus, frontal gyrus, insula, and amygdala. These identified regions are consistent with the findings reported by neuroscience papers [Henkel et al., 2022; Huckins et al., 2019; Rubinov and Bullmore, 2013].

5 Conclusion

In the field of cognitive and behavioral neuroscience, employing Graph Neural Networks (GNN) for brain network analysis is often challenging due to the scarcity of labeled data from both technique and cost intensive MRI scans. This paper proposed a Self-Promoted Cluster-ing-based Contrastive Learning (SPCCL) method to address these limitations with respect to the unique characteristics of brain network data that its topology should be strictly aligned with the brain's anatomical brain structure. The proposed SPCCL employs a Siamese Graph Convolutional Network (GCN) framework for extracting graph-level representations, a self-supervised readout mechanism to refine the representations, a supervised clustering-based contrastive learning module for clustering consistency, and then a self-promoted contrastive pair creation scheme to fully explore the dataset without distort the brain networks. It overcomes the limitations of existing Graph Contrastive Learning methods and enhances model robustness. Tested on an open-access schizophrenia dataset, SPCCL demonstrates significant potential in advancing the diagnosis and prognosis of brain disorders.

Ethical Statement

There are no ethical issues.