Multi-Semantic Aware Self-Supervised Learning for Multi-Label Node Classification

1 Introduction

Graph Self-Supervised Learning (GSSL) aims to pre-train graph neural network models without human-annotated labels [Liu et al., 2023; Xie et al., 2023]. Unlike supervised learning, GSSL constructs proxy tasks by mining selfsupervised signals from the graph data itself. This enables the model to effectively leverage large amounts of unlabeled graph data to learn meaningful representations and generalize to various downstream scenarios [Hu et al., 2020c; Qiu et al., 2020; Hu et al., 2020b]. Therefore, GSSL has garnered significant attention and has found widespread applications such as molecular prediction [Wang et al., 2023; Zhang et al., 2021], anomaly detection [Zheng et al., 2023; Liu et al., 2022], and e-commerce recommendations [Wu et al., 2019; Ren et al., 2026].

∗Corresponding author.

Despite achieving significant success, existing GSSL methods are mainly designed for single-label scenarios, i.e., each node belongs to only one class [Zhu et al., 2020; Kipf and Welling, 2016]. While, in the real world, nodes often possess multiple semantics, i.e., one node may belong to multiple labels [Sun et al., 2018; Hu et al., 2020a]. For example, in social networks [Sun et al., 2018], a user may have multiple interests, such as being a "music fan," "movie enthusiast," and "amateur photographer". This multi-label nature is pervasive. Meanwhile, compared to single-label graphs, multi-label graphs is more complex, making manual annotation more difficult and expensive. Therefore, it is very important to explore self-supervised representation learning on multi-label graphs. Unfortunately, most existing multi-label graph learning methods [Gao et al., 2019; Xiao et al., 2022; Bei et al., 2025] are limited to supervised settings. These methods typically heavily rely on manual annotation to learn the dependencies among node and multiple labels.

Therefore, we make a bold attempt to extend graph selfsupervised learning to multi-label scenarios. It is full of challenges, specifically manifested in the following two aspects: (i) Multi-semantic entanglement (Challenge 1). In multilabel settings, a node can belong to multiple categories simultaneously. Thus, two nodes may overlap in some semantics while differing in others. A pair of ideal representation should capture this partially aligned, partially separated semantics, thus being able to fully express the complex semantics of various categories hidden in nodes. However, existing GSSL paradigms typically assume that each node possess semantic of only one category. Thus, they [Zhu et al., 2020] often treat a pair of node in either positive or negative pair. This binary alignment cannot adapt to complex semantics among multi-label nodes. The generative GSSLs [Kipf and Welling, 2016] also neglect the entangled semantics brought by multiple labels. (ii) Hard self-supervised signal mining (Challenge 2). While existing works [Bei et al., 2025] have explored multi-label graph learning, their learning of the associations among representations and multi-labels heavily relies on manual annotations. However, acquiring high-quality multi-label annotations is costly, imposing limitations on supervised approaches. In the self-supervised scenario, it is difficult to extract appropriate signals to model multi-label dependencies. We even don't know how many categories the nodes possess. Therefore, as shown in Figure 1, although existing GSSL methods can achieve comparable performance to supervised methods in the single-label domain, their performance declines when transferred to the multi-label scenario.

To address these challenges, we propose a Multi-semantic Aware Self-Supervised graph pretraining method (MASS), which aims to learn discriminative and semantically disentangled node representations without labels. For Challenge 1, we design a multi-semantic feature disentanglement module. This module does not treat node representations as a whole. Instead, it projects them into multiple distinct semantic subspaces, governed by learnable prototypes. These subspaces enable the model to separately model different semantic aspects of each node, capturing nuanced relationships where nodes may align in certain semantic dimensions while diverging in others. For Challenge 2, we introduce a contextual structure encoding module, aiming to extract richer structural signals from the graph topology to complement and enhance the semantic representations. Specifically, this module captures high-order neighborhood context information of nodes through random walk to generate structural embeddings. The structural embeddings can provide topological self-supervised signals independent of node features, offering additional self-supervised constraints in the absence of label guidance. This design enables the model to understand nodes more comprehensively from both structural and semantic perspectives, effectively alleviating the signal sparsity problem.

Our contributions can be summarized as follows:

• We pioneer the exploration of self-supervised representation learning on multi-label graphs, effectively freeing
• it from dependence on manual annotations and offering a fresh perspective in this field.
• We propose MASS, a multi-semantic aware selfsupervised graph pretraining method, enabling model to capture complementary self-supervised signals that integrate both semantic disentanglement from node features and contextual information from graph topology.
• Comprehensive experiments on four datasets show that our MASS achieves competitive performance against supervised methods on seven key metrics, convincingly demonstrating its effectiveness and superiority.

2 RelatedWork

2.1 Graph Self-Supervised Learning

Graph Self-Supervised Learning (GSSL) has emerged as a powerful paradigm for learning representations from unlabeled graph data [Xie et al., 2023; Liu et al., 2023; Wu et al., 2023]. Existing methods can be primarily categorized into generative and contrastive methods [Liu et al., 2023]. Generative GSSL methods derive supervision signals by reconstructing node attributes or graph topology. Representative works, GAE and VGAE [Kipf and Welling, 2016], adopt an encoder-decoder architecture to reconstruct the graph's adjacency matrix. GraphMAE [Hou et al., 2022] introduces feature masking and learns representations by reconstructing the masked node features. Contrastive GSSL methods learn node representations by comparing similar and dissimilar instances across different granularities. Node-level approaches like GRACE [Zhu et al., 2020] and GCA [Zhu et al., 2021] create positive pairs via augmentations and contrast them against negative instances, thereby capturing local semantic invariance. Cross-level approaches such as DGI [Velickovic et al., 2019] maximize mutual information between local node embeddings and global graph summaries to learn discriminative representations.

Despite notable success, they typically treat node features as an indivisible whole, overlooking the inherent multi-semantic nature of nodes in multi-label graphs. Consequently, directly applying existing self-supervised frameworks to multi-label data often results in entangled representations and limited semantic discriminability.

2.2 GraphMulti-Label Learning

Traditional graph multi-label learning is usually based on supervised or semi-supervised frameworks, focusing primarily on modeling label correlations and incorporating label semantics into graph neural networks [Shi et al., 2020; Akujuobi et al., 2019]. Early studies, such as ML-GCN [Gao et al., 2019], propose projecting labels and nodes into a shared embedding space and constructing label-label and node-label correlation matrices to enhance node representations. Subsequently, some methods like LANC [Zhou et al., 2021a] and LARN [Xiao et al., 2022] incorporate label-aware attention mechanisms to capture fine-grained label dependencies. Then, recent work CorGCN [Bei et al., 2025] further addresses challenges such as label ambiguity and structural noise through its correlation-aware graph convolution mechanism. LIP [Sun et al., 2025] dynamically guides the model learning process by computing higher-order label correlations and propagating influence between labels.

Despite their effectiveness, these methods rely heavily on extensive high-quality labeled data, which is often costly or difficult to obtain in real-world multi-label scenarios.

2.3 Self-Supervised Learning onMulti-Label Data

Researches on self-supervised multi-label learning remains scarce, with only a few works focusing on the field of computer vision [Zhu et al., 2023; Chen and Yeh, 2024; Lin et al., 2021]. For instance, OASS [Xu et al., 2022] leverages object detection to generate region-level pseudo-labels. By designing region-specific contrastive learning tasks, it guides the model to learn semantically distinct representations for different visual concepts within a single image, thereby achieving semantic disentanglement at the object level.

However, in the graph domain, self-supervised learning for multi-label graphs remains unexplored. To the best of our knowledge, no existing work can address the core challenges of multi-label graph learning under a label-free setting. It's the gap that our work aims to bridge.

3 Preliminaries

Notations. Given a graph G = (V, E), consisting of N vertices V = {v0, v1, · · · , vN} and M edges E ⊂ V × V. The graph structure is represented by an adjacency matrix A ∈ {0, 1}N×N with Aij = 1 if (vi, vj) ∈ E. The node features are represented by a feature matrix X = [x⊤1 , x⊤2 , · · · , x⊤N ]⊤ ∈ RN×F , where each row xi ∈ RF denotes the F-dimensional feature vector of node vi. In multilabel graph learning, each node is associated with a label set Yi ⊆ C, where C = {c1, c2, · · · , cC} denotes the label space of C semantic categories. This association can be represented by a binary vector yi = [yi1, yi2, · · · , yiC] ∈ {0, 1}C , where yij = 1 if node vi belongs to category cj, and yij = 0 otherwise. The collection of all node labels forms the label matrix Y = [y1, y2, · · · , yN]⊤ ∈ {0, 1}N×C .

Problem Definition. In this work, we address the selfsupervised multi-label graph representation learning problem, where the label matrix Y is unavailable during training. Our objective is to learn an encoder fθ(X,A) that generates low-dimensional node embeddings Z = fθ(X,A) ∈ RN×d where d ≪ F, with zi denoting the embedding of node vi. The learned representations should capture the multiple semantic facets within nodes without any label, such that they serve as effective representations for downstream multilabel node classification tasks.

4 Methodology

In this section, we present our proposed MASS framework in detail, comprising multi-semantic feature disentanglement (Sec. 4.1) and contextual structure encoding (Sec. 4.2). Subsequently, we elaborate on the loss function (Sec. 4.3). The overall framework of MASS is illustrated in Figure 2.

4.1 Multi-Semantic Feature Disentanglement

To address Challenge 1, we introduce a multi-semantic feature disentanglement module that decomposes holistic node representations into multiple independent semantic subspaces, each corresponding to a potential semantic category. By this, the module can capture the differentiated characteristics of nodes across various semantic dimensions.

Semantic Subspaces

We first adopt two-layer GCN [Kipf and Welling, 2017] as the base encoder to obtain initial nodes feature representations Zf . Nevertheless, Zf remains a fused representation where different semantics are entangled. To explicitly disentangle these semantics, we propose to decompose the unified representation Zf into C independent semantic subspaces, each corresponding to a potential semantic category c ∈ {1, 2, . . . ,C}. Specifically, we design C independent projection headers, formulated as:

whereWc ∈ Rdh×dc and bc ∈ Rdc are learnable parameters, and Rc ∈ RN×dc denotes the representation of all nodes projected into the c-th semantic subspace.

This design allows each semantic to have an independent space, avoiding interference between different semantics. Thus, we obtain a set of independent semantic representations {Rc}Cc=1 , each capturing a distinct semantic facet of the nodes without mutual interference.

Codebook-Based Pseudo-Labels

Due to lacking guidance from ground-truth labels in selfsupervised learning setting, the model should seek supervisory signals from data itself. To address this, we try to learn a set of pseudo-labels as alternatives to guide model by leveraging a codebook-based prototype learning mechanism.

Specifically, we maintain a codebook of K prototypes for each category c. It can be denoted as pc = {p1c , p2c , . . . , pKc }, where each prototype pkc represents potential semantic within category c. These prototypes are dynamically aggregated through an attention mechanism to form the category's central representation as pseudo-labels:

where w⊤ a ∈ Rdc is a learnable attention vector.

Label Alignment Loss

To leverage the learned pseudo-labels to guide representation learning, we design a label alignment loss that encourages node embeddings to align with their corresponding semantic central representation.

Specifically, given the pseudo-label ¯ pc and the node representation rci in semantic subspace c, we first compute the confidence score that node vi belongs to category c:

where σ(·) is the sigmoid function, mapping it to (0, 1).

Based on this confidence score, we define positive and negative sample sets for each category c with a threshold τ ∈ (0, 1):

Then, to encourage positive samples to align with their corresponding pseudo-label while pushing negative samples away, the label alignment loss is formulated as:

where cos(·, ·) denotes cosine similarity, E denotes the mathematical expectation.

Thus, L(l) align provides self-supervised alignment signals to the model and enables each semantic subspace to autonomously learn category prototypes, achieving alignment of node representations in shared semantic subspaces and separation in non-shared ones.

Contrastive Regularization

Following the smoothing assumption of graphs, we introduce an aggregation constraint to preserve local consistency. Meanwhile, to prevent the representation space from collapsing into trivial solutions, we add a separation constraint to promote variance across semantic dimensions. Therefore, the overall contrastive regularization loss can be defined as:

where γsep is the weight coefficient, and Lagg and Lsep can be formulated as:

where |E ′ | = {(vi, vj) ∈ E|i ̸= j} denotes the set of edges excluding self-loops. Zf i represents the feature representation of node vi. ˆZ:,j denotes the normalized representation of all nodes in the j-th semantic dimension.

By enhancing local consistency and dimensionality diversity, Lcontrast preserves local semantic similarity between neighboring nodes and effectively prevents representation space from collapsing into trivial solutions during selfsupervised learning.

Consistency Regularization

Notably, in self-supervised learning, models are prone to premature convergence to suboptimal solutions or overconfident predictions, where predictions may collapse into a few dominant categories while ignoring other plausible semantic associations. This issue is particularly severe in multi-label learning, as the model must identify and balance multiple coexisting semantics for each node without guidance from groundtruth labels.

Thus, we design a temperature-scaled consistency regularization loss to guide the learning of semantic subspaces and pesudo-labels. This loss controls the entropy of the predicted distribution through a temperature parameter, dynamically balancing exploration and discrimination of model.

Specifically, for the confidence score si = [s1i , s2i , · · · , sCi ] of node vi, we introduce two temperature parameters T and T ′ , where T > T ′ > 0, to generate smoothed and sharpened distributions respectively:

The smoothed distribution qsmooth i has higher entropy with more uniform probability distribution across labels, encouraging the model to explore potential relationships between different labels. The sharpened distribution qsharp i has lower entropy with more prominent probabilities for highconfidence labels, prompting the model to make clearer distinctions. The consistency loss minimizes the KL divergence between them:

This design enables the model to focus more on exploring inter-label relationships in early training stages and gradually enhance discriminative capability as training progresses. Meanwhile, the signals provided by the soft distribution are richer than hard assignments, transmitting more detailed information through the KL divergence constraint between distributions, effectively mitigating pseudo-label noise issues.

4.2 Contextual Structure Encoding

To address Challenge 2, we aim to mine richer information from graph data in a self-supervised setting. Besides node attributes, we seek to extract more comprehensive signals from the topological structure of the graph. Therefore, we design a contextual structure encoding module that learns node embeddings from higher-order neighborhood patterns, complementing the multi-semantic feature disentanglement module.

We adopt a DeepWalk-based [Perozzi et al., 2014] structural encoding approach to obtain structural embeddings of the graph. Specially, we employ random walks to generate sequences that capture L-hop neighborhood structures. Starting from each node vi, we generate M random walk sequences of length L : Wi = {walk1i ,walk2i , · · · ,walkMi }, where walkmi = (v0, v1, · · · , vL-1) with v0 = vi.

Then, we follow the Skip-gram [Mikolov et al., 2013] model to learn structural embeddings of nodes by maximizing the log probability of co-occurrence between nodes and their context nodes in the walk sequences:

where w is the window size, ui is the structural embeddings vector of node vi. By optimizing the above objective, we obtain structural embeddings U. We apply a multi-layer perceptron (MLP) to the structural embeddings U to obtain the final structural representations Zs = MLP(U) ∈ RN×ds .

4.3 Loss Function

Considering that representations obtained from the feature space and structural space for the same node should exhibit consistency at the semantic level, we design a featurestructure alignment loss function:

where S = ˆZf ˆH⊤s is the similarity matrix between feature representations and projected structural representations.

Here, ˆZf and ˆHs represent the normalized feature and structural representations, respectively, with Hs = proj(Zs) obtained via a projection network proj(·) that maps Zs into the same semantic space as Zf .

The self-alignment component of this loss function minimizes the distance between the feature and structural representations of the same node. Concurrently, the exclusion component penalizes positive similarities between different nodes, thereby preventing their representations from becoming excessively close. From an information-theoretic perspective, this formulation maximizes mutual information between the two views of each individual node while minimizing mutual information between different nodes.

With the losses mentioned above, the overall loss function of MASS is as follows:

where α, β and γ are the controllable hyper-parameters.

After optimization, the model produces two complementary representations: the feature representation Zf and the structure representation Zs. Then, we obtain the final representation by concatenating them, i.e., Z = Concat(Zf ,Zs). The combined representation Z ∈ RN×(dh+ds) captures both feature semantics and graph topology, providing a comprehensive embedding for downstream tasks.

5 Experiments

In this section, we evaluate our proposed MASS framework through comprehensive experiments, aiming to answer three fundamental questions: (i) RQ1: How does MASS compare against state-of-the-art methods across diverse multilabel graph datasets? (ii) RQ2: What is the contribution of each component to the overall performance? (iii) RQ3: How sensitive is MASS to key hyperparameter settings?

5.1 Experimental Setup

Datasets. We conduct comprehensive experiments on four widely-used multi-label graph datasets: Humloc [Zhao et al., 2023], PCG [Zhao et al., 2023], Blogcatalog [Zhou et al., 2021b] and PPI [Zeng et al., 2020] for evaluation, covering various domains including social networks, academic networks, and biological networks.

Baselines. We compare MASS with the following seven state-of-the-art models: (i) Semi-supervised multi-label learning methods: LARN [Xiao et al., 2022], LIP [Sun et al., 2025], and CorGCN [Bei et al., 2025]; (ii) Self-supervised learning methods: GAE [Kipf and Welling, 2016], DGI [Velickovic et al., 2019], GRACE [Zhu et al., 2020], and FUG [Zhao et al., 2024]. All baseline methods are implemented using official code or reproduced according to original papers. We equip all models with the GCN backbone architecture to ensure fair comparison.

Evaluation Metrics. To comprehensively evaluate multilabel node classification performance, we employ seven standard evaluation metrics. Ranking Loss and Hamming Loss measure the proportion of misclassified labels and label ranking quality, respectively; Macro-AUC and Micro-AUC evaluate classifier discrimination capability at different granularities; Macro-AP and Micro-AP assess prediction accuracy based on precision-recall curves; LRAP specifically evaluates average precision for label ranking. We adopt a standard experimental protocol: randomly split 10% of nodes as the training set, 10% as the validation set, and the remaining 80% as the test set. To ensure reliability, all experiments are repeated 20 times with different random seeds, and we report average results with standard deviation, providing stable performance evaluation. The source code is available at https://github.com/hedongxiao-tju/MASS.

5.2 Experimental Analysis

For RQ1, we evaluate the performance of our proposed MASS framework against seven state-of-the-art baseline methods on four multi-label graph datasets. The results of seven metrics are reported in Table 1. We adopt Friedman test to assess the statistical significance. The results of the Friedman test for all seven evaluation metrics are shown in Table 2. With all p-values far less than the significance level α = 0.05, the null hypothesis that all methods perform equally is rejected, thereby confirming statistically significant differences across the compared methods.

From Table 1 and Table 2, we can observe that: (i) Our proposed MASS achieves significant improvements over other baselines across four experimental datasets. MASS achieves the best or highly competitive results on seven evaluation metrics, with an average rank of 1.32, significantly outperforming all other methods. Specifically, in terms of Ranking Loss and Hamming Loss, MASS generally outperforms the baselines, demonstrating clear advantages in label ranking quality and classification error rate. Regarding AUC and AP metrics, MASS achieves the best or second-best performances in macro and micro measures, highlighting its strong discriminative capability even under class imbalance and label scarcity. Especially on Blogcatalog, MASS achieves a Micro-AUC of 77.72%, outperforming the second-best model by 5.34%. (ii) MASS performs better than existing self-supervised learning methods. Compared with self-supervised baselines, MASS shows clear gains across nearly all metrics. This verifies that our model MASS can effectively capture the multi-semantics of nodes in multi-label graphs, which are overlooked by conventional self-supervised approaches designed for single-label scenarios. (iii) MASS achieves competitive performance even compared to semi-supervised multi-label learning methods. Notably, without manual annotation during training, MASS matches or surpasses semi-supervised baselines on multiple metrics. This indicates that the proposed pretext tasks in our model successfully extract rich supervisory signals from the graph data itself, thereby guiding model to learn more discriminative and semantically disentangled node representations. Overall, the strong performance validates the effectiveness of our proposed MASS in self-supervised multilabel graph learning.

5.3 Ablation Study

For RQ2, to validate the effectiveness of each component in our proposed MASS framework, we conduct systematic ablation studies by comparing MASS with the following variants: (1) w/o MFD removes the multi-semantic feature disentanglement module. (2) w/o CSE removes the contextual structure encoding module. (3) w/o Align removes the label alignment loss L(l) align . (4) w/o Contr removes the contrastive regularization loss Lcontrast. (5) w/o Consist removes the consistency regularization loss Lconsist.

Table 3 summarizes the ablation results. We observe that: (i) Effectiveness of multi-semantic feature disentanglement module and contextual structure encoding module. Removing either the feature disentanglement module (w/o MFD) or the structure encoding module (w/o CSE) leads to significant decline in performance across all datasets. This demonstrates that modeling multi-semantics of nodes and capturing signals from topological structures are essential to self-supervised multi-label learning. (ii) Effectiveness of regularization losses. The removal of any regularization term impairs model performance, which demonstrates that these regularization terms are not merely auxiliary. These losses respectively prevent representation collapse and mitigate over-confidence in early training, thereby enhancing the model's ability to identify and separate multiple semantics.

5.4 Parameter Study

For RQ3, we investigate the sensitivity of MASS to key hyperparameters including the number of prototypes K and threshold τ .

As shown in Figure 3, both K and τ influence model performance. When K is too small, the model fails to capture the semantic diversity within each category, This results in limited guidance from pseudo-labels, thereby leading to suboptimal performance. With K increasing, the model can capture finer semantics, and performance improves accordingly. When K becomes excessively large, the performance consequently declines. This indicates that redundant noise may be introduced into the pseudo-labels, causing semantic overlap or interference. Similarly, when τ is too low, some low-confidence samples were incorrectly classified as positive samples, causing the representations to deviate from the ideal semantic space. Conversely, when τ is too high, some high-confidence samples that should belong to the positives are erroneously excluded, hindering effective semantic alignment, which consequently degrades performance. Therefore, appropriate values of K and τ can enhance the model's overall performance.

6 Conclusion

In this paper, we explore self-supervised learning on multilabel graph data and propose a Multi-semantic Aware SelfSupervised graph pretraining method (MASS), which can effectively capture multiple semantics within nodes and free it from dependence on manual annotations. Extensive experiments demonstrates that our proposed MASS achieves competitive performances with supervised baselines.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No.62276187, No.62422210 and No.62372323).

Contribution Statement

Jiayu Zhang and Jitao Zhao contributed equally to this work.