Leveraging Peer-Informed Label Consistency for Robust Graph Neural Networks with Noisy Labels

1 Introduction

Graph Neural Networks (GNNs) have demonstrated excellent performance in various applications. Training a GNN requires data, but the labels of data can be noisy. This noise can originate from human annotators or automatic label extraction tools, such as crowd-sourcing and web crawling. Existing research indicates that GNNs tend to memorize label noise [Zhang et al., 2021], and noise can propagate through the graph structure [Dai et al., 2021; Qian et al., 2023], degrading the performance. Therefore, training a robust GNN to resist label noise is crucial. As label noise problems may appear in any context, such robustness enhances GNN reliability in many applications, including recommender systems

∗Corresponding Author

[Wu et al., 2022], traffic forecasting [Sun et al., 2022] and molecular property prediction [Wieder et al., 2020].

Current research presents various solutions to mitigate the issue of label noise. One prevalent approach involves identifying accurately labeled samples. A commonly adopted strategy is to consider samples with small loss values as clean. This strategy is based on the observation that, in the early stages of training, loss values of clean samples are smaller than those of mislabeled samples [Han et al., 2018; Yu et al., 2019]. Some studies also leverage representation space [Wu et al., 2020] or training dynamics [Pleiss et al., 2020] to identify noisy data. To enhance the robustness of GNNs against label noise, existing methods utilize graph structure. These approaches include graph structure refinement [Dai et al., 2021; Dai et al., 2022], topology-based curriculum learning [Wu et al., 2024], and introduction of additional supervision signals [Qian et al., 2023; Du et al., 2023; Yuan et al., 2023a; Chen et al., 2024]. Despite these advancements, existing approaches for identifying clean samples remain dependent on model outputs. However, noisy supervision during training can compromise the reliability of outputs. Consequently, sample selection that relies on model outputs can be misled by noisy labels, potentially resulting in confirmation bias [Tarvainen and Valpola, 2017] and suboptimal performance. The intrinsic properties of graph data are not adequately exploited, thereby reducing the effectiveness in dealing with label noise.

To address this issue, we propose a method that leverages the label consistency of semantically similar nodes for sample selection. For each target node, a set of peers is identified, comprising nodes with either similar features or structural adjacency within the graph. Label consistency is defined as the proportion of peers sharing the target node's assigned label, serving as a metric to evaluate its correctness. This approach is grounded in two general properties of graph data: (1) nodes with similar features are more likely to share the same label [Zhu et al., 2022], and (2) graph homophily [McPherson et al., 2001] and monophily [Altenburger and Ugander, 2018] suggest that nodes often belong to the same semantic class as their one-hop or two-hop neighbors. Consequently, correctly labeled nodes typically demonstrate higher label consistency with peers, while mislabeled nodes exhibit lower consistency. This discrepancy, as illustrated in Figure 1a, underpins the identification of mislabeled nodes through the analysis of peer-informed label consistency.

The challenge arises from the unavailability of ground truth labels, which prevents the calculation of true label consistency. One workaround is to use the predictions from a neural network classifier as a proxy. However, neural networks are prone to memorizing and overfitting noisy labels [Arpit et al., 2017; Zhang et al., 2021]. Additionally, the message-passing mechanism of GNNs inherently smooths outputs across neighboring nodes [Chen et al., 2020]. Consequently, classifier predictions for the peers of mislabeled nodes tend to align with incorrect labels, making label consistency between mislabeled and correctly labeled nodes indistinguishable, as shown in Figure 1b.

To address this challenge, we employ representations and prototypes to assign pseudo-labels to nodes and estimate true label consistency. A prototype is calculated as the average representation of samples in a class, serving as a representative point in the representation space. Intuitively, samples with the same ground truth label tend to cluster together in the representation space [Papyan et al., 2020; Zhu et al., 2021], with clean samples dominating the prototype calculation, as shown in Figure 1d. Consequently, prototypes are more robust and less prone to memorizing noisy labels compared to neural network classifiers. Each node is assigned the label of its nearest prototype, and the resulting label consistency is referred to as prototypical consistency. A Gaussian Mixture Model is then fitted to the prototypical consistency distribution of the training set to distinguish between correctly labeled and mislabeled nodes. The correctly labeled nodes identified in this process refine the prototypes, leading to more accurate label consistency estimation and enhanced reliability of label identification in subsequent iterations, forming a positive feedback loop. Since noisy labels can degrade the quality of node representations [Wang et al., 2024; Wu et al., 2020], we incorporate a graph self-supervised module to maintain high-quality node representations even in the presence of noisy labels. Our contributions are as follows:

1. We introduce a novel perspective, suggesting that the intrinsic properties of graph data can be exploited to identify

mislabeled nodes.

1. We propose ProCon, a novel graph learning framework for noisy labels, incorporating the peer-informed label consistency and node representations to identify mislabeled nodes.
2. Extensive experiments conducted on multiple datasets demonstrate that ProCon outperforms current state-of-the-art methods, demonstrating its superior performance in handling noisy-labeled graph data.

2 RelatedWork

2.1 Learning with Noisy Labels

Approaches to learning with noisy labels can be primarily divided into robust algorithms and sample selection strategies. Robust algorithms are designed to train networks on noisy datasets while mitigating performance degradation. This approach includes modifications in network architectures [Lee et al., 2019; Goldberger and Ben-Reuven, 2017], loss functions [Zhang and Sabuncu, 2018], and robust regularization [Wei et al., 2021; Menon et al., 2020].

Sample selection methods focus on identifying clean samples within noisy datasets. Motivated by the memorization effect of deep neural networks (DNNs) [Arpit et al., 2017], early studies typically employ peer networks and the small-loss trick, exemplified by methods such as Decoupling [Malach and Shalev-Shwartz, 2017], Co-teaching [Han et al., 2018], and Co-teaching+ [Yu et al., 2019]. Another line of research aims to establish new sample selection criteria. For instance, AUM [Pleiss et al., 2020] identifies mislabeled data by monitoring its prediction margin during training, while TopoFilter [Wu et al., 2020] identifies isolated data in the representation space as mislabeled. Upon obtaining clean data, a straightforward strategy is to train the network solely on clean data or to re-weight the data. Some studies [Li et al., 2020; Wang et al., 2024] treat selected noisy data as unlabeled and apply semi-supervised or contrastive learning techniques. Others [Xiao et al., 2015; Yi and Wu, 2019] develop labelcorrection modules to pseudo-label noisy data for training.

2.2 Robust Graph Neural Networks

Graph Neural Networks (GNNs) excel in tasks such as node classification, link prediction, and graph classification by leveraging graph structure to propagate information and capture both local and global patterns [Sun et al., 2024]. Nevertheless, existing work [Dai et al., 2021; Du et al., 2023; Qian et al., 2023] reveals that the performance of GNNs can be substantially compromised by noisy labels.

Recent studies have introduced various strategies to enhance the robustness of GNNs against noisy labels. Several approaches enhance message passing in GNNs to counteract the effects of label noise. For instance, NRGNN [Dai et al., 2021] connects unlabeled nodes with similar labeled counterparts to integrate clean label information, while RS-GNN [Dai et al., 2022] diminishes noisy edges by learning a denoised graph.

Other approaches introduce supplementary and reliable supervision, including self-reinforcement and consistency regularization [Qian et al., 2023], principles guided by information theory [Zhou et al., 2023], and pairwise interactions [Du et al., 2023]. Additionally, graph contrastive learning has been utilized to address label noise, as evidenced by RNCGLN [Zhu et al., 2024], ALEX [Yuan et al., 2023a], and CGNN [Yuan et al., 2023b]. In addition, UnionNet [Li et al., 2021] employs label aggregation to facilitate sample reweighting and label correction. TSS [Wu et al., 2024] introduces a Class-conditional Betweenness Centrality measure to optimize sample selection using topological information for curriculum learning. ERASE [Chen et al., 2024] adopts label propagation and maximize coding rate reduction. Furthermore, GNN Cleaner [Xia et al., 2024] presuppose the availability of a clean set.

While previous studies have effectively enhanced the robustness of Graph Neural Networks (GNNs) against label noise, few have leveraged the intrinsic properties of graph data to filter out noisy samples. In contrast, our method leverages peer-informed label consistency by selecting semantically similar peers based on feature similarity and graph adjacency. This intrinsic exploitation allows ProCon to accurately identify mislabeled nodes, thereby significantly improving the robustness and effectiveness of GNNs.

3 Method

3.1 Preliminaries

We focus on the node classification task with label noise. For an undirected graph G = (V, E) with n nodes, the node set is denoted as V = {v1, v2, ..., vn}, and the edge set is denoted as E ⊆ V × V. Let A ∈ Rn×n be the adjacency matrix of G. If nodes vi and vj are connected, then Aij = 1, otherwise Aij = 0. Let X ∈ Rn×d denote the node feature matrix, where Xi ∈ Rd represents the feature vector of node vi. Let Y = (y1, y2, . . . , yn) denote the node label vector, where yi ∈ [L] = {1, 2, . . . ,L} represents the label of node vi. The training node set is denoted as Vtr ⊂ V. The labels of the training set, denoted as Ytr, are corrupted by noise. Our objective is to train a GNN encoder f and a classifier g from the noisy training set such that g ◦ f : (X,A) →Y.

3.2 Divide by Peer-Informed Prototypical Consistency

Our goal is to identify correctly labeled nodes and estimate their probabilities of being correct. The key to our method lies in harnessing the disparity in peer-informed label consistency between correctly labeled and mislabeled nodes.

We first outline the process for identifying the peers of each node. The objective is to ensure that the labels of the selected peers are likely to align with the label of the target node. Peer selection is driven by two main factors: (1) feature similarity, as nodes exhibiting similar features are more prone to sharing the same label, and (2) graph adjacency, since nodes are more likely to share labels with their first- or second-order neighbors [McPherson et al., 2001; Altenburger and Ugander, 2018]. To quantify feature similarity, we construct a similarity matrix S ∈ Rn×n , where Si,j represents the cosine similarity between nodes vi and vj , computed as follows:

Subsequently, the similarity matrix S is refined by incorporating graph adjacency information:

Here, ⊙ represents the Hadamard product, and 1 is a column vector of ones. The matrix A[2] ∈ Rn×n represents the twohop reachability matrix of graph G. If nodes vi and vj are reachable within two hops, then A[2] i,j equals 1; otherwise, it is 0. The hyperparameter α ∈ (0, 1) controls the influence of graph structure on the similarity matrix. A smaller value of α emphasizes selecting peers from local neighbors, while a larger value places more weight on global feature similarity. The vector ˜Si represents the topology-aware similarity of all nodes with respect to node vi. For a target node vi, let the peer set be denoted as ˜ V (i) . Peers are selected based on the top-k largest values in ˜Si:

where Topk (·) represents the set of nodes corresponding to the top-k largest entries in the input.

To evaluate label consistency between training nodes and peers, pseudo-labels are assigned to unlabeled peers using a prototype-based method. We define the node representation matrix as Z, with Zi denoting the representation of node vi. Node representations are derived from a GNN encoder f , parameterized by θ, as Z = f(X,A; θ). For each class ℓ, a prototype representation cℓ is maintained as its representative vector. The pseudo-label ˆ yi for node vi corresponds to the class of the closest prototype:

We use cosine distance to measure the closeness between representations. The prototypical pseudo-label ˆ yi indicates the class to which node vi belongs in the representation space.

Subsequently, label consistency for training nodes is computed by given labels and prototypical pseudo-labels of peers. We term the label consistency value Peer-Informed Prototypical Consistency.

Definition 1 (Peer-Informed Prototypical Consistency). Peer-Informed Prototypical Consistency for a node vt is defined as the proportion of its peers whose prototypical pseudo-labels match the given label:

where yt is the given label of node vt, ˆ yj is the prototypical pseudo-label of node vj, and ˜ V (t) represents the peers of vt.

The prototypical pseudo-labels of peers represent the expected labels for the target node, derived from representations, features, and graph structure. Prototypical consistency measures this expectation, with lower values reflecting increased conflict between the given label and the expected label, thereby implying a higher likelihood of label error.

The prototypical consistency can vary across different nodes and graphs. Therefore, rather than setting a fixed threshold for sample selection, we analyze the empirical distribution of prototypical consistency value. Let the consistency values of training nodes be H = {ht : vt ∈ Vtr}. To distinguish between mislabeled and correctly labeled training nodes, a two-component Gaussian Mixture Model (GMM) is fitted to H using the expectation-maximization algorithm. For any training node vt, the GMM outputs the posterior probability p(m | ht) of its label being clean, where m represents the Gaussian component with the larger mean. For simplicity, we refer to p(ℓ | ht) as wt. Let W = {wt : vt ∈ Vtr} be the set of clean probabilities of training nodes. We set a threshold τ on clean probabilities, and consider nodes satisfying wt > τ as clean. The set of all selected clean nodes is denoted as Vc = {vt : vt ∈ Vtr ∧ wt > τ}. Note that Vc ⊂ Vtr. Using GMM to fit the distribution allows for a more flexible and accurate selection process than setting a fixed threshold, as it can adapt to different prototypical consistency distributions and provide the probability of each sample being clean.

3.3 Weighted Prototype Updating

The canonical way for updating prototypes is to take the simple average of representations from the same class after each training iteration. However, mislabeled samples may lead to the inclusion of representations that are not truly from the same class, thereby compromising prototype quality.

To achieve robust and accurate prototypes, we propose using a weighted average of selected clean sample representations. Specifically, the prototype updates are performed as follows:

where V ℓ c = {vi : vi ∈ Vc ∧yi = ℓ} denotes the set of identified clean training nodes with label ℓ. wi represents the clean probability of node vi, and γ ∈ (0, 1) is the prototype momentum factor. For each class, the average of filtered clean samples, weighted by clean probabilities, is computed. Subsequently, prototypes are updated using the exponential moving average method. Both the representations and the updated prototypes are normalized to the unit sphere.

Updating prototypes by computing the weighted average of selected clean node representations mitigates the impact of mislabeled nodes, thereby enhancing prototype quality. This enhanced prototype quality subsequently improves the accuracy of pseudo-labeling and label consistency estimation. As label consistency estimation becomes more precise, the GMM more effectively identifies noisy labels, resulting in cleaner samples for future prototype updates and establishing a positive feedback loop.

3.4 Robust Representation Learning

Noisy labels can compromise the quality of representations learned by the network [Wang et al., 2024; Yi et al., 2022; Li et al., 2022], resulting in suboptimal prototypes and inaccurate sample selection. To mitigate the impact of noisy labels and achieve robust node representations, ProCon incorporates a self-supervised learning module.

Drawing inspiration from bootstrap self-supervised learning techniques [Grill et al., 2020; Thakoor et al., 2022], this module enhances node representations by predicting alternative augmentations of the input. Two encoders, namely an online encoder and a target encoder, are employed to obtain node representations from different augmentations. Specifically, for the graph G with node feature matrix X and adjacency matrix A, two independent random graph augmentations are applied to G, generating two views G (1) and G (2) . The augmented feature matrices are denoted as X(1) and X(2) , with the corresponding adjacency matrices A(1) and A(2) . Besides the encoder f(·, ·; θ) serving as the online encoder, this module introduces a target encoder ˜ f(·, ·; ˜ θ) sharing the same architecture. The two different views G (1) and G (2) are processed through the online encoder and target encoder, respectively, yielding node representation matrices node representation matrices Z(1) = f(X(1) ,A(1) ; θ) and Z(2) = ˜ f(X(2) ,A(2) ; ˜ θ). The online representation Z(1) is subsequently fed into a predictor network fp, producing predictions H(p) = fp(Z(1) ; θp) for the target representation Z(2) . The loss function for updating the online encoder and predictor is defined as the mean negative cosine similarity between Z(2) and H(p) for each node:

where vector H(p) i and Z(2) i represent the i-th components of the matrices H(p) and Z(2) , respectively. The target encoder parameters ˜ θ are updated as the exponential moving average of the online encoder parameters θ with a decay rate µ:

The self-supervised learning module functions independently of noisy labels during training, thus mitigating the detrimental effects of noisy labels on node representations. Additionally, this module utilizes unlabeled nodes, facilitating the encoder in learning superior node representations.

3.5 Model Training

The training regime within our framework is partitioned into two modules. The first module entails supervised learning with only the selected clean training nodes, whereas the second module involves self-supervised learning encompassing all nodes. Both modules use the same GNN encoder f. For the supervised learning module, the cross entropy loss function is used, defined as follows:

where g(·; ϕ) is a linear classifier parameterized by ϕ, which takes a node representation as input and produces a logit vector. The term yℓ i represents the ℓ-th component of the one-hot encoded label yi, while gℓ (zi;ϕ) indicates the ℓ-th component of vector g(zi; ϕ).

For the self-supervised learning module, the loss function LU is defined in Eq. (7). The overall loss function integrates the cross entropy loss and the self-supervised loss as a weighted average:

The parameter λ ∈ (0, 1) operates as the loss balance factor, regulating the trade-off between representation learning and classification. The pseudo code of ProCon is presented in the Appendix A, with an overview depicted in Figure 2.

4 Experiments

4.1 Setup

Datasets and Settings. We conduct experiments on Cora [McCallum et al., 2000], CiteSeer [Giles et al., 1998], and WikiCS [Mernyei and Cangea, 2020] datasets. More experimental results from additional datasets and dataset statistics are detailed in Appendix D and B. For all datasets, 10% of the nodes are randomly selected for training, 10% for validation, and the rest for testing. Following previous works [Dai et al., 2021; Du et al., 2023], labels of training and validation sets are randomly corrupted in the following two ways:

1. Symmetric-p (Sym-p): Labels have a probability p of being uniformly flipped to any other class.
2. Asymmetric-p (Asym-p): Labels have a probability p of being non-uniformly flipped to specific other classes, reflecting common error patterns where some classes are more likely mislabeled as similar ones.

Details of the noise synthesis process are provided in Appendix C. The probability p is also referred to as the 'noise rate'. We set p ∈ {0.2, 0.4, 0.6} for symmetric noise and p ∈ {0.2, 0.3, 0.4} for asymmetric noise.

Implementation details including model architectures and hyperparameters are provided in Appendix E. All hyperparameters are tuned on validation sets. Each experiment is repeated 10 times with different random seeds. The means and standard deviations are reported.

Baselines. We compare ProCon with several state-of-theart noisy label learning methods to evaluate its effectiveness. General noisy label learning methods compared include Me-momentum [Bai and Liu, 2021], Co-teaching+ [Yu et al., 2019] and ProMix [Xiao et al., 2023], which use the memorization effect, AUM [Pleiss et al., 2020] based on training dynamics, and TopoFilter [Wu et al., 2020] leveraging representation space. Additionally, ProCon is compared against noisy label learning approaches for graph data, including NRGNN [Dai et al., 2021], PI-GNN [Du et al., 2023], RTGNN [Qian et al., 2023] and TSS [Wu et al., 2024].

4.2 Main Results

Table 1 reports the experimental results under different noise settings for the Cora, CiteSeer, and WikiCS datasets. ProCon consistently exhibites superior performance across diverse datasets and noise levels. For the Cora dataset, ProCon achieves an average improvement of 3.25% under symmetric noise and 3.51% under asymmetric noise compared to the best baseline. On the WikiCS dataset, ProCon shows an average improvement of 3.63% under symmetric noise and 4.88%

under asymmetric noise settings compared to the best baseline. Under high noise scenarios, our method demonstrates remarkable performance gains over other baseline methods. For example, with an asymmetric noise rate of 0.4, ProCon surpasses the vanilla Cross Entropy method by 8.74% on the WikiCS dataset, while the best baseline improves by only 3.72%. Additionally, graph-specific baseline methods consistently outperforms general ones, emphasizing the advantage of utilizing graph properties in noisy label learning. This partly elucidates the superior performance of ProCon, as it harnesses graph structure to identify mislabeled nodes.

Sample Selection Performance. Figure 3 illustrates the sample selection efficacy of ProCon. We compare ProCon with two alternative baselines: 1) RTGNN [Qian et al., 2023], a graph learning approach that utilizes loss distributions for sample selection, and 2) TopoFilter [Wu et al., 2020], a general sample selection method that leverages representational patterns. ProCon integrates label consistency with feature similarity, representation, and graph adjacency for sample selection. Performance is evaluated using three metrics: Precision, Recall, and F-score, where the F-score is calculated as the harmonic mean of Precision and Recall. Overall, ProCon demonstrates a consistent improvement over the other baselines. For example, with a Sym-0.6 noise on the Cora dataset, the F-score of ProCon surpasses that of TopoFilter by 11%.

4.3 Ablation and Analysis

Effect of Sample Selection. We conduct ablation studies on the sample selection module of ProCon. Specifically, ProCon is compared with two variants: (1) No Selection: ProCon without the sample selection module, where the entire training dataset is used for model updating. (2) ProCon-C: Employing labels predicted by the classifier to determine label consistency distribution. Figure 5 illustrates the accuracy comparison between ProCon and two variants on Cora and WikiCS dataset. Both ProCon-C and ProCon outperform the No Selection variant. This indicates that merely enhancing node representations through the self-supervised module is insufficient for robustness, and sample selection is crucial. Additionally, ProCon outperforms ProCon-C substantially (e.g., +7.8% on WikiCS with an asymmetric noise rate of 0.4), highlighting the effectiveness of using representations for sample selection.

Effect of Self-Supervised Learning and Loss Balance Factor λ. Figure 4 illustrates the performance of ProCon across the Cora and WikiCS datasets as the loss balance factor λ is varied. We evaluated the accuracy of ProCon for λ ∈ {1e-3, 5e-3, 1e-2, 5e-2, 1e-1, 5e-1, 1}. A larger λ signifies an increased emphasis on supervised learning with noisy labels. It is noteworthy that λ = 1 excludes the selfsupervised module from ProCon. As λ increases, the per-

(b) Cora, Asym-0.4

(c) WikiCS, Sym-0.6

Figure 5: Ablation on sample selection.

formance of ProCon generally deteriorates, with marked declines observed at λ = 5e-1 and 1. This trend highlights the detrimental impact of noisy labels and accentuates the necessity of self-supervised learning. Furthermore, for λ < 1e-2, ProCon's performance remains stable, indicating that smaller λ values do not significantly affect the results, thereby simplifying hyperparameter tuning. Consequently, we fix λ at 1e-2 across all datasets and noise settings.

Effect of Weighted Prototype Updating. We validate the effectiveness of weighted prototype updating. The following variants are considered: 1) Vtr S-AVG, which applies the simple average of all training samples, equivalent to setting wj = 1, τ < 0, and V k c = V k tr in Eq. (6). 2) Vc S-AVG, which utilizes the simple average of samples in the clean node set, equivalent to setting wj = 1 in Eq. (6). 3) Vtr W-AVG, the prototype updating method of ProCon. Table 2 summarizes the results on Cora and WikiCS dataset. The Vc S-AVG variant notably outperforms the Vtr S-AVG variant (e.g. +2.7% on Cora and +2.5% on WikiCS under asymmetric noise with a noise ratio of 0.4), highlighting the effectiveness of prototype updates utilizing clean nodes. Furthermore, the Vtr WAVG variant improves upon the Vc S-AVG variant, demonstrating the benefits of weighting samples.

5 Conclusion

In this paper, we introduce ProCon, a novel framework designed to enhance the robustness of Graph Neural Networks (GNNs) against label noise. ProCon identifies mislabeled nodes by leveraging peer-informed label consistency, selecting semantically similar peers based on feature similarity and graph adjacency. We assess label consistency using pseudo-labels derived from representational prototypes and employ a Gaussian Mixture Model to distinguish clean samples from mislabeled ones. Extensive experiments across multiple datasets demonstrate that ProCon consistently outperforms state-of-the-art methods, effectively mitigating label noise and strengthening GNN robustness. Future work will explore applying this method to other areas, such as fairness, partial label learning, and graph anomaly detection.

(d) WikiCS, Asym-0.4

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants 62402315, 62232011, and 61932014, and the Shanghai Science and Technology Innovation Action Plan Grant 24BC3201200.