Dynamic Multiple High-order Correlations Fusion with Noise Filtering for Incomplete Multi-view Noisy-label Learning

1 Introduction

The prevalence of multi-view and multi-label challenges has become increasingly prominent with the rise of diverse data modalities. For instance, users on social networking platforms can be characterized through modalities such as images, text, and audio, and may be associated with multiple labels, such as 'Symphony', 'Beethoven', 'Piano', and 'Classical school'. Accurately classifying such multi-view, multilabel data is crucial for comprehensive data analysis [Qian et al., 2014; Zhu et al., 2015].

∗ Corresponding Author.

The investigation of multi-view multi-label learning is more complex than addressing either multi-view [Wang et al., 2022] or multi-label [Liu et al., 2021] data independently, as it requires the simultaneous consideration of both the correlations among different data views and the intricate dependencies across multiple labels. In light of this complexity, researchers have increasingly shifted their attention to incomplete multi-view multi-label learning, recognizing that realworld applications frequently encounter incomplete data in both its feature representations and label annotations [Tan et al., 2018; Li and Chen, 2021; Li et al., 2024; Ou et al., 2024; Liu et al., 2023; Liu et al., 2024]. Consequently, this learning framework has emerged as particularly well-suited for addressing practical, real-world challenges.

Despite significant progress in multi-view multi-label learning, a crucial issue is often overlooked: label noise. In today's data-driven world, errors in data collection, storage, and annotation are common, particularly with crowdsourced multi-view data, where human mistakes introduce incorrect labels, creating label noise. As datasets expand, the impact of label noise becomes more severe (Figure 1). This typically arises from two sources: (1) ambiguous objects leading to random mislabeling (e.g., missing 'cloud' while incorrectly labeling 'harbor') and (2) confusion between similar objects (e.g., mistaking a 'column' for a 'fence'). The co-existence of label noise and missing labels presents a greater challenge.

In response to these challenges, incomplete multi-view noisy-label learning has emerged as a novel learning paradigm designed to address the complexities of real-world, large-scale multi-view and multi-label data with prevalent incomplete views and erroneous annotations. To tackle these issues, this paper introduces a novel approach: Dynamic Multiple High-Order Correlations Fusion with Noise Filtering for Incomplete Multi-View Noisy-Label Learning (DMHNF). The proposed model addresses two pivotal challenges: (1) the structural differences and correlations across multiple views, and (2) the detection and mitigation of label noise, which significantly hampers learning performance.

On one hand, in addressing the structural correlation differences across views, most existing studies emphasize integrating features and second-order correlations (e.g., graph structures [Hamilton et al., 2017; Veliˇckovi´c et al., 2017; Defferrard et al., 2016]) between samples from different views. However, limited attention has been given to the exploration and integration of high-order correlations between samples across views. High-order correlations [Gao et al., 2020] encapsulate the distinct distributional characteristics of each view, which become particularly critical when certain views are incomplete. In such cases, the high-order correlations present in the remaining views can provide crucial information to compensate for the missing data. On the other hand, regarding the challenge of detecting and mitigating label noise, most current multi-label learning approaches focus on leveraging label correlations to impute missing labels. However, these methods often fail to utilize high-order correlations between samples to assist in correcting label noise. Our approach addresses this gap by incorporating noise correction matrices within the hypergraph network, enabling the dynamic adjustment and correction of noisy labels. By leveraging high-order correlations in the feature space, we can also refine the label space, allowing for label noise correction from a distributional perspective. These integrated strategies enable the proposed method to attain both high accuracy and robustness. The key contributions of this study are as follows:

1. We introduce the incomplete multi-view noisy-label learning paradigm, which concurrently addresses view incompleteness and label noise. This paradigm is well-suited for large-scale data, where multi-view descriptions are often incomplete and annotations frequently contain errors.
2. We propose a novel dynamic multi-hypergraph fusion network that enhances multi-view data integration by concurrently fusing high-order structural and feature information. These two types of information are exploited in parallel, mutually reinforcing one another.
3. Our method facilitates communication between the feature and label spaces via the dynamic hypergraph, leveraging high-order correlations in the feature space to correct label noise. By integrating noise filtering matrices with the hypergraph neural network, we reduce the detrimental effects of label noise from a distributional standpoint.

2 RelatedWork

Multi-view multi-label data is increasingly common in modern datasets, but as data volume grows, ensuring completeness and label accuracy remains a challenge. This has elevated incomplete multi-view weak-label learning as a key focus area in machine learning research. Tan et al. pioneered the field with their model iMVWL [Tan et al., 2018], which jointly optimizes a shared subspace for incomplete views and a weak label classifier while learning local label correlations. This approach tackles the dual challenges of missing views and incomplete labels, laying the groundwork for subsequent research. Building upon these insights, Li et al. developed NAIM3L [Li and Chen, 2021], a concise yet powerful model that addresses non-aligned incomplete multi-view and multi-label learning. It leverages multi-view consensus and uncovers both global and local label structures, aligning individual labels across diverse views to manage non-aligned data more effectively. Given the impact of deep learning in this domain, Liu et al. introduced DICNet [Liu et al., 2023], a deep instance-level contrastive network designed for doubly incomplete multi-view multi-label classification. This model employs autoencoders for end-to-end feature extraction and implements a contrastive learning strategy, combined with a multi-view weighted fusion module to boost classification accuracy. Lastly, Liu et al. developed a method SIP [Liu et al., 2024] based on semantic invariance and prototype modeling. By applying the Information Bottleneck Theory, it compresses cross-view representations to maximize shared information, enhancing the accuracy of label prediction while modeling multi-label prototypes in latent space. Despite these advances, significant complexities remain when dealing with label noise in incomplete multi-view multi-label data, highlighting the need for further innovation in this rapidly evolving area of research.

Various methodologies [Cui et al., 2020; Zhang et al., 2019; Xie and Huang, 2022] have emerged to address the intricate scenarios characterized by the coexistence of label noise and missing labels. Chen et al. introduced a generalized noisy multi-label classification method [Chen et al., 2024] utilizing a label embedding network that captures the intricate correlations between the feature and label spaces. This innovative method employs regularization on noisy predictions based on label correlations, yielding promising outcomes in scenarios marked by the presence of both label noise and missing labels. From this analysis, it is evident that the complexities associated with multi-view multi-label learning escalate significantly when both label noise and missing labels are present. Consequently, models must be adept at integrating incomplete representations across diverse views while concurrently managing label noise and missing labels with efficacy. The dynamic hypergraph network [Jiang et al., 2019; Fu et al., 2022; Zhou et al., 2023] is suitable for discovering and utilizing the complex high-order correlations in the problem proposed in this paper. In this paper, we tackle this multifaceted challenge by designing a dynamic multihypergraph neural network supplemented by label filtering matrices, leveraging high-order correlations among samples to address these critical issues effectively.

3 Proposed Approach

The dataset is denoted as D = {{X(q) ∈ Rm×dq }sq=1 , Y }, where X(q) ∈ Rm×dq is the q-th view (with feature dimension dq), Y ∈ {0, 1}m×n is the label matrix, W ∈ {0, 1}m×s indicates missing views, and m and n denote the numbers of samples and label classes, respectively. The objective is to learn a mapping function f : X → Y through which the label ˆ y of a test instance ˆx can be accurately inferred. Our method addresses both incomplete data and label noise with two key contributions (Figure 2). First, we model high-order correlations across views, treating the label space as a special view, using a dynamic multi-hypergraph neural network. This network effectively integrates these correlations to enhance learning. Second, we design two label-filtering matrices that refine noisy labels and address missing information. These matrices, tightly integrated within the hypergraph neural network, mitigate the adverse effects of label noise while leveraging high-order structural information.

3.1 Dynamic Multi-hypergraph Neural Network

Building the multi-hypergraph neural network first requires constructing the corresponding hypergraphs in both the feature space and the label space. For the feature space hypergraph, we adopt a clustering-based approach to construct hyperedges, which in turn forms the hypergraph. Specifically, for each view, an independent hypergraph is built based on the data from that view. In this setup, each sample is treated as a vertex, with corresponding hyperedges generated for each vertex [Zhou et al., 2006]. A hypergraph G = (V, E, ω) consists of vertices V , hyperedges E, and weights ω(e). It is represented by an incidence matrix H, where h(v, e) = 1 if and only if v ∈ e. Vertex and hyperedge degrees, d(v) and δ(e), are computed based on H, enabling random walks on G to model relationships as a finite Markov chain. Following the method outlined in [Zhu et al., 2017a], hyperedges in the feature space H(q) are constructed using the formula:

e(q) i = {vj | θ(xi, xj) ≥ 0.1σ(q) i }, i, j = 1, . . . ,m (1) where θ(xi, xj) represents the similarity between samples xi and xj, and σ(q) i is the average similarity between xi and all other samples. For the label space hypergraph, we construct two hypergraphs. In the first hypergraph, labels are treated as hyperedges, allowing the label matrix to be directly represented as a hypergraph incidence matrix Hl = Y . In the second hypergraph, Hc is constructed to contains semantic information of labels, hyperedges in which are constructed using the formula:

where ω(yi, yj) denotes sum of the pairwise distances between the word-to-vector representations of corresponding relevant labels in the two label sets, and σc i is the average similarity between yi and all other samples.

This neural network is primarily used to fuse information frommultiple views of the data, including both the structural correlations information among the data and the feature information of the data. So the fusion occurs on two levels: the fusion of high-order structural information and the fusion of feature information from different views. In this study, we introduce a dynamic multi-hypergraph updating mechanism for the fusion of high-order structural information. This method aims to generate a new hypergraph by fusing hypergraph sequences from prior network iterations. Notably, here we do not directly construct a new hypergraph based on the preceding hypergraph sequence. Instead, we formulate the Laplacian matrix of the new hypergraph, and subsequently, build a hypergraph neural network based on this Laplacian matrix [Feng et al., 2019]. The specific fusion process is delineated as follows.

We associate each hypergraph (H(1) , · · · ,H(s) ) [Zhou et al., 2006] with a natural random walk [Chitra and Raphael, 2019; Carletti et al., 2020]. This association allows us to fuse the transition probabilities and stationary distributions of these hypergraphs to obtain the Laplacian matrix of the fused hypergraph. Let hypergraphs H(1) and H(q) representing the initial hypergraph structure and the hypergraph generated at the q-th layer, respectively.

Let T and Π represent the transition probability and stationary distribution matrix of the hypergraph random walk, respectively. The entries of T and Π are defined as follows:

The corresponding matrices for H(1) and H(q) are denoted as T1,Π1 and Tq,Πq. To elaborate on the fusion, the multiple hypergraph cut is explained through as follows:

The parameter αi is used to determine the relative importance of each hypergraph during fusion. The cosine similarity (cos(πi, πj)) between the stationary distributions πi and πj measures the degree of similarity between two hypergraphs. Notably, greater weight is assigned to the hypergraph more similar to the original, enhancing the overall convergence of the model.

The fused transition probabilities and stationary distribution are defined as:

Notably, the formulation of transition probability and stationary distribution does not follow a simple linear combination on each hypergraph. The hypergraph Laplacian correspond to hypergraph H(q) is denoted as L(q) . The Laplacian matrix of the fused hypergraph is obtained using the equation:

Subsequently, following the approach in [Feng et al., 2019], a hyperedge convolutional layer f(X,W,Θ) is built as:

Here, X(q) represents the hypergraph signal at the q-th layer, and σ denoting the nonlinear activation function. The obtained Y (q) is instrumental for subsequent learning endeavors.

In the aforementioned steps, we have completed the fusion of high-order structural correlations information across different views based on hypergraphs. The fusion of multi-view information based on the aforementioned hypergraph requires consideration of two key aspects. First, the dynamic integration of data, where information from different views is continuously incorporated into the model, much like the previous hypergraph fusion process. Second, the challenge of missing views, which is addressed by utilizing missing-view indicator matrix to facilitate data fusion. By leveraging the hypergraph correlations and multi-view information, the model effectively mitigates the adverse effects of missing views on its performance.

Here, X(q) represents the deep features extracted through a convolutional neural network. The hypergraph convolution based on feature space information can be expressed as:

where L(q) is laplacian matrix of the dynamic fused hypergraph of (H(1) , · · · ,H(q-1) ). Θk+1 f are the learnable parameters during network training, and σ(·) is the nonlinear activation function. After the convolution operation, we obtain the image features fused by multi-view data, which capture the high-order structural information of the samples in the feature space.

3.2 Hybrid Noise Filtering

It is important to highlight that our method differs significantly from traditional incomplete multi-view multi-label approaches at both the data and model levels, with the key difference lying in the complexity of the label space. While conventional incomplete multi-view multi-label problems consider only missing labels in the label space, real-world applications commonly involve the coexistence of both missing and noisy labels (Figure 3).

Here, it is assumed that the noise in the label space is multilabel noise, meaning there exists a certain transition correlations between two labels, and this transition is independent of the sample features. However, the label space in multi-label settings is quite complex, so we aim to perform this transformation from the perspective of data distribution. These two noise correction matrices function alongside their corresponding hypergraph matrices in different label spaces, primarily to counteract the changes in distribution caused by label noise, ensuring accurate training of the classifier.

To address this, we construct two noise filtering matrices T l and T c . The purpose of these matrices is to transform feature representations to align with the label space, thereby achieving distribution alignment between the two.

Next, we will explore how the label transformation matrix T (the probability of transitioning from y to ˜ y can be expressed as p(˜ y | y)) can enhance the robustness of the learned multi-label classification model against label noise, and the method for constructing this matrix.

Assume yi∗ = 1 indicates that sample x truly possesses the i-th label, and ˜ yj = 1 indicates that sample x has been labeled with the j-th label under noise interference. The transition between the two can be achieved through the label transformation matrix T ∗ ji = p(˜ yj = 1 | yi∗ = 1), where the label transformation matrix T ∈ Rn×n is an asymmetric matrix. Assume the classifier trained by the model predicts the true label probabilities as ˆ p(yi∗ = 1 | x, θ). By utilizing the label transformation matrix T, the model is adjusted to align the predicted distribution with the label distribution of the noisy data.

The training objective of the multi-label classification model is to predict the true and accurate label set y∗ , rather than the noisy label set ˜ y. After adding T to the model, another confusion matrix ˜U will be obtained:

where Sj is the set of training samples with label yj∗ = 1. The meaning of the confusion matrix is the probability that a sample belonging to label j is misclassified as label i. Therefore, if U can be made to become the identity matrix, it implies that the base model can predict the true labels in the training data.

According to Equation (11), we can obtain ˜U = TU. It should be noted that in multi-view multi-label learning, each sample carries multiple labels, and the likelihood of the true U becoming an identity matrix is very low. Thus, we only use this idea to correct noisy labels. The model with the transformation matrix T constrains the confusion matrix ˜U to be consistent with the true distribution of noisy labels.

By constraining T = T ∗ , this will force U to converge to the identity matrix. Therefore, using the model with the added transformation matrix T for training will reduce the probability of label misclassification and enhance the model's robustness against noise. From the above analysis, it is evident that the noise transformation matrix T is the core of the noise correction module. T can be approximately computed through the following two steps:

3.3 Overall Loss Function

To train the entire network, we employ the cross-entropy loss function, defined as follows:

To ensure consistency between the feature space and the label space after multi-view information fusion, we propose the following high-order structure discrepancy loss:

Additionally, we leverage the canonical Kullback-Leibler (KL) divergence to measure the alignment between the representations processed by the two noise-correcting matrices. These matrices derived from hypergraphs constructed based on the label space, focus on different aspects of the data.

In summary, the overall loss function for the proposed model is composed of three key components:

These loss terms guide training by aligning the feature and label spaces, improving classification accuracy and robustness in noisy and incomplete multi-view data.

4 Experiments

4.1 Data sets and Experimental settings

We evaluated the proposed DMHNF on five benchmark datasets: Corel5k [Duygulu et al., 2002], Pascal07 [Everingham et al., 2010], ESPGame [Von Ahn and Dabbish, 2004], IAPRTC12 [Guillaumin et al., 2009], and Mirflickr [Huiskes and Lew, 2008], using six feature sets: GIST, HSV, DenseHue, DenseSift, RGB, and LAB. Evaluation used six metrics: ranking loss (RL), average precision (AP), Hamming loss (HL), area under the curve (AUC), OneError (OE), and Coverage (Cov). For clearer comparison, we adopted 1-RL, 1-HL, 1-OE, and 1-Cov, where higher values indicate better performance. Our approach was compared with stateof-the-art methods in multi-label learning (GLOCAL [Zhu et al., 2017b], DM2L [Ma and Chen, 2021], CDMM [Xie and Huang, 2022]) and multi-view multi-label learning (LVSL [Zhao et al., 2023], iMVWL [Tan et al., 2018], NAIM3L [Li and Chen, 2021], DICNet [Liu et al., 2023], SIP [Liu et al., 2024]). We designed two experimental scenarios using 70% of the dataset for training. In the first scenario, 50% of instances in each view were randomly deactivated, ensuring each sample retained at least one view, while 50% of both positive and negative labels were removed to simulate partial label absence. In the second scenario, label noise was introduced alongside view incompleteness by randomly adding and removing labels, simulating 20% false positives and 20% false negatives to reflect a realistic noisy label space.

4.2 Experimental Results and Analysis

Tables 1 and 2 present experimental results on datasets with missing-label and noisy-label conditions in incomplete multiview data. Table 3 shows the ablation study, while Figure 4 illustrates performance trends with varying missing-view and noisy-label rates. Based on these findings, we conclude:

1. As shown in Tables 1 and 2, our method consistently outperforms existing approaches in handling missing views and noisy labels across all benchmark datasets. In particular, when compared with state-of-the-art multi-view multi-label learning models, as well as methods specifically designed to address missing modalities or incomplete labels, our approach achieves superior performance under both missingview and label-noise conditions. These results underscore the effectiveness and robustness of our model.
2. Our method excels in scenarios where both missing labels and label noise coexist with incomplete views, outperforming methods addressing only missing labels. This emphasizes the method's robustness in complex settings, particularly when label noise accompanies missing data, showcasing its broad applicability and enhanced practical utility.
3. Analysis of Tables 1 and 2 reveals that missing and noisy labels together present a more challenging problem than missing labels alone. Despite lower proportions of miss-

ing/noisy labels in Table 2, overall performance is weaker, confirming that our method is more effective in handling the combined challenges of missing and noisy labels.

4. We conducted ablation experiments, showing that highorder correlations are essential. Replacing them with secondorder correlations weakens their interaction with filtering matrices. Without these matrices, inaccurate high-order correlations degrade accuracy more than second-order ones, highlighting their complementary roles. Using two filtering matrices outperforms one due to the label space's complexity. A hypergraph based solely on word-to-vector representations results in sparse high-order correlations. Combining both approaches enhances accuracy, improving noise correction.

5 Conclusion

This paper presents a novel dynamic multi-hypergraph neural network model specifically designed for addressing the challenges of incomplete multi-view noisy-label learning. The model integrates multi-view data by capturing high-order correlations among samples, effectively leveraging both feature and structural information across different views. To address label noise, we propose label noise filtering matrices that corrects disrupted correlations in the label space by utilizing the intrinsic high-order correlations in the feature space. Extensive experiments demonstrate that our approach not only excels in incomplete multi-view multi-label learning scenarios, but also maintains robust performance in the more challenging setting of incomplete multi-view learning with noisy labels. These findings further underscore the effectiveness, adaptability, and broad applicability of the proposed model.

Moreover, this work introduces a novel problem formulation that closely mirrors the complexities of real-world multi-label learning scenarios, thereby offering a principled and practical solution to challenges arising from incomplete views and noisy annotations. We believe these contributions lay a solid groundwork for future research, encouraging further exploration and advancement of dynamic multihypergraph methods in diverse application domains.

Acknowledgements

This work is supported by the National Key Research and Development Program of China (No. 2022YFB3305504).