Label Distribution Learning with Biased Annotations Assisted by Multi-Label Learning

1 Introduction

Multi-Label Learning (MLL) [Zhang and Zhou, 2014] has gained significant attention due to its ability to associate multiple labels with a single instance, making it widely applicable in tasks such as text classification [Liu et al., 2017] and image annotation [Jing et al., 2016].

Label Distribution Learning (LDL) 1 [Geng, 2016] extends MLL by assigning a real-valued label description degree [Jia et al., 2019] to each label, offering more detailed supervisory information. Leveraging label correlations is a key strategy in MLL [Huang and Zhou, 2012], and applying low-rank constraints in the output space is an effective approach to capturing these correlations [Zhu et al., 2017]. Building on the advancements in label correlation modeling in MLL, a branch of algorithms [Jia et al., 2018][Jia et al., 2019][Kou et al., 2023] has extended these techniques to LDL by assuming low-rank structures in label distribution spaces, aiming to capture label correlations more comprehensively.

Annotating label distributions is inherently challenging and often leads to biased label distributions, where the collected label distributions deviate from the true distributions due to variations in annotators' expertise or subjective judgments[Xie and Huang, 2018]. The existing method [He et al., 2024] [Kou et al., 2024a] [Xu and Zhou, 2017] aims to address this issue by leveraging label correlations modeling the clean label distribution and training LDL models effectively. For instance, IncomLDL [Xu and Zhou, 2017] aims to model the learned label distribution space using low-rank label correlations, thereby completing the missing entries in the label distribution matrix. And in LRS-LDL[Kou et al., 2023], the noisy label distribution is modeled as ˆD = D+E, where D is the true label distributions and E represents noise. It assumes a low-rank structure on the output space (D = WX) and sparsity of the noise. Similarly, IDI-LDL [Kou et al., 2024a] applies a low-rank assumption on the output space while employing an ℓ2,1-norm constraint on the noise.

However, recent evidence shows that the label distribution tends to be full-rank [Wang and Geng, 2021], and naive apply of low-rank approximation on biased observation leads to inaccurate recovery and performance degradation. In this paper, we address the LDL with biased annotations problem from a novel perspective, where we first degenerate the soft label distribution into a hard multi-hot label and then recover the true label information for each instance. This idea stems from an insight that assigning hard multi-hot labels is often easier than assigning a soft label distribution, and it shows stronger immunity to noise disturbances, leading to smaller label bias. As shown in Fig. 1, one images from the Real-world Affective Faces (RAF) [Li and Deng, 2019] reveals significant discrepancies between the biased and true soft label distributions ( ˆD and D). However, their corresponding multi-hot label (ˆL and L) exhibit much smaller differences. This phenomenon holds across datasets, as demonstrated by the Tables in Fig. 1. Moreover, while label distributions are inherently full-rank [Wang and Geng, 2021], multi-label spaces are widely regarded as low-rank [Zhu et al., 2017], making them more computationally efficient for correlation modeling. The contributions of this work are summarized as

1 LDL is similar to learning from soft labels, but the soft-label formulation focuses on single-label problems (i.e., there is only one true label for each instance), while LDL considers multi-label problems (i.e., each instance can have multiple true labels).

• We utilize multi-label information and label correlations to model the recovery of the true label distribution and propose the BLDL algorithm (Sections 2).
• Extensive theoretical analysis is provided, including convergence guarantees and generalization error bounds (Section 4).
• The effectiveness of the method is validated through comprehensive experiments, with superior performance demonstrated and insights verified. (Section 5).

1.1 Preliminaries

Denote X ∈ Rd×n as the feature matrix, where d is the feature dimensionality and n is the number of instances. The label space is Y = {y1, . . . , ym}, where m is the number of labels. The accurate training set for LDL is T = {(xi, di)}ni=1 , with di = [dy1 xi , · · · , dym xi ]⊤ ∈ Rm, satisfying dyxi ≥ 0 for all y ∈ Y and ∑y dyxi = 1. The label distribution matrix is D = [d1, . . . , dn] ∈ Rm×n . We assume the observed label distribution ˆD ∈ Rm×n is biased, while the true label distribution is unknown. The goal is to learn a decision function G : Rd×n → Rm×n using the training set {X, ˆD}, such that G(Xi:) ≈ Di:.

2 The BLDL Approach

A common approach in multi-label learning to capture label correlations is leveraging low-rank modeling on the output space. This can be formulated as the following optimization problem:min

where W ∈ Rm×d is the learned weight matrix mapping the feature space to the multi-label space L ∈ Rm×n . The first term captures label correlations via low-rank modeling [Jia et al., 2019].

Annotating label distributions is challenging, often introducing bias. As an extension of MLL, certain bias LDL methods leverage label correlations to recover true distributions from biased ones while learning an effective model. These methods can be formulated as:

where D ∈ Rm×n is the recovered label distribution, E ∈ Rm×n is the annotation bias, and R applies regularization. δ1 and δ2 represent the reconstruction errors in the optimization process.

However, recent evidence shows that the label distribution tends to be full-rank [Wang and Geng, 2021], and naive apply of low-rank approximation on biased observation leads to inaccurate recovery and performance degradation. In this paper, we address the above limitations of existing methods. Our method are enhance as below:

where O ∈ Rm×m models the degradation from label distribution to multi-label, and ˆL ∈ Rm×n represents the multilabel derived from biased distribution. D is recovered label distribution. Parameters α, β, and λi control the term weightings. Our method consists of three parts. The first part is label distribution recovery, where we use both the biased label distribution and its multi-label representation. Recovering the label distribution via the multi-label space is more reliable than relying directly on the biased distribution, as the discrepancy between the biased and true distributions is much larger than between their multi-label representations. By constraining the degradation of the recovered true distribution to the multi-label space (Condition 2), the reliability of the process is ensured. We also require that the difference between the recovered true distribution and the biased distribution be bounded by δ1 (Condition 1) to control the recovery of the label distribution. The second part is label distribution learning, where we impose a low-rank constraint on the multi-label space (instead of the label distribution space, since it is fullrank) to capture label correlations (the first, third, and fourth terms of Eq. (3)). The final part is the multi-label mapping process, where we learn the mapping from multi-labels to label distributions (the second and fifth terms of Eq. (3)). The algorithm flow chart is shown in Fig. 2,

3 Optimization

To solve model (3), we relax the rank by its convex alternative, nuclear norm [Gu et al., 2014], and then apply the ADMM [Boyd et al., 2011] for efficient optimization. The corresponding augmented Lagrangian function is:

where Z is a splitting variable of WXO, Λ is the Lagrange multiplier, and ρ is positive penalty parameter. The optimization is performed by iteratively updating W,O,D,Z,Λ as follows:

1) W-subproblem is formulated as:

To obtain the solution, we set the derivative of the objective with respect toW to zero, and we get:

2) O-subproblem is formulated as:

Similar to the solution for W, and we get:

3) D-subproblem is formulated as:

Similar to the solution above, and we get:

4) Z-subproblem is formulated as:

The solution has closed-form via the soft-thresholding operator [Rajwade et al., 2013]:

5) Finally, update the Lagrange Multipliers

and update ρ by µρ for some µ > 1.

4 Theoretical Analysis

We first analyze the convergence of the above algorithm in solving the proposed BLDL model.

Theorem1. All these iterative solutionsW,O,D,Z,Λgenerated by the above ADMM procedure are bounded and convergent.

We then establish the generalization error bound for the proposed BLDL framework.

Theorem 2. The generalization error of the model, defined as Egen = ED [∥WX - Dtrue∥2F ], which is bounded as follows:

where δ4 is the upper bound imposed by the optimization constraint, δ is the Restricted Isometry Property (RIP) constant, δ3 bounds the deviation between D and ˆD, and O ( rank(WXO) √n ) accounts for the complexity induced by nuclear norm regularization.

According to this theorem, it ensures that the recovered label distribution is robust to noise, consistent with biased observations, and achieves model simplicity via low-rank constraints. All the proof detail can be found in the appendix.

5 Experiments

5.1 Datasets

We evaluate our proposed method on 12 real-world datasets. The datasets cover diverse domains: Flickr, Twitter [Yang et al., 2017], and Emotion6 [Peng et al., 2015] describe emotional responses to images. Fbp5500 and SCUT-FBP focus on facial beauty perception [Ren and Geng, 2017]. RAF-ML is a text dataset for sentiment analysis [Li and Deng, 2019]. The Gene dataset analyzes relationships between genes and diseases [Yu et al., 2012]. Scene is derived from a multi-label dataset by converting label rankings into label distributions [Geng and Xia, 2014]. SJAFFE and SBU-3DFE are facial emotion datasets collected from JAFFE [Lyons et al., 1998] and BU-3DFE [Yin et al., 2006], respectively. Finally, Spo5 and Spoem are yeast datasets obtained from biological experiments [Geng, 2016].

5.2 EvaluationMetrics:

We employ six metrics [Kou et al., 2025] to evaluate the performance of all LDL methods: Chebyshev distance (Chebyshev↓), Clark distance (Clark↓), Kullback-Leibler divergence (KL↓), Canberra metric (Canberra↓), intersection similarity (Intersection↑), and cosine coefficient (Cosine↑.

5.3 Biased LDLGeneration:

We simulate biased label distributions by modeling experts' annotations as a voting process, consistent with the Central Limit Theorem. Specifically, Gaussian noise (0, Cm ) is added to the true label distributions, followed by normalization to ensure the distribution constraint holds. Here, m denotes the number of labels, and C controls the deviation degree of the biased label distributions. In our experiments, C is set to 0.1, 0.2, and 0.3 to evaluate varying levels of bias. Parameter C was set to 0.1, 0.2, and 0.3.

5.4 ComparingMethods

We compare our proposed method with seven state-of-the-art LDL methods, briefly introduced as follows:

• DN-ILDL [Kou et al., 2024a]: Handles label-dependent and instance-dependent noise by utilizing linear mappings, group sparsity, and graph regularization.
• LDL-SCL [Zheng et al., 2018]: Explores local sample correlations through the construction of a local correlation vector.
• LDLLC [Jia et al., 2018]: Captures label correlations via a distance-based mapping function.
• LRS-LDL [Kou et al., 2023]: Learns a low-rank linear mapping for ground truth and a sparse mapping for noise.
• LDLLDM [Wang and Geng, 2021]: Models both global and local label correlations by learning the underlying manifold structure of label distributions.
• EDL-LRL [Jia et al., 2019]: Utilizes local label correlations to effectively capture varying intensities of multiple emotions.
• TLRLDL [Kou et al., 2024b]: Integrates an auxiliary multi-label learning process within LDL to capture lowrank label correlations.

The hyperparameters setting. The hyperparameters for all methods were set according to their respective publications. For BLDL, the parameters α, β, γ, λ1, and λ2 were fine-tuned over the range {0.1, 0.05, 0.01, 0.005, 0.001}. The parameter η was selected from {1, 10, 50, 100, 150}, and T was fixed at 0.5. Each method was evaluated using ten-fold cross-validation to ensure robustness.

5.5 Results and Discussion

Table 2 shows the experimental results (mean±std) of various methods on eight datasets for Clark and Cosine metrics. For C = 0.1, the Friedman test [Demˇsar, 2006] rejected the null hypothesis that "all methods perform equally" (Table 3). Subsequently, the Bonferroni-Dunn test [Demˇsar, 2006] was used to compare BLDL with others, where methods differing by more than one Critical Difference (CD) are considered significantly different. CD diagrams in Fig. 3 highlight methods within one CD of BLDL connected by a thin line, confirming BLDL's significant advantage. From these results, we conclude:

• Top-1 dominance. BLDL achieves top-1 performance in 85.42% (41/48) of all configurations and consistently ranks first across all metrics by effectively addressing bias in the learning process.
• Benefit of joint modeling. Compared to methods focusing solely on label correlations (e.g., LDLLC, LDLLDM), BLDL performs better since those methods ignore the bias present in label distributions.
• Superiority over bias-only approaches. BLDL also outperforms methods that address bias alone (e.g., DNILDL, LRS-LDL) by additionally leveraging label correlations-an exploit these bias-only methods fail to capture effectively.

5.6 Ablation Study

To validate the proposed method, we design two ablated versions: (i) BLDL-a: Removes the multi-label recovery process (i.e. drops the first term in Eq. (3)). (ii)BLDL-b: Replaces the low-rank constraint onWXOwith one onWO, thus applying low-rank modeling in the label-distribution space instead of the multi-label feature space.

We compare BLDL-a and BLDL-b with the full BLDL using biased label distributions generated with C = 0.1. Fig. 4 shows results on seven datasets for Chebyshev and Intersection metrics. Wilcoxon signed-rank tests [Demˇsar, 2006] confirm the statistical significance of BLDL over both ablated versions (Table 4). From these experiments, we draw three key observations:

• Importance of multi-label recovery. Comparing BLDL-a vs. BLDL reveals a significant performance drop when the multi-label recovery step is removed, underscoring the critical role of leveraging multiple pseudo-labels to denoise and recover true distributions.
• Effect of low-rank constraint location. Comparing BLDL-b vs. BLDL shows that enforcing low-rank structure on WXO (the recovered multi-label space) outperforms constrainingWO alone, since the multi-label embedding X is intrinsically low-rank and more robust to annotation noise.
• Combined benefits. The full BLDL, which both recovers multi-label distributions and constrains the low-rank structure in the correct space, consistently achieves the best results across all metrics and datasets.

5.7 Validation of Hypothesis

We computed the difference between the recovered label distribution and the biased label distribution during the training phase, denoted as δ1, and the difference between the multilabel corresponding to the recovered label distribution and the multi-label corresponding to the biased distribution, denoted as δ2. As show in Fig. 5, it can be observed that δ1 is consistently greater than δ2 until convergence. Therefore, during the label distribution recovery phase, multi-label information is more reliable than the biased label distribution information.

5.8 Analysis of Label Distribution Recovery

We evaluate the recovery error of label distributions during training using the Frobenius norm ∥Drecover - Dtruth∥F , as shown in Fig. 6. Our method achieves lower recovery error compared to LRS-LDL, demonstrating the benefit of leveraging multi-label information. Additionally, it outperforms TLRLDL by addressing biases in the learning process, further improving recovery accuracy.

5.9 Parameter Sensitivity Analysis

We analyze the sensitivity of α, β, η, λ1, with α, β, λ1 chosen from {0.1, 0.05, 0.01, 0.005, 0.001} and η from {1, 10, 50, 100, 150}. Experiments on five datasets (Scene, SBU, Emotion, Spo, Spoem) show in Fig. 7 that BLDL exhibits stable performance, demonstrating robustness to parameter variations.

6 Conclusion

This paper introduces a novel framework, Biased Label Distribution Learning (BLDL), to address label distribution learning under biased annotations. Unlike conventional methods, BLDL first converts biased soft label distributions into multi-label representations, effectively mitigating annotation noise and bias. It further exploits the intrinsic low-rank structure of multi-label spaces to reliably recover true label distributions. Comprehensive theoretical analysis and experiments on diverse real-world datasets demonstrate that BLDL significantly enhances the accuracy and robustness of label distribution recovery, outperforming state-of-the-art methods.

Acknowledgments

This work was supported in part by the Jiangsu Science Foundation (BK20243012, BG2024036, BK20230832), the JST ASPIRE Program (JPMJAP2405), the National Natural Science Foundation of China (62125602, U24A20324, 92464301, 62306073), the China Postdoctoral Science Foundation (2022M720028), the Xplorer Prize, and the National Natural Science Foundation of China under Grant U24A20322. Additional support was provided by the Big Data Computing Center of Southeast University. Zhiqiang Kou and Hailin Wang conducted this work during their visit to the RIKEN Center for Advanced Intelligence Project, Japan.

Zhiqiang Kou, Si Qin, and Hailin Wang contributed equally to this work. Yuheng Jia and Xin Geng are the corresponding authors.