Learning from Long-Tailed Noisy Data with Sample Selection and Balanced Loss

1 Introduction

Deep neural networks have made great successes in machine learning applications [He et al., 2016] but require wellcurated data for training. These data, such as ImageNet [Russakovsky et al., 2015] and MS-COCO [Lin et al., 2014], are usually artificially balanced across classes with clean labels obtained by manual labeling, which is costly and timeconsuming. However, the data in real-world applications are long-tailed and noisy, since data from specific classes are difficult to acquire and labels are usually collected without expert annotations. To take WebVision dataset as an example, it exhibits long-tailed distribution, where the sample size of each class varies from 362 (Windsor tie) to 11,129 (ashcan), and contains about 20% noisy labels [Li et al., 2017]. Thus,

is a great challenge.

Many methods have been proposed for long-tailed learning or learning with noisy labels. In terms of long-tailed learning, re-sampling methods [Jeatrakul et al., 2010], reweighting methods [Cui et al., 2019; Cao et al., 2019; Menon et al., 2021], transfer learning methods [Liu et al., 2019] and two-stage methods [Kang et al., 2019; Cao et al., 2019] are included; in terms of learning with noisy labels, designing noise-robust loss functions [Ghosh et al., 2017], constructing unbiased loss terms with the transition matrix [Patrini et al., 2017], filtering clean samples based on smallloss criterion [Han et al., 2018; Li et al., 2020] and correcting the noisy labels [Tanaka et al., 2018; Yi and Wu, 2019] are included. Despite learning from long-tailed or noisy data has been well studied, these methods cannot tackle long-tailed noisy data in real-world applications. A few methods are proposed to deal with long-tailed noisy data [Karthik et al., 2021; Yi et al., 2022; Fang et al., 2023; Zhao et al., 2023], which count on observed label frequency to handle class imbalance. Nevertheless, the model bias on different classes is not merely relevant to label frequency and the true frequency is unavailable owing to label noise.

To deal with long-tailed noisy data, an intuitive way is to select clean samples with small-loss criterion and then apply long-tailed learning methods. For the sample selection process, [Gui et al., 2021] revealed that the losses of samples with different labels are incomparable and chose each class a threshold for applying the small-loss criterion. For long-tailed learning, existing methods are commonly based on label frequency to prevent head classes from dominating the training process. However, it is shown that the model bias on different classes may not be directly related to label frequency (see Appendix 1 E), and the true label frequency is also unknown under label noise. In this paper, we propose a robust method for learning from long-tailed noisy data. Specifically, we separate the noisy training data into clean labeled set and unlabeled set with class-aware sample selection and then train the model with a balanced loss based on model bias in a semi-supervised manner. Experiments on the long-tailed versions of CIFAR-10 and CIFAR-100 with synthetic noise and the long-tailed versions of mini-ImageNet-Red, Clothing1M, Food-101N, Animal-10N and WebVision with realworld noise demonstrate the superiority of our method.

2 RelatedWork

Learning from Long-Tailed Data. The purpose of longtailed learning is to alleviate the performance degradation caused by the small sample sizes of tail classes. Popular methods include re-sampling/re-weighting methods which commonly simulate a balanced training set by paying more attention to tail classes [Jeatrakul et al., 2010; Cui et al.,

1 The appendix is available at https://arxiv.org/abs/2211.10906

2019], transfer learning methods which boost the recognition performance on tail classes by utilizing the knowledge learned from head classes [Liu et al., 2019] and twostage methods which argue that decoupling the representation learning and the classifier learning leads to better performance [Kang et al., 2019]. Recently, a new line of works are proposed, which utilize the contrastive learning to learn a class-balanced feature space and further achieve better longtailed learning performance [Kang et al., 2021].

Learning from Noisy Data. The purpose of learning from noisy data is to achieve good generalization capability in the presence of noisy labels [Song et al., 2022]. Works proposed to address the problem of learning with noisy labels can be divided into three categories: 1) methods based on robust loss [Ghosh et al., 2017], 2) methods based on loss correction [Patrini et al., 2017] and 3) methods based on noise cleansing [Han et al., 2018; Tanaka et al., 2018; Yi and Wu, 2019]. According to whether remove or correct the noisy labels, the third category can be further divided into sample selection methods and label correction methods. Sample selection methods try to identify clean samples, which are commonly based on small-loss criterion [Han et al., 2018; Gui et al., 2021]. Label correction methods try to improve the quality of raw labels [Tanaka et al., 2018; Yi and Wu, 2019]. Sample selection can also be combined with label correction, e.g., DivideMix [Li et al., 2020] conducts sample selection via a two-component gaussian mixture model and then applies the semi-supervised learning technique MixMatch with label correction, NDCC [Qi and Chelmis, 2023] automatically chooses a loss value threshold for sample selection and uses counterfactual learning to correct the noisy labels.

Learning from Long-Tailed Noisy Data. Methods designed for learning from long-tailed noisy data are under-explored. [Shu et al., 2019] and [Jiang et al., 2021] learned a weighting function from training data in a meta-learning manner, which can be applied to deal with long-tailed noisy data, yet these methods require extra unbiased meta-data. [Zhang and Pfister, 2021] used a dictionary to store valuable training samples as a proxy of meta-data. These meta-learning methods attempt to learn a weighting function mapping training loss into sample weight and assign large weights to clean samples and tail samples. Nevertheless, in the learning process, clean samples usually have small losses and tail samples usually have large losses. [Cao et al., 2021] proposed a heteroskedastic adaptive regularization, which regularizes different regions of input space differently. However, they assigned the same regularization strength to samples with the same observed label without considering label noise, which made the method sensitive to noisy data. [Karthik et al., 2021] followed a twostage training approach by using self-supervised learning to train an unbiased feature extractor and fine-tuning with an appropriate loss. [Yi et al., 2022] proposed an iterative framework H2E, which first trains a noise identifier invariant to the class and context distributional change and then learns a robust classifier. These methods use the observed class distribution to handle class imbalance, while in real-world applications the class distribution is inaccessible under label noise. [Lu et al., 2023] devised the two-stage bi-dimensional sam-

Procedure 1 Class-Aware Sample Selection (CASS)

Require: Parameter of model θ, training data (X, Y).

• 1: for c = 1 to C do
• 2: Draw ˜Dc = {(xi, ˜ yi)|(xi, ˜ yi) ∈ (X, Y) ∧ ˜ yi = c} from ˜D = (X , Y)
• 3: Wc=GMM({SoftmaxCE(xi, ˜ yi; θ)|(xi, ˜ yi) ∈ ˜Dc})
• 4: Lc = {(xi, ˜ yi)|(xi, ˜ yi, wi) ∈ ( ˜Dc,Wc) ∧ wi > 1 2 }
• 5: Uc = {xi|(xi, ˜ yi, wi) ∈ ( ˜Dc,Wc) ∧ wi ≤ 1 2 }
• 6: end for

Ensure: L = ∪Ci=1Li, U = ∪Ci=1Ui.

ple selection and then trained the model in a semi-supervised manner. However, they did not consider long-tailed distribution in the semi-supervised learning process. [Fang et al., 2023] proposed a cross-augmentation matching criterion and a regularization to detect the noisy samples and eliminate their effects, then a penalization term based on confidence level was used for rebalancing, where the estimation of confidence level was still influenced by the corrupted label frequency. [Zhao et al., 2023] adopted sample selection with harmonizing factor strategy and dynamic cost-sensitive learning based on the estimated clean sample frequency of each class. Nevertheless, the estimation of clean sample frequency may not be accurate and the selected noisy samples were not used in the following process. Generally, most existing methods depend on label frequency to deal with long-tailed distribution, while the model bias on different classes is not solely related to label frequency and the observed frequency is corrupted due to label noise.

3 Methodology

Let X denote the instance space, for each x ∈ X , there exists a true label y ∈ Y = {1, . . . ,C}, where C is the number of classes. Let ˜D = {(x1, ˜ y1), . . . , (xN, ˜ yN)} denote the training data, where xi ∈ X, ˜ yi ∈ Y is the observed label of xi that may be corrupted and N is the number of training samples. In real-world applications, the data may follow a long-tailed distribution. Let ˜Dc = {(xi, ˜ yi)|(xi, ˜ yi) ∈ ˜D ∧ ˜ yi = c} and nc = | ˜Dc|. Without loss of generality, the classes are sorted by their cardinalities nc in the decreasing order, i.e., n1 > n2 > · · · > nC, and the imbalance ratio ρ = n1/nC. The deep neural network f(·; θ) : X → RC is learned from the long-tailed and noisy data ˜D, where θ represents the model parameter. Given x ∈ X, the network outputs f(x; θ) = [f1(x; θ), . . . , fC(x; θ)]T and fi(x; θ) represents the output logit of class i, we omit the model parameter θ and denote f(x; θ) as f(x) for brevity. Let pimodel (x) = exp(fi(x)) ∑Cj=1 exp(fj(x)) , the classifier induced by f is ϕf(x) = argmaxi∈{1,...,C} pimodel (x). For (x, ˜ y), the loss is calculated as ℓ(f(x), ˜ y) with a loss function ℓ(·, ·). The empirical loss of f on ˜D is 1N ∑Ni=1 ℓ(f(xi), ˜ yi). We focus on learning the optimal model parameter θ∗ which minimizes the expected loss, i.e., θ∗ = argminθ E(x,y)[ℓ(f(x; θ), y)].

The training data in real-world applications usually contain noisy labels, while deep neural networks are generally learned by minimizing the empirical risk on the training data. [Zhang et al., 2021] demonstrated that noisy labels can be easily fitted by deep neural networks, which harms the generalization of the neural networks. In order to alleviate the effect of noisy labels, we adopt the popular strategy which first warms up the model and then selects clean samples with small-loss criterion [Han et al., 2018; Li et al., 2020]. Since the training data are long-tailed in real-world applications, we introduce a regularization term Lreg in the warm up process to prevent the model from being influenced by the long-tailed distribution:

where πi = 1C . Lreg forces the model's average output on all classes to be the uniform distribution. Considering that head classes have more samples and the calculation of the model's average output may be biased, we conduct the calculation in a class-balanced manner by assigning a weight 1 nj to samples from class j. We pay more attention to tail classes by assigning a regularization strength nC ni to class i, which guarantees a decent performance on tail classes. During warming up, hyper-parameter λwarm controls the strength of Lreg:

where LCE denotes the cross-entropy loss.

After warming up, small-loss criterion can be used to select small-loss samples as clean ones following [Arpit et al., 2017]. Recently, [Gui et al., 2021] revealed that the losses of samples with different labels may not be comparable and proposed to select samples class by class accordingly. They set each class a threshold based on the noise rate. Unfortunately, each class's noise rate is unknown in practice. Instead, we adopt a two-component (g1 and g0) Gaussian Mixture Model (GMM) for each class in the sample selection process to fit the loss as that in [Arazo et al., 2019] and [Li et al., 2020], where g1 represents the clean distribution and g0 represents the noisy distribution. For class i, let L( ˜Di) denote the samples' cross-entropy loss, and it reflects how well the model fits the training samples:

A two-component GMM can be fitted with respect to L( ˜Di), and the clean probability of a sample (xj, ˜ yj ) is given by its posterior probability of belonging to g1 P(g1|L( ˜Di), (xj, ˜ yj)). The sample with clean probability P(g1|L( ˜Di), (xj, ˜ yj)) > 1/2 is selected as clean; otherwise, the sample is regarded as noisy. We determine whether a sample is clean according to the posterior probability, so we choose 1/2 as the threshold. Here, we do not need to know the proportion of clean samples which depends on the inaccessible noise rate. In this way, the clean labeled set L can be selected from the training data ˜D, and an unlabeled set

.

U = {xi|(xi, ˜ yi) ∈ ˜D\L} is constructed. This class-aware sample selection is described in Procedure 1. We further plot the loss distributions of different classes in order to demonstrate that the distributions follow the two-component GMM (see Appendix C for details).

With clean labeled samples L and unlabeled samples U, the widely-used semi-supervised learning method MixMatch [Berthelot et al., 2019] can be adopted to learn the model. MixMatch uses the learned model to generate pseudo labels for U, applies MixUp to transform L and U into L′ and U ′ with soft labels q ∈ [0, 1]C and then utilizes the cross-entropy loss and the mean squared error on L′ and U ′ respectively:

In Eq. (2) of MixMatch, training with long-tailed data leads to model bias, and the model with bias prefers to predict head classes and has a poor performance on tail classes. This motivates us to develop a loss to correct the model bias. That is, we introduce αij into L(x, q) in Eq. (2):

Specifically, for a pair of classes (i, j), if the learned model prefers class j, then a weight αij > 1 is assigned to exp(fj(x)) exp(fi(x)) for samples from class i to suppress class j, and a weight αji < 1 is assigned to exp(fi(x)) exp(fj(x)) for samples from class j to relax class i. Many methods use label frequency as the estimation of model bias in long-tailed learning [Cui et al., 2019; Menon et al., 2021]. However, the model bias on different classes is not directly related to label frequency (see Appendix E), and the true label frequency is also unknown under label noise. To characterize the model bias, we calculate the matrix M ∈ RC×C before each training epoch, where the entry Mij represents the probability that the learned model predicts a sample from class i to class j. For epoch t, M t ij = 1 |Li | ∑(x, ˜ y)∈Li pj,t model (x), where Li = {(x, ˜ y)|(x, ˜ y) ∈ L ∧ ˜ y = i}. Inspired by [Laine and Aila, 2017], we further temporally average the calculated matrices for a stable estimation of model bias with Exponentially Moving Average (EMA) and obtain the averaged model bias matrix ¯M, i.e., ¯M t = σ ¯M t-1 + (1 - σ)M t , where σ controls the contribution of the estimated matrix from each epoch. If ¯Mij is

Require: Deep neural network f(·; θ), training data (X, Y), total training epochs T, model bias estimation epochs E, unsupervised loss weight λu, regularization term weight λreg, hyper-paramter of EMA σ. 1: θ =WarmUp(X, Y, θ) ▷ training with Eq. (1) 2: Fill ¯M ∈ RC×C with zeros 3: while e < E do 4: Fill M e ∈ RC×C with zeros 5: (L, U) = CASS(X , Y, θ) ▷ apply Procedure 1 6: for iter = 1 to num_iters do 7: Draw a mini-batch LB = {(xb, ˜ yb)|b ∈ {1, ..., B}} from L 8: Draw a mini-batch UB = {ub|b ∈ {1, ..., B}} from U 9: for c = 1 to C do 10: Draw LB,c = {(x, ˜ y)|(x, ˜ y) ∈ LB ∧ ˜ y = c} from LB 11: M e c =M e c + ∑(x,˜ y)∈LB,c Softmax(f(x; θ)) ▷ update the c-th row of M e 12: end for 13: (L′ , U ′ ) = MixMatch(LB, UB) ▷ apply MixMatch with augmented LB and UB 14: LL, LU = SoftmaxCE(L′ ; θ), SoftmaxMSE(U ′ ; θ) 15: L = LL + λuLU + λregLreg 16: θ = SGD(L, θ) ▷ update the parameters of f(·; θ) with Stochastic Gradient Descent 17: end for 18: Normalize M e 19: ¯M = σ ¯M + (1 - σ)M e ▷ update the model bias matrix ¯M 20: end while 21: Rij = ¯Mij ¯Mji , R ∈ RC×C ▷ compute R 22: while e < T do 23: (L, U) = CASS(X , Y, θ) ▷ apply Procedure 1 24: for iter = 1 to num_iters do 25: Draw a mini-batch LB = {(xb, ˜ yb)|b ∈ {1, ..., B}} from L 26: Draw a mini-batch UB = {ub|b ∈ {1, ..., B}} from U 27: (L′ , U ′ ) = MixMatch(LB, UB) ▷ apply MixMatch with augmented LB and UB 28: L′L, LU = Lα(L′ ; θ, R), SoftmaxMSE(U ′ ; θ) 29: L = L′L + λuLU + λregLreg 30: θ = SGD(L, θ) ▷ update the parameters of f(·; θ) with Stochastic Gradient Descent 31: end for 32: end while

larger than ¯Mji, then the model has a bias to class j. Let Rij = ¯Mij/ ¯Mji, g(·) = γsup · I(Rij > 1)+γrel · I(Rij ≤ 1) and αij = Rg(Rij) ij , where g(·) is a function of Rij in which γsup and γrel are hyper-parameters for controlling the power of suppressing and relaxing respectively. Formally, we have

Since the training data are in long-tailed distribution and there are only a few samples in tail classes, we augment clean labeled samples in L and re-exploit Lreg as a regularization on L′ with its hard labels. Thus, we use the balanced loss L = L′L + λuLU + λregLreg in the training process, where

λu controls the unsupervised loss and λreg controls the regularization term. The whole training process of our SSBL is described in Algorithm 1.

4 Experiment

4.1 Setup

Datasets. We validate our method on seven benchmark datasets, namely CIFAR-10, CIFAR-100 [Krizhevsky et al., 2009], mini-ImageNet-Red [Jiang et al., 2020], Clothing1M [Xiao et al., 2015], Food-101N [Lee et al., 2018], Animal10N [Song et al., 2019] and WebVision [Li et al., 2017]. On CIFAR, we consider two kinds of label noise, i.e., symmetric noise and asymmetric noise. For symmetric noise, we first construct long-tailed versions of CIFAR with different imbalance ratios ρ following [Cui et al., 2019]. In specific, we reduce the sample size per class according to an exponential function ni = Oi 1ρ i-1 C-1 , where Oi is the original sample size of class i, ρ = n1 nC and i ∈ {1, . . . ,C}. Then, we generate symmetric noise in long-tailed CIFAR according to the following noise transition matrix N:

where y denotes the ground truth of x, ˜ y denotes the corrupted label of x and r ∈ [0, 1] denotes the noise rate. We set imbalance ratio to be ρ ∈ {10, 50, 100} and set noise rate to be r ∈ {0.2, 0.5}. In the experiments of learning from long-tailed noisy data, the noise rate is usually lower than 0.5, as that in [Zhang and Pfister, 2021] and [Cao et al., 2021]. For asymmetric noise, we construct step-imbalanced versions of CIFAR-10 with asymmetric noise following [Cao et al., 2021]. In specific, we corrupt semantically-similar classes by exchanging 40% of the labels between 'cat' and 'dog', and by exchanging 40% of the labels between 'truck'

and 'automobile'. Then, we remove samples from the corrupted classes. Here, the imbalance ratio ρ is the sample size ratio between the frequent (and clean) classes and the rare (and noisy) classes. Mini-ImageNet-Red, Clothing1M, Food101N, Animal-10N and WebVision are datasets with realworld noise, so we directly create their long-tailed versions following [Cui et al., 2019]. For mini-ImageNet-Red, Clothing1M, Food-101N and Animal-10N, we set imbalance ratio to be ρ ∈ {20, 50, 100}. For Clothing1M, we randomly select a small balanced subset of Clothing1M following [Yi and Wu, 2019] and construct its long-tailed versions. For WebVision, we train and test on the subset mini WebVision which contains the first 50 classes of WebVision following [Li et al., 2020]. The original imbalance ratio of mini WebVision is approximately 7, we further create long-tailed versions of mini WebVision with ρ ∈ {50, 100} following [Cui et al., 2019].

Baselines. We compare our method with state-of-the-art label noise learning methods and long-tailed label noise learning methods, including Empirical Risk Minimization (ERM) which trains the model with the cross-entropy loss, Coteaching [Han et al., 2018] which trains two networks simultaneously and updates one network on the data selected by the other with small-loss criterion, ELR+ [Liu et al., 2020] which capitalizes on early learning via regularization, DivideMix [Li et al., 2020] which conducts sample selection via a two-component gaussian mixture model and then applies the semi-supervised learning technique MixMatch with label correction, HAR-DRW [Cao et al., 2021] which regularizes different regions of input space differently to handle heteroskedasticity and imbalance issues simultaneously, MW-Net [Shu et al., 2019] which learns a weighting function in a meta-learning manner to assign each sample a weight, H2E [Yi et al., 2022] which first trains a noise identifier and then learns a robust classifier, DivideMix-LA which combines DivideMix and Logit Adjustment (LA) [Menon et al., 2021], DivideMix-DRW which combines DivideMix and Deferred Re-Weighting (DRW), and TABASCO [Lu et al., 2023] which proposes a two-stage bi-dimensional sample selection and trains the model in a semi-supervised manner. We set the hyper-parameters of the baselines by following their original papers or source codes.

Implementation Details. On CIFAR-10, CIFAR-100, miniImageNet-Red and Animal-10N, we use an 18-layer PreAct ResNet and train for 200 epochs. On Clothing1M and Food101N, we use a ResNet-50 and train for 200 epochs from scratch. On WebVision, we use an Inception-ResNet v2 and train for 100 epochs following [Li et al., 2020]. Since the baseline H2E trains for 200 epochs in its original paper, we also conduct experiments on WebVision with H2E for 200 epochs. On all datasets, γsup is set as 3 and γrel is set as 1 in L′L (refer to Appendix A for more details). The baselines ELR+, DivideMix, DivideMix-LA, DivideMix-DRW and TABASCO are based on two models. We also run our method SSBL twice with random initialization and use the ensemble of two runs for a fair comparison with them. We refer to our method with two models as SSBL2 and also report its performance in Tables 1∼5. For a fair comparison with H2E, which uses strong augmentation technique RandAugment [Cubuk et al., 2020] in training, we also report the performance of SSBL with RandAugment (SSBL+RandAug).

4.2 Evaluation on Benchmarks

Table 1 summarizes the results on long-tailed versions of CIFAR-10 and CIFAR-100 with symmetric noise, and Table 2 summarizes the results on step-imbalanced CIFAR-10 with asymmetric noise by following [Cao et al., 2021]. From these tables, it can be found that SSBL with single model outperforms ERM, Co-teaching, HAR-DRW, MW-Net and H2E under all settings, and SSBL2 with two models outperforms ELR+, DivideMix, DivideMix-LA, DivideMix-DRW and TABASCO under all settings.

Tables 3 and 4 summarize the results on long-tailed miniImageNet-Red, Clothing1M, Food-101N and Animal-10N with real-world noise. From these tables, it can be found that SSBL with single model outperforms Co-teaching, HARDRW, MW-Net and H2E under all settings, and its performance can be further boosted with RandAugment. SSBL2 with two models outperforms ELR+, DivideMix, DivideMixLA, DivideMix-DRW and TABASCO under all settings.

Table 5 summarizes the results on original and long-tailed versions of mini WebVision with real-world noise, where we report Top-1 and Top-5 test accuracy on both WebVision and ImageNet following [Li et al., 2020]. From Table 5, it can be found that SSBL with single model outperforms Coteaching, HAR-DRW, MW-Net and H2E in Top-1 test accuracy under all settings, and has the comparable or better performance in Top-5 test accuracy. The performance of SSBL with single model can be further boosted with RandAugment. SSBL2 with two models also outperforms ELR+, DivideMix, DivideMix-LA and DivideMix-DRW in Top-1 test accuracy under all settings, and has the comparable or better performance in Top-5 test accuracy.

4.3 Ablation Study

In order to study the effects of removing components of our method SSBL with single model, we conduct the ablation study on long-tailed versions of CIFAR-100 with symmetric noise and summarize the results in Table 6.

• To study the effect of rebalancing the model bias after having estimated the model bias matrix ¯M, we replace our loss L′L with the cross-entropy loss LL. The decrease in test accuracy suggests that L′L is effective for learning from long-tailed noisy data.
• To study the effect of rebalancing the model bias with the estimated model bias matrix ¯M instead of label frequency, we test the performance of rebalancing with label frequency. The degradation in performance verifies the advantage of tackling long-tailed noisy data with the estimated model bias.
• To study the effect of the regularization term Lreg, we test our method without Lreg in warm up and without

Lreg in the whole process. The results imply that Lreg in warm up usually brings better performance and Lreg in the whole process ensures the robustness under extremely long-tailed distribution.

• To study the effect of class-aware sample selection, we replace our class-aware GMM with a single GMM for all classes. The degeneration in performance justifies the superiority of class-aware sample selection over sample selection with single GMM. To further demonstrate that class-aware sample selection is robust to long-tailed noisy data, we visualize the precision and recall of samples selected as clean in Figure 1. In comparison with sample selection with single GMM, our method with class-aware sample selection achieves comparable or better performance in both precision and recall on all splits of classes.

The hyper-parameters γsup and γrel control the power of suppressing and relaxing, respectively. We conduct experiments on both of them on long-tailed versions of CIFAR-10 with symmetric noise and summarize the results in Table 7. γsup = 1 (too little suppression for head classes) or γsup = 4 (too much suppression for head classes) will degrade the performance; γrel = 0 (without any relaxation for tail classes) or γrel > 1 (too much relaxation for tail classes) will harm the performance. In general, γsup = 3 and γrel = 1 is an appropriate choice (see Appendix F for more discussion about other hyper-parameters).

In our method, we separate training data into a clean labeled set L and an unlabeled set U with class-aware sample selection, and then train the model in a semi-supervised manner with our balanced loss. There are other Class-Imbalanced Semi-Supervised Learning (CISSL) methods, e.g., DARP in [Kim et al., 2020] and ABC in [Lee et al., 2021]. We conduct the ablation study with the existing CISSL methods based on our L and U (see Appendix D). The results show that our method performs better than existing CISSL methods.

5 Conclusion

We propose the robust method for learning from long-tailed noisy data with sample selection and balanced loss. In specific, we separate the training data into the clean labeled set and the unlabeled set with class-aware sample selection, and then train the model in a semi-supervised manner with a novel balanced loss. Extensive experiments across benchmarks demonstrate that our method is superior to existing state-of-the-art methods.

Acknowledgments

This work is supported by the National Science Foundation of China (62276125, 61921006), the Fundamental Research Funds for the Central Universities (022114380013), and the Collaborative Innovation Center of Novel Software Technology and Industrialization.

Contribution Statement

Lefan Zhang and Zhang-Hao Tian have equal contribution. Wei Wang is the corresponding author.