FedES: Federated Early-Stopping for Hindering Memorizing Heterogeneous Label Noise

1 Introduction

Federated learning (FL) empowers collaborative deep learning model training without the need to share sensitive data, making it particularly valuable in privacy-centric domains such as healthcare and finance [McMahan et al., 2017; Dayan et al., 2021; Rieke et al., 2020]. Many FL systems utilize datasets with associated labels, but labeling data is an expensive and resource-intensive endeavor [Song et al., 2022]. Differences in multi-source labeling skills and class distribution result in varying noise rates, known as noise heterogeneity [Fang and Ye, 2022]. For example, doctors in different hospitals may face varying cases and have differing diagnostic skills, leading to varying misdiagnosis rates [Ju et al., 2022; Bernhardt et al., 2022; Karimi et al., 2020].

* Corresponding author

Existing federated noisy label learning (FNLL) addresses noise heterogeneity by distinguishing noisy clients from clean ones. Some methods discard clients based on specific criteria, assuming that the number of clients needed for training is smaller than the total participating clients (S < N) [Nagalapatti and Narayanam, 2021; Deng et al., 2021]. However, discarding clients, even if noisy, may result in the loss of valuable information since they could contain clean data. Furthermore, the clients who are retained may not be entirely clean. Alternatively, other methods detect noisy clients and employ de-noise strategies like pseudo-labeling or knowledge distillation to them [Xu et al., 2022; Wu et al., 2023]. Nevertheless, these approaches still treat clients as either noisy or clean, potentially leading to suboptimal performance. One straightforward way to address noise heterogeneity adaptively is to tailor de-noise strategies for each client, such as data selection. However, removing the noisy samples in each client may still lose valuable information and leave residual noise [Tuor et al., 2021; Zeng et al., 2022; Zeng et al., 2023]. So far, there has been limited exploration of adaptive approaches to handling noise heterogeneity in the context of FNLL.

In the landscape of federated learning systems, the question arises: How can a de-noise strategy be adapted to clients with varying noise rates without causing information loss or noise residue? For example, early-stopping explores the dynamic optimization policies during the training of deep neural networks (DNNs) [Rolnick et al., 2017; Li et al., 2020; Nguyen et al., 2019; Tanaka et al., 2018], focusing on the memorization effect that DNNs tend to first memorize clean labels and then memorize noisy ones [Arpit et al., 2017]. Although stopping training at a certain time point has shown promising results, it remains difficult to avoid memorizing noisy labels from the beginning [Han et al., 2020].

Early-stopping on a non-critical segment of DNNs has arisen to prevent DNNs from memorizing noisy labels [Bai et al., 2021; Xia et al., 2020]. Some methods involve stopping the training of noise-sensitive layers, while other methods involve stopping the training of non-critical parameters. However, these methods all require some prior knowledge, such as the noise rate of training data. Unfortunately, in federated learning, noise rates remain unknown and exhibit variations among heterogeneous clients.

In this paper, we introduce a general noise-robust FL framework to handle clients with varying noise rates, as depicted in Figure 1. To ensure privacy, we introduce a signed distance to estimate clients' noise rates by measuring the distribution of criticality of parameters (CoP) of local models. Extensive experiments have shown a positive correlation between the amount of clean data and critical parameters [Frankle and Carbin, 2018; Xia et al., 2021], suggesting more clean data need more critical model parameters to memorize them. Hence, clients with more clean data will exhibit more large CoP values. The computation of CoP is related to the parameter and its gradient, thus no additional information needs to be requested from clients. Based on this measure, we then employ various degrees of early-stopping during local updating on clients. To enhance training stability and mitigate negative noise impact, we introduce a noise-aware global aggregation using weights based on estimated noise rates.

The primary contributions of our work can be summarized as follows:

1. We present a general noise-robust framework, FedES, to handle noise heterogeneity where clients have varying
2. noise rates instead of a binary noisy-vs-clean problem.
3. We present a general noise-generation approach for modeling federated label noise, incorporating varying noise rates for clients with a continuous spectrum.
4. We estimate each client's noise rate via a signed EMD based on the local and global gradient, without requiring additional information from clients.
5. We demonstrate that FedES outperforms state-of-the-art FL methods on both varying synthetic federated label noise and real-world label noise.

2 RelatedWork

2.1 FederatedMethods

Detecting noisy clients and then handling them separately from clean ones is a common practice for addressing the FNLL problem. For example, S-FedAvg [Nagalapatti and Narayanam, 2021] discards clients with lower Shapley-based marginal contribution. Similarly, Fair [Deng et al., 2021] discards clients with lower loss differences. However, these methods assume clients are either highly clean or noisy, leading to information loss in the 'noisy' clients and noise residue in the 'clean' clients. Without discarding clients, [Xu et al., 2022] conducts pseudo-labeling for detected noisy clients. FedNoRo [Wu et al., 2023] employs knowledge distillation for better noisy label learning on detected noisy clients. They somewhat alleviate the information loss but still, handle clients with varying noise rates in a binary way. A more general way is to conduct data selection for each client. [Tuor et al., 2021] filters out noisy training samples early before training, while [Zeng et al., 2022] iteratively removes them. However, both methods may cause information loss and leave some noise residues. Recently, an FNLL library named FedNoisy [Liang et al., 2023] has organized multiple centralized de-noise strategies that can be employed by clients locally [Zhang et al., 2017; Han et al., 2018; Wang et al., 2019]. However, such centralized strategies face limitations in estimating label noise in small client data and do not produce a noise-robust weighting scheme for global aggregation.

2.2 Criticality of Parameters (CoP)

The memorization effect reveals that DNNs tend to first memorize clean labels and then memorize noisy ones [Arpit et al., 2017]. Based on this, the parameters that contribute to optimality at an early stage are important for clean labels. Informally, CoP [Xia et al., 2021] is a metric for quantifying the criticality of model parameters, which can be computed as follows:

where wi denotes the parameter, l(wi) denotes the loss function such as cross entropy, ∇l(wi) denotes the gradient of l(wi) and m is the number of all parameters. If gi is large, wi is viewed as a critical parameter. ∇l(wi) indicates how much the output of the network would change in the parameter. However, a large gradient alone doesn't necessarily imply that the parameter is critical if the parameter itself is small. In other words, even though the gradient is large, the small value of the parameter means that it doesn't have a substantial impact on the output of a DNN. Generally, more clean labels require more critical model parameters in the early stage.

3 ProblemDefinition

3.1 Preliminaries

Let's consider a federated learning system that has N clients and a distributed dataset D = {Dn}Nn=1 . Here, Dn = {(xin , yin )}|Dn| i=1 represents the local dataset for client n, and Wn denotes the model parameters of client n. Each client has a noise rate of τn, which is defined as:

where y∗i n represents the correct label for instance xin . In this way, data quality is defined as:

which is the proportion of clean labels to total data. To estimate the noise rates and support the parameter-adaptive updating, two major foundations are used: the federated noise model and FedAvg updating process. In the following subsections, we will provide more details on these two foundations.

3.2 Federated Noise Model

We propose a federated noise model to simulate label noise in FL systems. Our model captures varying noise rates across clients, avoiding binary classification of clients as clean or noisy [Xu et al., 2022]. With a bell-like shape N(µ, σ), our model characterizes the noise rate distribution among clients. Using τ ∼ N(µ, σ) offers insights into the central tendencies and variabilities in noise rates across clients. Additionally, the independence of clients allows us to apply the central limit theorem (CLT) [Fischer, 2011], suggesting that the distribution of noise rates across independent clients is likely to approximate a normal distribution.

The generation of noise rates for N clients can be expressed as:τn = min(max(τ, 0), 1), τ ∼ N (µ, σ), (4)

where τn represents the n-th client and is clipped by the range [0, 1]. With a specified noise rate, one can easily create both symmetric and asymmetric label noise [Wang et al., 2019]. In the case of symmetric label noise, instances undergo random label flipping, with an equal probability determined by the noise rate, ensuring an even likelihood for each class. Conversely, for asymmetric label noise, instances are more inclined to receive class-conditioned labels, introducing a biased likelihood for incorrect labels, again based on the specified noise rate.

3.3 FedAvg Updating Process

FedAvg proposes a standard round of federated learning that involves local updating and global aggregation [McMahan et al., 2017]. Let W(t) be the global model (i.e., DNN) at round t and distribute it to each client. The local update process for client n is expressed as:

where η is the learning rate and ∇Ln(W(t)) represents the local gradient on W(t). The global update process can then be expressed as:

where the weight of each local model is determined by the data size |Dn|. It can be stated that the aggregation of the model parameters is equal to the weighted average of local gradients.

4 Methodology

To address clients with varying noise rates, we propose a general noise-robust framework named FedES (see Algorithm 1). Our method comprises two stages: federated noise estimation and parameter-adaptive updating. In the first stage, a global model is pre-trained by FedAvg, which is used to calculate the global and local parameter-wise CoPs. Then, the distance between global and local parameter-wise CoPs is used to estimate clients' noise rates. In the second stage, according to the noise estimation results, different degrees of early-stopping are employed during local updating. Moreover, a noise-aware global aggregation is deployed to maintain the updating stability and reduce the negative impact of noise.

4.1 Federated Noise Estimation

In Section 2.2, we discussed the positive correlation between the amount of clean data and critical parameters. To estimate

Algorithm 1: FedES: Federated Early-Stopping

Input: N (number of clients ), T (number of training rounds), D = {Dn}Nn=1 (dataset), Wpre (s1) (global model before the last pre-training round), Wpre n (s1 +1) (local model after the last pre-training round), W(1) (initialized global model in the second stage)

Output: Global W(T )

Wpre (s1 + 1) = ∑Nn=1

|Dn|

n (s1 +1)

∑k |Dk| Wpre

// Federated noise estimation

gs ← |(Wpre (s1 + 1) -Wpre (s1)) ∗ Wpre (s1)| //

global CoP

for each client n = 1 to N in parallel do

end

// Parameter-adaptive local updating

for each round t = 1, ..., T do

Wn ← W(t)

for each client n = 1, ..., N in parallel do

for each local iteration t′ = 1, 2, ...E do

Dn ← select a minibatch of size B ⊆ Dn

Wn ← Wn - ηρnMn∇L (Dn, Wn)

//Mn obtained in Eq. 15

end

end

// Noise-aware global aggregation

W(t + 1) = ∑Nn=1

end

return Wn(T )

|Dn|ρn

∑k |Dk |ρk Wn

noisy clients, we examine the distribution of CoP for each client. For a more accurate computation of CoP, we first pretrain a global model Wpre (s1 + 1) for s1 rounds by FedAvg (s1 will be discussed in Section 5.4).

Based on the global update process described in Eq. 6, the parameter-wise global CoP expressed as the product of the parameter and its gradient is given by:

where the global gradient is essentially the average of all local gradients. Similarly, based on the local update process described in Eq. 5, the local CoP is given by:

where the parameter-wise local gradient denoted by Wpre n (s1 + 1) - Wpre (s1) can be easily obtained in a federated round. The learning rate η can be omitted for clarity as it only serves as a scaling factor. This method allows us to obtain both local and global CoPs by computing parameter-wise gradients during a federated round. It eliminates the need for additional information.

Considering that higher data quality is associated with a CoP distribution having many large values, and the shape of distributions with varying noise rates differs from the global distribution, Earth Mover's Distance (EMD) is suitable for measuring the shape difference between local and global CoP [Rubner et al., 2000]. As EMD quantifies the mass required to transition between distributions, the distance concerning the CoP of a low-data-quality client may be the same as that of a high-data-quality client (a horizontally flipped version of a low-data-quality client).

To resolve this conflict, a signed EMD is required. We first gather N + 1 parameter-wise local CoPs and global CoP as G = [g1, ..., gN, gs]. Then, we use an N + 1-component Gaussian Mixture Model (GMM) to fit and predict G. We exploit the vector of means of predicted results of G, which is denoted as [µ1, ...µN,µN+1]. Thus, the signed EMD between local and global CoP can be calculated by:

where EMD(·, ·) denotes the minimum-cost transportation plan π∗ of moving the distribution mass from one to another, and the cost is a ground distance metric d(x, y). sgn(·) is the sign of the difference between µn and µN+1, which is expressed as

where [·] is the Iverson bracket.

In this way, clients with high data quality have a large proportion of critical model parameters that can be reflected in a large dn (positive). Conversely, clients with low data quality have a large proportion of non-critical model parameters that can be reflected in a small dn (negative). Based on this, the data quality can be generated by:

where the dn is scaled to the range of [0, 1] using min-max scaling. The experimental results on data quality estimation will be presented in Section 5.4.

4.2 Parameter-adaptive Updating

With the data quality ρn, different degrees of early-stopping are employed for local updating. The CoP ranking for each client is computed as follows:

where g↓n [i] is the sorted CoP for each parameter, mcn is the number of critical model parameters which is given by:

where m is the number of model parameters.

The critical and non-critical parameters are then determined as follows:

which is a parameter-wise indicator. For each client, only critical model parameters undergo gradient decay. The process of selective gradient decay (SeGD) for each client is carried out in the following manner:

Wn (t′ +1) ←Wn (t′ ) - ηρnMn ⊙∇L(Wn (t′ )) , (16) where t′ represents the current iteration, the gradient decay coefficient ρn is set to prevent overconfident descent steps. It can be seen that clients with low data quality have fewer critical model parameters and should be stopped early. Conversely, clients with high data quality have more critical model parameters and should be updated.

In the following analysis, we show how parameter-adaptive local updating based on ρn is equivalent to adapting the underlying loss function across clients in FL.

Theorem 1. Let Wn and ̂ Wn refer to two sets of model parameters that use selective gradient decay and standard one, respectively. When an update is performed at iteration t′ , two different model parameters are obtained: Wn(t′ + 1) and ̂ Wn(t′ + 1). The extent of regularization, i.e., the loss difference at the next iteration, is controlled by ρn as follows:

where ∇li represents a specific entry in ∇L(Wn(t′ )). For the sake of clarity, the proof can be found in Appendix A.

The following corollaries illustrate the extent of regularization on varying ρn:

Corollary 1. For ρn = 1, ∀i,Mn[i] = 1: all model parameters are critical and updated in a standard way, therefore L(Wn(t′ + 1)) - L(̂ Wn(t′ + 1)) = 0.

Corollary 2. For ρn = 0, ∀i,Mn[i] = 0: all model parameters are non-critical and early stopped, therefore the L(Wn(t′ + 1)) - L(̂ Wn(t′ +1)) = η∥∇L(Wn(t′ ))∥2 , which is the maximum of L(Wn(t′ + 1)) - L(̂ Wn(t′ + 1)).

Corollary 3. For ρn ∈ (0, 1): apparently, L(Wn(t′ + 1)) - L(̂ Wn(t′ + 1)) ∈ (0, η∥∇L(Wn(t′ ))∥2 ). Decreasing ρn increases the loss difference from the traditional update process, as it multiplies a larger proportion of gradients by 1 and the remaining small proportion by the increased (1-ρn). ρn controls the regularization effect that prevents the model from memorizing noise, so a small ρn exactly suggests stronger regularization.

In global aggregation, we assign different weights to models based on their noise rates. High-quality data from clients with large ρn leads to significant updates. To maintain the influence of substantial updates in the right direction while reducing the negative impact of noise, we propose noise-aware aggregation (NaAgg):

This ensures that clients contributing significantly to the update maintain their influence after global aggregation, emphasizing the importance of cleaner data in guiding the optimization direction.

5 Experimental Results

5.1 Dataset

Two groups of datasets are used for evaluation: CIFAR10/100 with synthetic label noise and Clothing1M with realworld label noise:

1. CIFAR-10/100 datasets [Krizhevsky et al., 2009]: CIFAR-10 has 60,000 32x32 color images in 10 categories, with each category having 6,000 images. CIFAR-100 has 60,000 32x32 color images in 100 categories, grouped into 20 supercategories with 5 specific categories each. Both datasets are divided into train/test sets with a ratio of 5:1.
2. Clothing1Mdataset [Xiao et al., 2015]: Clothing1Mhas 1M clothing images in 14 classes. The dataset contains real-world label noise due to it being collected from multiple online shopping websites. As commonly practiced, Clothing1M is divided into train/test sets with a ratio of 4:1.

Note that for both datasets, 20% of the test set is reserved as a benchmark dataset so that some comparison methods can use it as a reference to evaluate noisy clients.

5.2 Experimental Setup

Data Partition and Noise Generation

We consider the Dirichlet distribution [Liang et al., 2023; Li et al., 2021] and the Federated noise model described in Section 3.1 to generate the N = 20 heterogeneous-noise clients. We consider both IID and non-IID data partitions in our work. In the case of IID partitions, we randomly distribute the entire dataset D among N clients [Kairouz et al., 2021]. For non-IID partitions, we sample pk ∼ DirN(β) and assign a proportion pk,j of instances of class k to client j, where Dir(β) is the Dirichlet distribution with a concentration parameter β (default = 0.5). For evaluation under different settings of label noise, we use N ∼ (0.3, 0.2) and N ∼ (0.5, 0.2) to generate the varying noise rates of clients. As detailed in Section 3.1, both symmetric and asymmetric label noise are considered. Note that for the Clothing1M dataset, the noise is in its original form and not artificially generated.

Implementation Details

The ResNet-18 [He et al., 2016] model serves as the backbone for both the CIFAR-10/100 and Clothing1M datasets. For the CIFAR-10/100 dataset, the local epoch is 10. For the Clothing1M dataset, the local epoch is set to 5. The learning rate is 0.01 for the CIFAR-10/100 dataset and 0.001 for the Clothing1M dataset. The batch size is 128 for CIFAR-10/100 and 32 for Clothing1M. Other settings remain the same for both datasets, including 10 rounds for pre-training, a total of 200 communication rounds, and an optimizer of SGD with a weight decay of 5e-4 and momentum of 0.9.

5.3 Comparison with SOTAMethods

We have categorized state-of-the-art methods into two groups based on whether they treat clients as either noisy or clean (i.e., in a binary manner). The first group is the binary denoise methods, which includes S-FedAvg [Nagalapatti and Narayanam, 2021], Fair [Deng et al., 2021], and FedNoRo [Wu et al., 2023]. Particularly, S-FedAvg and Fair exploit a benchmark dataset to identify noisy clients, while FedNoRo exploits client-wise features. The second group is the general de-noise methods, which consist of some well-known centralized methods that can be applied to the client's local environment. These methods are organized by FedNoisy library [Liang et al., 2023] and include Fed-SCE [Wang et al., 2019], Fed-Coteaching [Han et al., 2018], and Fed-Mixup [Zhang et al., 2017].

Tables 1, 2, and 3 show the average (5 trials) and standard deviation of the best test accuracies on CIFAR-10/100 and Clothing1M. FedES achieves the best performance among comparison methods by adapting to clients with varying noise rates. Without a specific solution to noise heterogeneity, the six FNLL methods fail to fully improve performance. All comparison methods suffer from performance degradation as noise rates increase and the shift from IID to NonIID occurs. Though general de-noise strategies exhibit better performance than binary methods, which divide clients as noisy or clean, simply employing these strategies on clients is sub-optimal. Comparatively, FedES consistently outperforms other methods by a large margin.

5.4 Ablation Study

How Each Step in Federated Noise Estimation Works

CIFAR-10/100 datasets are used to conduct experiments on the first stage of FedES. The aim is to examine the impact of each step on the estimation of noisy clients. Results are summarized in Table 4. This study compares local and global distributions using two methods: Mean and Earth Mover's Distance (EMD). The former uses the means of a Gaussian Mixture Model (GMM) of all local CoPs, and the latter measures the distance between the two distributions. The Sign is applied to EMD to determine if a positive or negative sign should be used. The number of training rounds in the first stage of all methods is kept constant for fairness in comparisons. The methods' effectiveness in estimating noisy clients is evaluated using mean squared error (MSE) between the client-wise indicator ρn and actual data quality qn.

In Table 4, the first two indicators from Fair [Deng et al., 2021] and S-FedAvg [Nagalapatti and Narayanam, 2021] rely on a limited dataset, leading to poor results. Using mean for local distribution improves MSE performance, while EMD can enhance it further by measuring the distance between local and global distribution. FedES, a combination of Sign and EMD, accurately estimates data quality values of noisy clients, laying a strong foundation for parameter-adaptive updating. Compared to Fair and S-FedAvg, which only identify high and low-quality clients, FedES stands out in data quality estimation. See Figure 2 for a visual comparison.

How pre-training Affects Federated Noise Estimation In the first stage, the global model warms up for s1 rounds with the FedAvg updating process before federated noise estimation. Due to the lack of prior knowledge on multi-source label noise, it is hard to exactly determine the optimal setting of s1. To evaluate the effect of s1, federated noise estimation is conducted under different settings of s1 on the CIFAR-10 dataset (µ = 0.5, noise type: asymmetric, data partition: Non-IID) and CIFAR-100 dataset (µ = 0.3, noise type: asymmetric, data partition: Non-IID) as shown in Fig. 3. It can be seen that, after a small number of pre-training rounds, federated noise estimation performance becomes stable in a certain range. In other words, the setting of s1 would not significantly affect FedES's performance.

How SeGD and NaAgg benefit Noise-Robust FL

Table 5 shows FedES test accuracy with different de-noise schemes. When noisy clients are involved in federated learning without any de-noise schemes, there is a significant degradation in classification performance. Although client selection (CS) can increase test accuracy by discarding clients with data quality below 0.5, it still results in information loss for discarded clients and noisy labels for retained clients. A softer solution is to allow all clients to engage in and apply SeGD to perform selective gradient decay on model parameters for each client, resulting in a significant improvement in performance. NaAgg assigns updated local models corresponding criticality levels during aggregation, further improving the global model's performance. Using SeGD and NaAgg can help explore valid information in noisy clients meanwhile maintaining the criticality of model parameters and clients, considerably outperforming the classification performance when there are only high data quality clients (i.e., larger than 0.5).

6 Conclusion

In conclusion, this paper introduces a pioneering approach to federated label noise modeling, departing from the binary noisy-vs-clean clients initialization to accommodate the nuanced variations in noise rates among clients. The proposed Federated Early-Stopping (FedES) framework is specifically crafted to address the challenges posed by clients exhibiting diverse noise rates. Our experimental results, conducted on a diverse set of datasets containing both synthetic and realworld label noise, unequivocally demonstrate the superior performance of FedES compared to state-of-the-art federated noisy label learning (FNLL) methods. This empirical evidence substantiates the efficacy of our federated noise model and the FedES framework in achieving noise-robust learning in federated settings. We anticipate that our contributions will not only advance the field of federated learning but also inspire further research in the development of practical and adaptable federated learning frameworks for real-world applications.

Acknowledgments

This work is supported by the National Key Research and Development Plan of China No.2021YFC2501202, National Natural Science Foundation of China No.62202455, 61972383, Beijing Municipal Science & Technology Commission No.Z221100002722009, Beijing Natural Science Foundation No.L222100, and China Scholarship Council No.202204910370.