Towards Debiased Generalized Category Discovery

1 Introduction

Deep learning has achieved remarkable success in the field of image recognition [He et al., 2016; Krizhevsky et al., 2017], typically leveraging enormous labeled training samples. However, these deep learning algorithms rely on enormous labeled training samples to achieve state-of-the-art performance, which is expensive and hinders the widespread application of existing deep learning. Semi-supervised learning (SSL) methods [Berthelot et al., 2019; Guo et al., 2024] provide an effective paradigm to alleviate the burden of labeling by leveraging a large corpus of unlabeled samples. However, one underlying assumption in SSL algorithms is that the labeled data and unlabeled data share the same label space, which is impractical in many real-world scenarios where the unlabeled data contains novel classes and old classes [Han et al., 2019; Han et al., 2020; Han et al., 2021; Fini et al., 2021]. Initially, it was studied as Novel Category Discovery (NCD) [Han et al., 2019], which depends on an disjoint assumption between labeled and unlabeled categories. By contrast, Generalized Category Discovery (GCD) [Vaze et al., 2022] relaxes the disjoint assumption between labeled and unlabeled categories, where the unlabelled data can also contain old and new categories from the labelled data. The goal of GCD [Vaze et al., 2022] is to recognize both old and novel categories from a series of unlabeled data by taking advantage of a small amount of labeled data from old categories.

* Corresponding author: Yonghong Song.

Existing GCD studies mainly includes two major paradigms: clustering-based methods and representation learning methods. Clustering-based methods [Han et al., 2021; Zhang et al., 2023a; Vaze et al., 2022] employ a semisupervised k-means [MacQueen, 1967] to recognize both old and new categories. But these works are prone to overfitting to old categories [Vaze et al., 2022]. Representation learning methods [Vaze et al., 2022; Wen et al., 2023; Chiaroni et al., 2023] exploit pre-trained self-supervised models (e.g., DINO [Caron et al., 2021]) and partially finetune their parameters on the GCD task to recognize both old and new categories.Although effective in leveraging the strong generalization properties of these representations, they may result in overfitting to new classes. In summary, the two paradigms either overly focus on old classes or excessively prioritize new classes.

In this study, we reveal a intractable challenge in GCD: competition between old and new classes (shown in Fig. 1) . Specifically, in Fig. 1, we can easily find that the performance on the old categories peaks early in training, then declines as the new category's performance improves. To diagnose this issue, we analysis confidence distribution, as shown in Fig. 2. We can observe this phenomenon, the confidence distribution of new classes is higher than that of old classes. This observation leads us to conclude the reason behind the competition is that GCD classifiers are overconfident and biased towards new categories. This over-confidence in new classes, impairing distribution of old classes and resulting in biased predictions. Notably, different from catastrophic forgetting across tasks [Tadros et al., 2022], the competition in GCD occurs within the same task. This finding raises a pivotal question: Can GCD models achieve unbiased feature learning to alleviate intra-task competitions between new and old classes?

In this work, to address these issues, we introduce a Debiased Generalized Category Discovery (DeGCD) with a debiased classifier, alleviating competition between old and new categories. To elaborate, we first design a semantic calibration loss to guide the GCD classifier in debiasing by enforcing neighborhood prediction consistency with the latent representation of the debiased head. Second, we propose a debiased contrastive objective to refine the similarity matrix from the GCD classifier and the debiased classifier, suppressing the overconfidence in new classes in unlabeled data. Finally, we introduce an alignment constraint loss to prevent impairing the distribution of the old categories caused by overconfidence in the new categories.

Our contributions are summarized as follows:

• To the best of our knowledge, this is the first work to reveal competition issue in generalised category discovery. We propose a novel DeGCD approach with a debiased classifier to mitigate competition between old and new categories.
• We propose a semantic calibration objective to facilitate debiasing in the GCD classifier by promoting neighbor-
• hood prediction consistency with the latent representation of the debiased head.
• We devise a debiased contrastive objective to refine the similarity matrix from the GCD classifier and debiased head, mitigating overconfidence in new classes within unlabeled data.
• We introduce an alignment constraint loss to prevent damaging the distribution of the old categories caused by overconfidence in the new categories.

2 RelatedWorks

Open-Set Semi-Supervised Learning addresses scenarios where unlabeled data may include outliers that do not belong to any of the labeled categories in the training set. The objective is to train a classifier that accurately recognizes the labeled categories despite the presence of noisy, unlabeled data [Han et al., 2021; Han et al., 2019; Chen et al., 2020b; Guo et al., 2020]. Since the primary focus is on the classification performance of the labeled categories, outliers from novel categories are merely identified and rejected, without requiring further classification.

Novel Category Discovery (NCD) [Troisemaine et al., 2023] was initially formalized as deep transfer clustering[Han et al., 2019], aiming to identify unlabeled new classes using knowledge from labeled ones. Han et al.[Han et al., 2019] employed self-supervision for representation learning and utilized ranking statistics for knowledge transfer. Zhong et al.[Zhong et al., 2021] introduced OpenMix, which applies MixUp [Zhang et al., 2018] to blend old and new classes, mitigating overfitting. UNO [Fini et al., 2021] provides a unified objective that jointly optimizes old and new class predictions via swapped prediction [Caron et al., 2020]. NCD relies on assumption that all unlabeled data belong to novel classes.

Generalized Category Discovery (GCD) [Vaze et al., 2022; Cao et al., 2022] is designed to generalize NCD for openworld scenarios where unlabeled instances belong to old and new categories. GCD aims to simultaneously cluster both old and new classes in unlabeled data. Early studies[Vaze et al., 2022; Zhang et al., 2023a] integrate supervised [Khosla et al., 2020] and unsupervised contrastive learning [Chen et al., 2020a], leveraging semi-supervised K-means [MacQueen, 1967] for clustering. Subsequent works [Pu et al., 2023; Zhang et al., 2023b; Zhao et al., 2023a] enhance feature representations by capturing underlying relationships. Recently, some efforts to improve the performance in new categories [Wen et al., 2023; Hao et al., 2024]. Among them, SimGCD [Wen et al., 2023] rethinks the failure of GCD classifiers and proposes a simple method to achieve promising results in new classes. CiPR [Hao et al., 2024] effectively improves the performance of novel classes with new categories by leveraging cross-instance relations and selective clustering. Although GCD has made great advancements [Vaze et al., 2023; Gu et al., 2023], it inherently suffers from competition between old and new classes, which is intractable due to the over-reliance on new class feature on unlabeled data. In this paper, we propose DeGCD to address this challenge.

Entropy Calibration is a crucial advancement in uncertainty quantification, addressing overconfidence issues and improving model interpretability. Recent studies [Zhang et al., 2022; Braverman et al., 2020; Liu et al., 2024] have demonstrated its effectiveness across various applications, from supervised classification to large-scale language modeling. As machine learning models become more integral to high-stakes decision-making, refining entropy calibration techniques will be essential for ensuring reliability and robustness. However, existing methods assume identical label spaces across domains and are unsuitable for GCD, where competition between old and novel categories exists.

3 Preliminaries and Analysis

Here, we briefly introduce the setting and methods of Generalized Category Discovery (GCD) (Sec. 3.1) and give empirical results to reveal inherent issues (Sec. 3.2), which motivates us to propose our DeGCD in Sec. 4.

3.1 Setup and TrainingMethods of GCD

Problemdefinition of GCD. Given a labeled data set Dl = {(xli , yl i )}ni=1 consisting of n samples with labels belonging to C l classes, C l is the set of old classes. An unlabeled data set Du = {xui }mi=1 consisting m unlabeled samples, each of which belongs to one of the classes in C u . C u is the set of classes in unlabeled training set. Generally, m ≫ n. Here, C l ⊂ C u , then the set of new classes is C n = C u /C l . The classes included in the testing set are denoted as C u . Feature extractor F(·) is used to extract the ℓ2-normalized feature by hi = F(xi). Then, a parameter classifier Φ is utilized to obtain class prediction by zi = Φ(hi). The debiased head is composed of two key components: a multi-layer perceptron (MLP) that transforms the feature representation hi into a new projection space ri, an auxiliary classifier Ψ that produces predictions based on the projection representation ri.

Related Training Methods. Generalized category Discovery (GCD) [Vaze et al., 2022] introduces to integrate both supervised [Khosla et al., 2020] and self-supervised [Chen et al., 2020a] contrastive learning paradigms, applying them to the labeled subset Bl and the mini-batch B respectively.

where N(i) denotes the set of same-class samples in the batch, τu and τs are temperature hyperparameters. The overall contrastive loss Lcon = (1 - λ)Lucon + λLlcon .

SimGCD [Wen et al., 2023] proposes a parametric prototype-based classification framework, denoted as C = {c1, · · · , cK}, where K = Kold +Knew represents the total number of old and novel classes. The posterior probability distribution is formulated as:

In SimGCD, self-distillation is applied to two views (random augmentations), alongside entropy regularization H(·) for each sample:

Here, q′i is a sharpened probability of another view, and p = 1 2|B| ∑i∈B (pi +p′i ), ℓ(·) represents cross-entropy loss. The supervised loss is also employed on Dl with labels yi:

3.2 Intractable Problems in GCD

We adopt the state-of-the-art SimGCD [Wen et al., 2023] for confidence [Guo et al., 2017] and accuracy analysis, training under practical low-label conditions to investigate real-world GCD challenges.

GCD suffers from severe competition between old and new classes. As shown in Fig. 1, the accuracy of new categories increases while that of old categories decreases, mainly due to the overconfidence in new classes (Fig. 2), which is a key issue for parameterized GCD classifiers and motivates our proposed strategies.

4 Method Overview

We provide the framework of DeGCD in Fig. 3, which has three main components: 1) A semantic calibration loss aids the GCD classifier's debiasing, aligning the neighborhood prediction of the GCD unbiased classifier with the latent representation of the GCD classifier. 2) A debiased contrastive objective is designed to optimize the similarity matrix between the GCD classifier and the debiased classifier, suppressing the influence of noise for new and old classes within unlabeled data. 3) Alignment constraint loss, which is introduced to prevent damaging the distribution of the old categories caused by overconfidence in the new categories.

4.1 Semantic Calibration Objective

In the GCD task, unlabeled data usually contains a large number of noisy samples [Li et al., 2020], which can easily lead to conflicts between the old and new categories in the shared feature space, and thus cause the GCD classifier to generate erroneous pseudo-labels. To mitigate the effects of bias in the parameters classifier on GCD, we introduce a semantic calibration objective that fosters neighborhood-level predictive coherence by aligning with the underlying representations of the GCD debiased head.

Firstly, we use shared feature extractor F(·) to derive the ℓ2-normalized feature by hi = F(xi) as initial representation memory bank M, xi ∈ Dl ∪ Du. Meanwhile, a projection head Φ is utilized to obtain class predicted probability memory bank P by pi = exp(Φ(hi)) ∑m+n j=1 exp(Φ(hj)) . The memory bank M and P are updated in batches, using a momentum update mechanism defined as:.

where ϵ, µ ∈ [0, 1) are the momentum coefficient controlling the update rate. The hb and pb denote the newly obtained feature representations and predicted probability of the current batch, respectively. Then, to avoid conflicts between the old and new categories in the existing GCD classifier, we design a semantic calibration objective between GCD classifier and debiased classifier. Specially, we first compute similarity distance db between representations rb from the current batch in debiased head and representation memory bank.

where τ is the temperature scaling factor, db ∈ RB×N , where B is the batch size and N is the size of the memory bank. Subsequently, for each feature ri ∈ rb. We retrieve its knearest neighbors in the representation memory bank M and express Ti as the index set of the gathered neighbors:

Following the clustering assumption [Jiang et al., 2022], samples that exhibit locally consistent predictions tend to stay away from decision boundaries, thus they are more likely falling into correct clusters. We employ T j i as the index set to select neighborhood-level class predicted probability:

Here, T j i represents the index of the j-th neighbor of xi. The semantic calibration objective is designed to minimize the difference between the current predicted probability pi and its nearest neighbors Si.

By minimizing KL divergence, the semantic calibration objective Lsa enforces prediction consistency among nearest neighbors, promoting coherent predictions and improving bias mitigation in parameterized classifiers. DeGCD further maintains consistency by continuously updating the representation and probability memory banks during training.

4.2 Debiased Contrastive Objective

Current representation learning methods employing traditional contrastive learning (Eq. 2) aggregate samples solely based on similarity, which misguides unlabeled noise samples into incorrect clusters, thereby amplifying noise-induced uncertainty. To address this issue, we devise a debiased contrastive objective to refine the similarity matrix from the GCD head and debiased head, effectively suppressing the influence of noise for new and old classes within unlabeled data. To be more specific, to adaptability refine reliability and comprehensiveness of the predicted logits, we combine GCD predicted ligits P g i = Φ(hi) and debiased predicted logits P d i = Ψ(ri) using Gaussian distributions[Stacy, 1962] to mix the controlled formulas.

where the mixing factor α is sampled from a Gamma distribution parameterized by the β hyper-parameter. P gd i is blended predicted logits. Sampling the mixing factor α from a Gamma distribution provides flexibility, robust control over variability and sparsity, and improved regularization, enhancing stability and generalization in probabilistic. Then, we propose the debiased contrastive objective using blended prediction logits to suppress noise effects on old and new classes in unlabeled data. Specifically, we first calculate the softmaxnormalized similarity as.

Here, η is a temperature parameter, ygd i is the one-hot groundtruth similarity (with logits of 0 for negative pairs and 1 for the positive pair), and the debiased contrastive loss is defined as the cross-entropy between ˆ pgd i and ydc i .

In this way, we pull the instance representation closer to the representation of its predicted class, and push it away from the representations of other classes, using the predicted probabilities and their corresponding pseudo-labels.

4.3 Alignment Constraint Loss for Prediction Refining

In the context of GCD, the learning process on unlabeled data excessively focuses on new categories, leading to insufficient constraints on the distribution of old categories, thereby degrading or conflicting with the discriminative ability of old categories. To prevent damaging the distribution of the old categories caused by overconfidence in the new categories, we design an alignment constraint loss for prediction refining. Specifically, based on the generated predicted logits P d i on debiased classifier in Section 4.2, we employ a temperature η and max to P d i to obtain a refined pseudolabels P di = max( P d i η ). Finally, we introduce our alignment constraint loss, which applies confidence filtering to mitigate the negative impact of unreliable pseudo-labels. Specifically, only predictions with confidence scores above a threshold η contribute to the loss. Formally, we introduce the alignment constraint function as:

In this way, we can ensure that learning new classes does not excessively disrupt the distribution of old classes.

4.4 Joint Optimization

The framework of DeGCD is elaborated in Fig. 3. The overall objective of our model is defined as:

where ω and ψ are the weights used to balance the strengths of the three loss functions. More details about ω and ψ in Fig. 4. This work proposes a general debiased method that can be adapted to existing GCD classifier models. For these models, the overall optimization objective is formulated as a combination of the original model's loss function and the loss functions Lt introduced by the proposed method. This optimization strategy effectively balances the learning of new and old class features to alleviate competitions between them.

5 Experiments

5.1 Experimental Setup

Datasets. We validate the effectiveness of the proposed DeGCD on the generic benchmarks (including CIFAR10/100 [Krizhevsky et al., 2009] and ImageNet-100 [Deng et al., 2009]), the recently proposed Semantic Shift Benchmark (SSB, including CUB [Wah et al., 2011], Stanford Cars [Krause et al., 2013] and the harder Herbarium 19 [Tan et al., 2019]. For each dataset, following [Vaze et al., 2022], we sample a subset of all classes as the labeled ("Old") classes Yl. Half of the images from these classes form Dl, while the rest are treated as unlabeled data Du. Table 1 shows the statistics of the datasets we evaluate on.

Evaluation protocol. We evaluate model performance using clustering accuracy (ACC), following standard practice [Vaze et al., 2022]. Given ground truth labels y∗ and predicted labels ˆ y, ACC is computed as ACC = 1N ∑Ni=1 1(y∗ i = p(ˆ yi)) where N = |Du|, and p denotes the optimal permutation aligning predicted clusters with ground truth labels.

Implementation details. Following standard practice in GCD [Vaze et al., 2022], we adopt ViT-B/16 [Dosovitskiy et al., 2021] pre-trained by DINO [Caron et al., 2021] as the backbone, and fine-tune only the last transformer block for all experiments. The output of [CLS] token (768-dimensional) serves as the feature representation. We train for 200 epochs using a batch size of 256 and an initial learning rate of 0.1, which follows a cosine decay schedule for each dataset. In this study, the balancing factor ψ is set to 0.35, while the parameter ω is set to 0.2. Following [Wen et al., 2023], the temperature values η is 0.1. η is set to 0.95 follows [Sohn et al., 2020]. Meanwhile, according to [Wen et al., 2023], τs is fixed at 0.1, and τt is initialized at 0.07, then gradually reduced to 0.04 using a cosine schedule over the first 30 epochs. In this work, we set the Gamma parameter β to 0.5. Following [He et al., 2020], ϵ, µ = 0.99.

5.2 Comparative Results

To assess the performance of DeGCD, we combine DeGCD with three strong GCD classification methods, including SimGCD [Wen et al., 2023], and CiPR [Hao et al., 2024]. We also compare our models with competitive baselines, i.e., RankStats [Han et al., 2021], UNO [Fini et al., 2021], GCD [Vaze et al., 2022], GPC[Zhao et al., 2023b], ORCA [Cao et al., 2022] and NGCN [Yang et al., 2025].

Comparison on Fine-Grained Datasets. Fine-grained datasets exhibit subtle inter-class differences, making finegrained visual understanding particularly challenging for GCD. We evaluate the effectiveness of DeGCD by comparing it with existing methods on fine-grained image recognition datasets. As shown in Table 2, DeGCD consistently outperforms all baselines across "All" "Old" and "New" classes. Specifically, for "All" classes, it improves the state-of-theart performance by 8.3%, 2.8%, and 6.7% on CUB, Stanford Cars, and Herbarium 19, respectively. Moreover, DeGCD achieves notable gains in both "New" and "Old" categories, outperforming the SOTA by 8.1%, 4.5%, and 2.2% on CUB, Stanford Cars, and Herbarium 19 ("Old" category), respectively, demonstrating its effectiveness in capturing unbiased information and reducing competition between categories.

Comparison on Generic Datasets. Table 3 presents a comprehensive comparison, showing that our DeGCD consistently outperforms existing methods on generic image recognition benchmarks. Compared to the prior state-ofthe-art (SOTA) approaches, SimGCD or CiPR, DeGCD achieves performance gains of 3.9% on CIFAR-100, 2.2% on ImageNet-100 and 0.3% on CIFAR-10 when considering all categories. The advantage is even more pronounced for "New" classes, with improvements of 4.0%, 3.3%, and 0.2%, respectively. These findings show that the debiased head improves prediction consistency and bias mitigation, leading to more balanced confidence and reduced competition between old and new classes (see Fig. 6 and Fig. 7).

Visualization of the Representation Space. Fig. 5 shows latent representation space of different methods on the CIFAR10. Fig. 5-(a)-(b) reveals some representations of "New" classes are closer to "Old" classes in SimGCD and CiPR. Fig. 5-(c) demonstrates that the debiased head mitigates the disruption of old class distributions, leading to better separation.

5.3 Ablation Studies

In this section, we conduct comprehensive ablation experiments on CUB and CIFAR-100, as shown in Table 4.

The effect of semantic calibration objective. To explore the benefits of semantic calibration objective, we conduct ablations in Table 4. From the results in Table 4, we can observe that semantic calibration objective has a clear contribution. Specifically, compared with the previous works SimGCD and CiPR, our method obtains consistent improvements in both "Old" and "New" classes (+1.8% and +4.9% in "Old" and "New" on CUB), indicating the importance of semantic calibration. Our approach consistently exceeds baseline on Cifar100. This improvement proves that it is critical to enhances the consistency between neighborhood predictions and unbiased latent representations for fine-grained classes.

The effect of debiased contrastive objective. As shown in Table 4, we can see that debiased contrastive objective can improve performance. Specifically, compared to baseline, debiased contrastive objective improves accuracy by 1.8% and 1.1% on CUB and CIFAR-100 in all classes, respectively. This improvement highlights the ability of debiased contrastive objective to mitigate the effects of overconfidence in "New" classes within unlabeled data.

The effect of alignment constraint loss. We conduct ablations on two datasets, as shown in Table 4. Specifically, the alignment constraint loss improves accuracy by 1.2% and 1.1% on CUB and CIFAR-100 in all classes, respectively. It can be seen that the alignment constraint loss can prevent damaging the distribution of the "Old" categories caused by overconfidence in the "New" categories.

5.4 Further Analysis

Different Mixed methods in Eq. 6. Our cross-batch mixing mechanism is grounded in momentum-based stability (MoCo[He et al., 2020]) and Mixup[Zhang et al., 2018], operating across batches (vs. Mixup's intra-batch mixing) to enhance the stability of prediction results. The effectiveness of this design is quantitatively validated in Tab. 5.

Analysis of Hyperparameters. We analyze the effect of the Gamma parameter β (Eq. 11) on classification accuracy across datasets (Table 4). On Stanford Cars, "New" accuracy first rises, then slightly drops but stays above SimGCD at 45.5%. On CIFAR-100, it decreases but remains higher than SimGCD (77.9%). To balance maximizing "Old" accuracy while maintaining or surpassing SimGCD on "New", we set β = 0.5 for both datasets.

We further examine the impact of ω and ψ (Eq. 15) on CIFAR-100 and Stanford Cars (Fig. 4). Optimal results are achieved at ω = 0.35 (Fig. 4-a) and ψ = 0.2 (Fig. 4-b).

Evaluation with the number of new class. We also consider the real scenarios with class number |C| cannot be accessed in advance. We evaluate our DeGCD an off-the-shelf number estimation algorithm [Vaze et al., 2022] to get an estimation of C in advance, and use it to construct classifiers. As shown in Table 7, DeGCD consistently outperforms SimGCD on both datasets under estimated class numbers.

6 Conclusions and Limitations

This study addresses the key challenge of competition between new and old classes in GCD caused by classifier overconfidence. This work proposes a DeGCD, which mitigates bias by introducing a semantic calibration loss to ensure neighborhood prediction consistency and an alignment constraint loss to prevent the distribution of old categories from being affected by overconfidence in new categories. Experiments confirm SOTA performance on various GCD benchmarks, demonstrating robust handling of both new and existing classes. Additionally, it can be easily integrated with other GCD methods to further enhance performance and maintain balance between new and old classes. There are two main limitations. First, while computational overhead is manageable, further acceleration is needed. Second, generalization under extreme data imbalance requires further study.

Acknowledgments

This research is supported in part by National Natural Science Foundation of China (Grant No.: 61973245, 61771199).