Reconfigurability-Aware Selection for Contrastive Active Domain Adaptation

1 Introduction

Unsupervised domain adaptation (UDA) is one of the most researched paradigms, which aims to generalize models trained on labeled source domain to unlabeled target domain, alleviating the dependence of training on a large amount of labeled data. However, the performance of UDA methods is still significantly worse than that of corresponding supervised learning methods, especially when there is a significant domain shift [Fu et al., 2021; Xie et al., 2022a; Huang et al., 2023a]. In practical scenarios, labeling a small portion of target samples is feasible. However, as shown in Figure 1, not all target samples are beneficial for further improving performance. It is not necessary to label valueless samples that are similar to the source domain.

∗Corresponding author.

Active learning (AL) aims to select the most valuable samples from a large number of unlabeled samples for annotation [Tang and Huang, 2021; Liu et al., 2021; Zhan et al., 2021; Ren et al., 2022]. However, AL assumes that unlabeled and labeled samples subject to the same distribution. Therefore, in the presence of domain shift, AL methods cannot select the most valuable target samples for annotation. Recently, a new domain adaptation paradigm called active domain adaptation (ADA) has been proposed to address the above-mentioned issue. ADA aims to select the most valuable target samples for annotation in the presence of domain shift and use these samples for training to further improve performance. However, current ADA methods [Su et al., 2020; Fu et al., 2021; Xie et al., 2022b; Han et al., 2023] rely on the original prediction probability, auxiliary modules, or multiple query functions for sample selection, and do not fully explore the semantic information of source and target domain features.

With the consideration above, in this paper, we propose the Reconfigurability-Aware Selection for Contrastive active domain adaptation (RASC). Different from previous methods, we utilize the relationship between semantic information of source and target domain features for sample selection. We not only consider the sample-level value but also design a class prediction reliability-based sample rejection strategy to jointly consider the class-level value. To better utilize the selected target samples, inspired by [He et al., 2020] and [Singh, 2021], we further design a contrastive learningbased gradual active domain adaptation framework to reduce the distribution discrepancy between labeled target domain and source domain, as well as between unlabeled target domain and labeled domains. In addition, considering that the current ADA methods require experts to label the selected target samples correctly, however, it is difficult for experts to remember and distinguish all categories when there are thousands of categories. Inspired by [Zhang et al., 2022], we also propose a variant of RASC called RASC-Ob, where experts only need to judge whether the model predictions are correct, without having to label the samples correctly.

Our main contributions can be summarized as follows:

• We propose a reconfigurability-aware sample selection strategy, which utilizes the semantic information of source and target domain features to select the most valuable target samples for annotation.
• We propose a contrastive learning-based gradual active domain adaptation framework to drastically improve the adaptation performance.
• We propose a variant of RASC called RASC-Ob that experts only need to judge whether the prediction is correct and supplements the learning of misclassified samples.
• RASC attains excellent results on multiple benchmarks. Numerous experiments and analyses demonstrate the effectiveness of RASC.

2 RelatedWork

2.1 Domain Adaptation

Domain adaptation (DA) aims to transfer the knowledge learned from the source domain to the target domain. Unsupervised domain adaptation assumes that the target domain has no labels at all. Previous methods [Long et al., 2015; Long et al., 2017; Li et al., 2020] are based on discrepancy metric minimization. Recently, adversarial learning-based methods [Ganin and Lempitsky, 2015; Long et al., 2018; Wang and Zhang, 2020; Li et al., 2021] inspired by GAN further improve the performance. Although UDA methods have achieved excellent results, the corresponding supervised learning methods still significantly outperform UDA methods. Semi-supervised domain adaptation (SSDA) [Saito et al., 2019; Jiang et al., 2020; Singh, 2021; Yan et al., 2022; Huang et al., 2023b] assumes a small number of labeled target domain samples are available and uses contrastive learning or adversarial learning to alleviate the domain shift. However, the labeled target samples in SSDA are passively provided before the training begins and not all of these samples are beneficial for training. In addition, contrastive learning-based SSDA methods [Singh, 2021; Huang et al., 2023b] use noisy target pseudo labels to construct prototypes, and do not consider the distribution discrepancy between the source and the labeled target domains.

2.2 Active Learning

Active learning (AL) aims to select the most valuable samples for annotation and these samples will be used for training to further improve performance. AL methods require defining a query function to measure the value of samples and finally select the samples with the highest value for annotation [Zhan et al., 2021; Wu et al., 2022]. The uncertainty-based methods [Wang and Shang, 2014; Yan and Huang, 2018] select the difficult-to-distinguish samples near the decision boundary for annotation. The diversity-based methods [Sener and Savarese, 2018; Sinha et al., 2019] aim to select samples to more comprehensively represent the entire dataset. However, AL assumes that unlabeled and labeled samples are subject to the same distribution, therefore, the AL methods are ineffective in the presence of domain shift.

2.3 Active Domain Adaptation

AADA [Su et al., 2020] is the first work to introduce active domain adaptation into vision tasks, utilizing the outputs of domain discriminator and the entropy to measure the domainness and uncertainty of samples, and combining with the adversarial-based DA method. TQS [Fu et al., 2021] proposes a transferable query selection strategy that combines multiple mechanisms. EADA [Xie et al., 2022a] utilizes the free energy biases across domains for sample selection, and proposes a regularization term to reduce domain shift. TLADA [Han et al., 2023] proposes a transferable loss-based sample selection strategy. DiaNA [Huang et al., 2023a] proposes a data subdivision selection strategy and designs customized learning for samples with different values.

However, these methods mostly rely on the output of domain discriminator or the original prediction probability to design selection strategies, and most of them require complex networks or multiple query functions. More importantly, they have not fully explored the semantic information of source and target domain features, and the DA strategies are also relatively rough. Thus motivated, we propose ReconfigurabilityAware Selection for Contrastive active domain adaptation (RASC), which utilizes the relationship between semantic information of source and target domain features for sample selection, and design a more refined contrastive learning-based gradual active domain adaptation framework.

3 Methodology

In this section, we first formalize ADA and then introduce the details of the proposed Reconfigurability-Aware Selection for Contrastive active domain adaptation (RASC) and its variant.

3.1 ProblemFormulation

In ADA, we can access a source domain Ds = {(xsi , ys i )}ns i=1 with ns labeled data and a target domain Dut = {xut i }nut i=1 with nut unlabeled data. Ds and Dut are sampled from P(xs , ys ) and Q(xt , yt ) with P ̸= Q, respectively. We conduct R rounds sample selection and select nlt = B · |Dut| target samples in total for annotation, where B is the labeling budget. These labeled samples will be removed from Dut and form Dlt = {(xlt i , ylt i )}nlt i=1 . For convenience, we denote Dl = Ds ∪Dlt as all labeled data across domains. Our model consists of a feature extractor F to extract feature f = F(x) and a classifier H to categorize a feature into a logit vector over all K classes, without adding any auxiliary modules. The framework of RASC is illustrated in Figure 2.

3.2 Reconfigurability-Aware Selection

To select the most valuable target samples in the presence of domain shift, we propose the reconfigurability-aware selection strategy (RAS), which utilizes the relationship between semantic information of source and target domain features. RAS not only considers the sample-level value but also jointly considers the class-level value by a class prediction reliability-based sample rejection strategy (CPR).

Reconfigurability-aware value. Existing ADA methods do not fully explore the semantic information of source and target domain features, which may lead to the selection of uninformative target samples. In contrast to the previous methods, we propose a reconfigurability-aware selection strategy that utilizes the relationship between semantic information of source and target domain features for sample selection and integrates the value of the target domain sample into a reconfigurability-aware value. First, we attain the source domain global prototypes for each class as follows:

where µs contains semantic information for each class feature in the source domain. We utilize µs and combine it with the prediction probability of xut to reconstruct the original feature of the unlabeled target sample xut :

where σ(·) is the softmax operation.

Suppose xut does not contain target domain-specific feature, i.e. xut has a high similarity to the source domain (xut is valueless). Since the model is obtained by supervised training on the source domain, it can classify xut confidently and correctly. Therefore, the feature reconstructed using the prediction probability of xut and µs can effectively express the semantic information of xut . In other words, the semantic information of the original feature is the same as that contained in the reconstructed feature ˜ f ut . Thus, the discrepancy between the prediction probabilities of the original feature and ˜ f ut is small. On the contrary, for valuable target samples, their semantic information cannot be reconstructed using the source domain prototypes, therefore the discrepancy between the prediction probabilities of the original feature and the reconstructed feature is significant.

Motivated by this, we integrate the value of the target sample into a simple value function: reconfigurability-aware value (RAV). We can obtain RAV(xut ) by using only the prediction probability for the original feature of xut : put = σ(H(F(xut ))) and the prediction probability of the reconstructed feature ˜ f ut : ˜ put = σ(H( ˜ f ut )):

For those valuable target samples, namely those containing a large amount of target domain-specific knowledge, their RAVs are relatively high, while for valueless target samples, their RAVs are relatively low. Additionally, only when the model produces consistent and completely certain predictions for the original and reconstructed features (e.g. [1,0,0] and [1,0,0]), RAV reaches the minimum value of zero. We theoretically explain the role of RAV in measuring the value of target samples in the Appendix.

Class prediction reliability-based sample rejection strategy. RAV reflects the sample-level value. To comprehensively measure the sample value from different perspectives, we propose a class prediction reliability-based sample rejection strategy (CPR) to jointly consider the class-level value. We use the average confidence of c-th class predictions to measure the c-th class prediction reliability:

where ̂ yut = argmaxput . A high CPR[c] indicates a high prediction confidence of sample belonging to the c-th class, which means that the predicted label of the c-th class is more likely to be correct. Due to our desire to select more hard samples for annotation, we hope to reject samples with pseudo label c with a high probability. Naturally, we use CPR as the rejection probability for various classes. For a candidate sample with pseudo label c, we first generate a random number ξ from a uniform distribution U(0, 1) and then reject the sample when ξ < CPR[c].

3.3 Contrastive Learning-Based Gradual Active Domain Adaptation

After sample selection, we obtain the labeled target domain Dlt. Previous methods merge Dlt and Ds as a whole and then use traditional UDA methods for adaptation, which produce suboptimal results. In ADA, sample selection and model training alternate, and a good DA strategy should reduce the discrepancy between the source and target domains while also having a positive impact on the next round of sample selection, prompting it to select more valuable target samples.

With the above consideration, we design a contrastive learning-based gradual active domain adaptation framework. Following [He et al., 2020], we use bank Bl to store source domain and labeled target domain samples, and bank Bul to store unlabeled target domain samples. And we use the momentum model F ′ (·) to encode the samples in the bank. Due to the significant discrepancy between the selected target samples and the source samples, it is suboptimal to directly consider Dlt and Ds as a whole, as in previous methods. To alleviate the discrepancy between Dlt and Ds and better learn discriminative features, we hope to pull samples belonging to the same class across Dlt and Ds closer to each other, while push away samples from different classes. To accomplish it, we design an instance contrastive loss:

where ̂ xi = Aug(xi) is the strongly augmented version of xi, P(xi) is the set of samples with the same label as xi in Bl, N (xi) = Bl \P(xi), h(q, k) = exp(Sim(F (q), F ′ (k))/τ ) is the similarity between the features of q and k extracting by F and F ′ respectively, and τ is the temperature parameter.

After aligning Dlt and Ds, the distribution discrepancy between Dut and Dl can be further reduced. Inspired by [Singh, 2021], we mitigate the discrepancy by aligning prototypes of the same class across domains. Unlike prior works, we do not use pseudo labels when calculating prototypes for each class in Dut because they contain some noise. Instead, we use prediction probability to weight and average the features, which can disperse the features of samples to different classes rather than just the classes corresponding to the pseudo labels to alleviate the impact of noisy pseudo labels. The prototype of the c-th class in Dut for each training batch is formulated as:

where nb is the size of the batch.

After computing the prototypes for the current batch, we update the global prototypes in a moving average manner:

where λ is the update trade-off parameter. Then, we calculate and update prototypes for each class in Dl in the same way and use µlc to denote the global prototype of its c-th class. We utilize prototype-based contrastive loss to align prototypes of the same class across domains to reduce the discrepancy between unlabeled target domain and labeled domains:

where Ω(q, k) = exp(Sim(q, k)/τ ).

For unlabeled target domain data, we hope that they can form good clustering, which is not only beneficial for aligning the source and target domains but also has a positive impact on sample selection. We utilize self-supervised contrastive loss to achieve this goal:

Overall, the final objective of RASC can be stated as:

where Lce is the cross entropy loss of labeled data, α and β are the parameters to trade-off different loss.

3.4 Variant: Reconfigurability-Aware Selection with One-bit Annotation

Current ADA methods require experts to label the selected samples correctly. However, when data comes from hundreds or thousands of categories, such annotation is difficult for experts. Inspired by [Zhang et al., 2022], we introduce one-bit annotation into ADA and propose a variant of RASC called RASC-Ob to reduce the workload of experts when labeling samples. Unlike the traditional annotation method, one-bit annotation only requires experts to judge whether the model prediction of the selected sample is correct, that is, experts only need to answer yes or no. From the perspective of information content, one-bit annotation is more efficient. For example, for a K-classes classification problem, labeling the sample correctly requires log2K bits of information, while asking experts to judge whether the model prediction is correct (yes or no) only requires one bit. Therefore, using one-bit annotation allows us to query more samples under the same labeling budget and effectively reduce the burden on experts.

Unlike [Zhang et al., 2022], which requires the initial labeled data used for training, our RASC-Ob does not require labeled initial data at all, meaning that all labeling budgets are used in a one-bit annotation manner. In RASC-Ob, for samples judged by experts to be correctly predicted, the utilization method is the same as in RASC. For samples judged by experts to be incorrectly predicted, we use them to form Derr. Inspired by [Zhang et al., 2022], we design the negative contrastive loss:

where Nerr(xi) contains samples with the same real label as the incorrect label of xi in Bl and all samples in But.

In addition, to keep the model prediction of incorrect samples away from known incorrect category, we also utilize negative loss [Kim et al., 2019]:

where yi is the known incorrect category of xi. Overall, the final objective of RASC-Ob can be stated as:

where α and β are the same as Eq. (10).

4 Experiments

Datasets. To evaluate the superiority of RASC, we conduct experiments on four benchmark datasets: Office-31 [Saenko et al., 2010] is a mainstream DA dataset. It contains 3 domains: Amazon (A), Webcam (W), DSLR (D) and 31 categories. Office-Home [Venkateswara et al., 2017] is a more challenging dataset. It contains 4 domains: Artistic (A), Clipart (C), Product (P), Real-World (R) and 65 categories. VisDA [Peng et al., 2017] is a large-scale dataset containing 150k synthetic images and 55k real images. It contains two domains: Syntatic (S), Real (R) and 12 categories. MiniDomainNet [Zhou et al., 2021] is a subset of DomainNet [Peng et al., 2019], consisting of four domains: Clipart (C), Painting (P), Real (R), Sketch (S) and 126 categories. It preserves the complexity of DomainNet and reduces the consumption of computing resources.

Implementation details. All experiments are implemented using the Pytorch platform. For fair comparison, like [Fu et al., 2021; Xie et al., 2022b; Huang et al., 2023a], we use ResNet-50 [He et al., 2016] pre-trained on ImageNet [Krizhevsky et al., 2017] as the backbone. For training, we use SGD optimizer with a learning rate of 0.01, momentum of 0.9, and weight decay 5e-4. We set the prototype update trade-off parameter λ to 0.9, the temperature τ in the similarity function to 0.1 and the trade-off parameters α and β to 0.5 and 0.7 respectively. The batch size is 36 for VisDA and 32 for other datasets. For sample selection, we set the labeling budget B to 5% and conduct 5 rounds of selection. More implementation details can be seen in the Appendix. Code is available at https://github.com/zeyuz22/RASC.

4.1 Comparative Results

We compare RASC with three types of methods. The first type is two baseline methods: Source Only (ResNet-50 trained with source domain only) and Random (target samples are randomly selected). The second type is several AL methods: Entropy [Wang and Shang, 2014], CoreSet [Sener and Savarese, 2018], and BADGE [Ash et al., 2020]. The third type is the recently proposed ADA methods: AADA [Su et al., 2020], CLUE [Prabhu et al., 2021], TQS [Fu et al., 2021], DBAL [de Mathelin et al., 2022], SDM-AG [Xie et al., 2022b], EADA [Xie et al., 2022a], TL-ADA [Han et al., 2023], and DiaNA [Huang et al., 2023a].

Office-31. Results on Office-31 are shown in Table 1. We can observe that the performance is saturated, but our RASC still outperforms other methods significantly in most tasks, especially in some difficult tasks (e.g. D→A and W→A). In addition, due to the small size of Office-31, one-bit annotation cannot obtain sufficient supervision information, resulting in a decrease in the performance of RASC-Ob, but it still achieves performance second only to EADA.

Office-Home. Results on Office-Home are shown in Table 2. We can observe that our method performs best on all tasks. RASC and RASC-Ob also achieve +2.3% and +2.1% improvement in average accuracy compared with the SOTA method DiaNA respectively. It demonstrates the superiority of our sample selection and domain adaptation strategies.

VisDA. As shown in the first column of Table 2, RASC performs significantly better than AL and recent ADA methods. Besides, RASC-Ob also achieves performance second only to EADA. It demonstrates the effectiveness of our method for large-scale datasets with significant domain shift.

MiniDomainNet. We also conduct experiments on a hard dataset: MiniDomainNet, and the results are shown in Table 3. Similarly, our method still achieves the best performance, with RASC and RASC-Ob achieving +4.6% and +4.9% improvement in average accuracy. It is noteworthy that the performance of RASC-Ob is superior to RASC and other methods. This is because MiniDomainNet contains a large number of images from 126 categories, and for largescale datasets with such a large number of categories, RASCOb can use one-bit annotation to obtain enough supervised information to improve performance. It demonstrates that using one-bit annotation in ADA can effectively alleviate the issue of ADA methods being difficult to apply to large-scale datasets containing a large number of categories.

4.2 Additional Analysis

Ablation studies. To demonstrate the effect of the main components of RASC and RASC-Ob, we conduct ablation studies on VisDA. The results are shown in Table 4. Specifically, when RAS is removed, it indicates that the target samples are randomly selected for annotation. From Table 4, we can observe that the performance of the full RASC and RASC-Ob significantly outperform other variants. RASC surpassing RASC w/o RAS demonstrates that our sample selection strategy can select valuable target samples for annotation, which can significantly improve adaptation performance when used for training. RASC surpassing RASC (w/o Lsup, w/o Lalign or w/o Lreg) demonstrates that our contrastive learning-based gradual active domain adaptation framework can effectively alleviate domain shift. Furthermore, RASCOb surpassing RASC-Ob (w/o Lneg or w/o Lnl) demonstrates that it is necessary to supplement the learning of misclassified samples when using one-bit annotation. More experiments and analyses can be seen in the Appendix.

Hyper-parameter sensitivity. To investigate the impact of the trade-off parameters α and β on the performance of RASC, we conduct experiments in task A→C on OfficeHome. The results are shown in Figure 3. The larger α emphasizes more on reducing the distribution discrepancy between the labeled target and source domains, while the larger β emphasizes more on reducing the distribution discrepancy between the unlabeled target domain and labeled domains. We first set β to 0.7 while varying α from 0.1 to 1.0. We can observe that when α is very small (0.1), the performance of RASC is poor because the labeled target and source domains are not well aligned. As α increases, the performance gradually improves, demonstrating that it is necessary to consider the inter-domain discrepancy of labeled data in ADA. But when α is too large (> 0.5), the performance will decrease due to the over-alignment of the labeled target and source domains. Then, we set α to 0.5 while varying β from 0.1 to 1.0, and we can observe similar patterns for β.

Varying labeling budget. We show the performance of RASC and RASC-Ob under varying labeling budgets (1% to 10%) in Figure 4. We can observe that RASC and RASC-Ob outperform other ADA methods with varying labeling budgets, demonstrating that our method is suitable for various labeling budgets. More importantly, even if the labeling budget is small (1%), our method can still select the most valuable samples. Moreover, we can observe that as the labeling budget increases, our method can continuously improve the adaptation performance, demonstrating that our sample selection strategy is stable and effective, and can continuously select the most valuable samples for annotation.

Howdoes RAS ensure diversity? While our sample selection strategy RAS mainly focuses on sample value estimation without explicitly introducing constraints on sample diversity, we identify that the class prediction reliability-based sample rejection strategy (CPR) can effectively ensure the diversity of the selected samples, as shown in Figure 5. The blue points are unlabeled target samples and the red stars are the selected samples. We can observe that samples selected with CPR are diverse and more dispersed in the feature space, while without CPR, the selected samples are confined to a small region and many of them are redundant. This is because CPR introduces randomness into the sample selection process, even if the RAVs of samples in a small area are high, these samples still have a probability of being rejected, increasing the probability of selecting samples from other areas.

5 Conclusion

In this paper, we propose a concise but effective method for active domain adaptation problem, termed ReconfigurabilityAware Selection for Contrastive active domain adaptation (RASC). Specifically, we first design a reconfigurabilityaware sample selection strategy to select the most valuable target samples in the presence of domain shift. Then, we design a contrastive learning-based gradual active domain adaptation framework to better reduce the distribution discrepancy between labeled target domain and source domain, as well as between unlabeled target domain and labeled domains. Moreover, we propose a variant of RASC called RASC-Ob, which uses one-bit annotation method and supplements the learning of misclassified samples. Extensive experiments and analyses demonstrate the superiority of RASC.

Acknowledgments

We sincerely thank the anonymous reviewers for their careful work and thoughtful suggestions, which have greatly improved this article. This work was supported by the Natural Science Research Foundation of Jilin Province of China under Grant No. 20220101106JC, the Youth Growth Technology Program of Jilin Province Science and Technology Development Plan of China under Grant No. 20240602108RC, and the Fundamental Research Funds for the Central Universities of China under Grant Nos. 2412022ZD018, 2412022Q D040 and 93K172022K10.