Navigating Continual Test-time Adaptation with Symbiosis Knowledge

1 Introduction

Deep neural networks have achieved remarkable success in visual tasks when training and testing data obey the same distribution. Such networks, however, suffer from the generalization problem due to the ubiquitous domain shift [Wang et al., 2023b]. For example, a classification network pretrained in the normal, natural images may not recognize the corrupted images. Thus, domain adaptation is essential to transfer knowledge from the source domain to the target one by reducing the shift. However, the target domain labels are usually unavailable, and the problem is primarily explored at Unsupervised Domain Adaptation (UDA) [Li et al., 2020; Wang et al., 2023c; Yang et al., 2022]. More realistically, the source data is often inaccessible during test time due to privacy or business problems, making the adaptation problem more challenging. Initial approaches attempt to employ the source model and unlabeled target data for testing, such as Source-Free and Test-Time domain Adaptation (TTA) [Chen et al., 2022; Yang et al., 2021; Liu et al., 2021].

∗Corresponding Author

Common techniques in Test-Time Adaptation (TTA) typically address the challenge of domain shift by updating adapted model parameters using either generated pseudolabels or entropy regularization [Yang et al., 2023]. While effective for static target distributions, these approaches exhibit instability when the target domain's distribution is in a continual state of flux [Wang et al., 2022; Prabhu et al., 2021]. The presence of noisy pseudo-labels, stemming from the constantly shifting distribution, significantly hampers the adaptation process [Wang et al., 2022; Prabhu et al., 2021]. To address this issue, CoTTA [Wang et al., 2022] introduces the concept of Continual Test-Time Domain Adaptation, wherein a source pre-trained model must adapt to an evolving stream of target domains without recourse to source data. CoTTA leverages a weight-average teacher network to enhance the quality of generated pseudo-labels. Additionally, Robust Mean Teacher [D¨obler et al., 2023] employs a multi-viewed contrastive loss to guide test features back towards the initial source space and learns invariant features concerning the input space. Nevertheless, recent studies [Marsden et al., 2023] have demonstrated that tuning network parameters based solely on the current domain may result in a loss of generalization and impair performance on subsequent domains. Some strategies [Gong et al., ; Marsden et al., 2023] advocate for updating only the network's normalization parameters while freezing all others, which can mitigate the rapid loss of generalization. However, this approach may lead to a lack of discriminative power in certain domains due to the constrained learning parameters.

The aforementioned methods have motivated our primary research objective: enhancing network discrimination within the current domain while preserving generalization for subsequent domains. Our focus lies in developing a dual-stream architecture leveraging distinct optimization parameters to encapsulate synergistic knowledge of generalization and discrimination. One stream of this dualstream network exclusively adjusts normalization parameters to maintain generalization prowess, while the other stream harnesses all learnable parameters to enhance discriminative capacity. Furthermore, to mitigate the risk of catastrophic forgetting, we propose continual integration of source knowledge into each stream using varied strategies to fine-tune the adapted dual-stream model. Specifically, a weight-average strategy is employed in both the source and adapted models for the normalization-tuned stream. To achieve this, we introduce a weighted soft parameter alignment mechanism that encourages similarity between the adapted network and the source. Importantly, the absence of supervisory signals in generated pseudo-labels inevitably introduces noise. We propose a novel weighted guidance approach, inspired by the observation that latter layers in a network are more susceptible to label noise while former layers exhibit greater robustness [Bai et al., 2021]. These weights modulate the similarity between the adapted model and the source model across layer depths, enabling noise-resistant former layers to undergo more adjustment and noise-sensitive latter layers to undergo less adjustment.

The second objective of this paper is to uncover prior knowledge aimed at enhancing supervision signals. As noisy pseudo-labels accumulate over time, the model's discriminative capacity faces significant challenges. In practical scenarios, the distribution of features among samples often reflects their semantic properties, yet this knowledge remains largely unexplored. To address this gap, we leverage source predictions to approximate the lower and upper bounds of individual class probabilities, facilitating the calibration of pseudo-labels and averting trivial solutions. Additionally, we employ global and local strategies to independently determine thresholds for each class, thereby selecting more reliable pseudo-labels. Subsequently, we construct a soft-weighted contrastive learning module based on these dependable components, which brings potential same-class samples closer while effectively discriminating against unrelated samples. This approach harnesses various priors to enhance the model's adaptability to target domains.

Contributions. The highlights of the paper are threefold: 1) By analyzing the update properties of parameters, we design a dual-stream framework for continual test-time domain adaptation. The different learnable parameters are tuned in each stream to form a symbiotic knowledge for long-term generalization and instantaneous discrimination, while the source parameters are appropriately introduced into dual streams to alleviate catastrophic forgetting; 2) We explore reliable supervision signals with prior knowledge to guide the test time tuning. The lower and upper bounds of individual class probability are employed to calibrate the pseudo-labels, and the reliable samples are selected in a selfadaptive manner. The feature distribution of the source pretrained model is adopted to guide a soft-weighted contrastive module for capturing potential semantics lost during adaptation; 3) Extensive experimental results demonstrate that our method achieves state-of-the-art performance on several datasets. The ablation experiments are conducted to verify the effectiveness of each module.

2 RelatedWork

2.1 Domain Adaptation

Domain adaptation [Cao et al., 2022; Jiang et al., 2021] refers to acquiring knowledge from labeled data within the source domain to achieve effective performance across diverse yet related target domains. A fundamental challenge in domain adaptation lies in the misalignment between the feature and label spaces of the source and target domains [Wang et al., 2023a; Ding et al., 2023; Xie et al., 2022]. To tackle this challenge, certain domain adaptation techniques aim to guide deep models in learning domain-invariant representations [Sun et al., 2022] and classifiers [Wang et al., 2023c]. Notably, some methodologies [Ganin and Lempitsky, 2015; Tzeng et al., 2017; Ganin et al., 2016] employ adversarial training to align feature distributions with a domain discriminator, while others impose constraints on the cross-domain feature space, such as entropy constraint [Saito et al., 2019] or maximum prediction rank [Cui et al., 2020]. It's worth noting that all the aforementioned methods necessitate access to both source and target data during the adaptation process, rendering the learning transductive.

2.2 Test-Time Domain Adaptation

In recent studies on test-time domain adaptation, attention has shifted towards a more demanding scenario where solely the source model and unlabeled target samples are accessible. Certain test-time domain adaptation approaches [Li et al., 2020] leverage generative models to effectuate feature alignment between the source and target domains without requiring additional source data acquisition. Additionally, some methodologies achieve test-time domain adaptation by refining the source model with the aid of target data, obviating the need for explicit domain alignment. Test Entropy Minimization (TENT) [Wang et al., 2020] introduces entropy minimization as a test-time optimization objective, wherein normalization statistics are estimated, and channel-wise affine transformations are optimized online with each batch update. Source Hypothesis Transfer (SHOT) [Liang et al., 2020] endeavors to learn the optimal target-specific feature learning module to align with the source hypothesis.

Most test-time adaptation methodologies primarily address the offline scenario, wherein the complete test data set is available during the training phase. However, CoTTA [Wang et al., 2022] extends test-time adaptation from the offline setting to an online continual scenario. This extension tackles a more challenging yet realistic problem termed Continual Test-Time Domain Adaptation, wherein a source pre-trained model must adapt to a continuously evolving stream of target test data without access to any source data. [Gan et al., 2022] employs visual domain prompts to dynamically update a small portion of input image pixels, thereby mitigating the issue of error accumulation. Additionally, NOTE [Gong et al., ] introduces instance-aware batch normalization to rectify normalization for out-of-distribution samples.

Our Study. Our proposed method incorporates an efficient dual-stream network to ensure long-term generalization and improve instantaneous discrimination for continual test-time domain adaptation, and we first explore prior knowledge from the source pre-trained model to guide the adaptation stages.

3 ProposedMethod

Following [Wang et al., 2022], we consider a continual testtime domain adaptation setting, where a pre-trained model needs to adapt to a continually changing target domain online without source data. Consider a pre-trained model Fθ(x) with parameter θ trained on the source data. Unlabeled target domain data Xt is provided sequentially, and the data distribution continually changes. At testing stage t, when the unlabeled target data Xt = [x1t , ..., xBt ] is sent to the model Fθt , where B is the number of samples. The model Fθt needs to make the prediction Pt = [p1t , ..., pBt ] and adapts itself accordingly for the next input (θt → θt+1). It is worth noting that the total evaluation process is online, and the model only has access to the data Xt of the current stage t. We design a dual-stream network, which optimizes different parameters independently in each stream, to capture knowledge from continual domains. Meanwhile, we explore prior knowledge from the source pre-trained model. The framework is shown in Figure 1.

3.1 Dual-streamNetwork

We first design a new dual-stream pipeline for continual testtime domain adaptation to capture symbiosis knowledge of generalization and discrimination. For the convenience of expression, θt in the following mainly refers to the parameters of the dual-stream network. The parameters θt of the dualstream network are divided into two parts {ˆ θt, ¯ θt}, where only the batch normalization layers are tuned in ˆ θt and all parameters are tuned in ¯ θt, and the learning process can be denoted as follows.

where pbt represents classification result of the sample b at time t, and k ∈ K is the k-th class. The learnable parts can improve the instantaneous discriminative adaptability and long-term generalization adaptability of the model. We use the predictions of the source model as auxiliary information to predict γ. The basic idea is that a similar prediction with the source model may represent a higher reliability stream.

where sim(·) is the cosine similarity of smaples. Previous literature [Frankle et al., 2020] supports the averages of the parameters of the source model potentially has good generalization capabilities and the adapted model. Thus, we continually ensemble the parameters of the initial source model and the weights of the current model using an Exponential Moving Average of the form.

where α = 0.99 is a momentum term. For the stream that optimizes all network parameters, we hope that the objective function can be employed to directly guide the parameter transfer of the source model and the adapted one, and the Weighted Soft Parameter Alignment can be defined as follows.

where l is the layer of the network and βl repressents the similarity strength of l-th layer. BN represents the batch normalization layers. βl is increased with the deeper layers. During this process, the accumulation of noisy labels inevitably misleads parameter learning. The following sections will detail the construction of the supervision signals.

3.2 Supervision Signals

Label Calibration with Prior Knowledge. We hope to extract prior knowledge from the source pre-training model to optimize the prediction results. The first is the rough estimation of class distribution at each stage t. To achieve this, the samples are fed into the source pre-trained network, and the occurrence probability of each class ckt is then calculated.

where qb t is the class prediction of the sample b. Here, we choose the source pre-trained model to estimate this probability mainly because its parameters are fixed and not affected by pseudo-labels. Admittedly, such an estimation is not entirely accurate, so we need to relax this estimation to calibrate the supervision labels. The objective is defined as follows.

where <,> is the inner product, and δ is a relaxation factor, making the probability of each class in a limited range. Ht = [h1t , ..., hBt ] is the variable that needs to be solved and obeys three constraints. The first two constraints ensure that the result conforms to the one-hot distribution of groundtruth. δ is a relaxation factor, making the probability of each class in a limited range with ckt . The objective is a Zero-One Programming problem and can be solved with standard solvers [Wolsey, 2020]. We combine the calibrated pseudo-labels with the original predictions to ensure model stability while employing an adaptive threshold to select the final supervision labels.

Label Selection with Self-adaptive Thresholds. The calibration can suppress noisy labels, but cannot eliminate them. Thus, we adopt a confidence threshold to filter reliable labels. Thus, we present self-adaptive thresholding that automatically defines and adaptively adjusts the confidence threshold for each class by leveraging the current predictions during adaptation. The global threshold should represent the confidence of the model, reflecting the overall learning status. We set the global threshold τt as the average confidence from the model, and estimate the global confidence at each stage t. τt is defined and adjusted as:

Except for the global threshold, the local threshold is utilized to modulate the global threshold in a class-specific fashion to account for the intra-class diversity and the possible class adjacency. We compute the expectation of the model's predictions on each class k to estimate the class-specific learning status:

After integrating the global and local thresholds, we can obtain the final self-adaptive threshold of each class k.

Based on such thresholds, the samples at current batch can be divided into two parts, the reliable part Nrel(t) = {b|b ∈ B,max(yb t ) ≥ τt(argmaxyb t )} and unreliable one Nunrel(t) = {b|b ∈ B,max(yb t ) < τt(argmaxyb t )}. The objective of Lrce(Xt) can be denoted as follows:

Soft-weighted Contrastive Learning. Undeniably, the source pre-trained model is fully trained with labels, so even if the domain shift causes the classification results to be biased, it is still a suitable feature extractor. In other words, the source domain training model can still judge samples' similarity. Based on this, we design a contrastive learning framework to improve the discriminative ability of the model further. Specifically, we first exploit the source pre-trained model to extract the sample features and establish a similarity matrix.

where d represents the dth sample at time step t. Such a matrix can be further utilized to promise the model more substantial representation power, while previous methods have not achieved it. Moreover, in the context of contrastive learning, in particular, these semantic class structures can give helpful guidance in selecting contrastive pairs with similar semantics to improve training efficiency. We adopt the weighted similarity matrix wt to guide the traditional contrastive loss, which can be rewritten as follows,

where zb t = γFˆ θt (xbt ) + (1 - γ)F¯ θt (xbt ). We then introduce the components of the objective function in detail.

Positives. We attempt to present more potential positive samples by utilizing the correlation between samples during the instantaneous learning process. We select the samples of the same class with b as the positive sample set from the calibrated labels Yt.

Negtatives. The traditional contrastive loss strives to maximize the cosine distances between b and every d in the batch. Instead, we argue that not pushing away same-class pairs helps learn better semantically meaningful clusters. Specifically, we adopt the labels to exclude reliable same-class pairs from all negative pairs:

Here, we believe that the confidence of the selected sameclass samples is higher, so these samples are excluded from the negative samples. The same-class samples with lower confidence are considered potentially similar samples. Therefore, we remove them from the positives and negatives, expecting to optimize their distribution using the soft weights transfer.

3.3 Overall

The overall objective of our method is as follows.

where λ1 and λ2 are hyperparameters. In general, we do not directly use the results of pre-trained models as supervision signals, but apply them as prior knowledge to calibrate pseudo-labels, and design a soft-weighted contrastive learning method. In order to prevent the influence of noisy labels, adaptive thresholds are devised to select reliable samples.

4 Experiments

In this section, we evaluate the effectiveness of the proposed method on three benchmark datasets in terms of 1) whether our dual-stream network learns meaningful results, 2) whether the proposed label selection and correction strategies can improve the discrimination, and 3) the parameters analysis of the proposed method.

4.1 Datasets

We adopt CIFAR10, CIFAR100, and ImageNet as the source domain datasets, and CIFAR10C, CIFAR100C, and ImageNet-C as the corresponding target domain datasets, respectively. The target domain datasets were created to evaluate the robustness of classification networks [Hendrycks and Dietterich, 2019]. Each target domain dataset contains 15 types of corruption with five levels of severity. Following [Wang et al., 2022], for each corruption, we use 10000 images for both CIFAR10C and CIFAR100C datasets and 5000 images for ImageNet-C.

4.2 Implementation Details

Following [Wang et al., 2022], the corrupted images are provided to the network online, which means these images can be utilized to update the model only once in the adaptation process. In addition, unlike traditional test-time adaptation methods, which adapt to each corruption type data individually, we adjust the source model to each corruption type sequentially. We evaluate the adaptation performance immediately after encountering each corruption type data. The total type of corruption is set at 15, and the corruption level is set to the highest level of 5 (except for the gradual experiments on CIFAR10-to-CIFAR10C).

For CIFAR10-to-CIFAR10C, we use a pre-trained WideResNet-28 [Zagoruyko and Komodakis, 2016] model from the RobustBench benchmark[Croce et al., 2020]. We use Adam to optimize the network and set the learning rate to 1e-3. The data augmentation strategy is the same as [Wang et al., 2022], including color jitter, gaussian blur, gaussian noise, random affine, and random horizontal flip. For CIFAR100-to-CIFAR100C, we use a pre-trained ResNeXt29 [Xie et al., 2017] from [Hendrycks et al., 2019]. For ImageNet-to-ImageNet-C, we use the standard pre-trained ResNet-50 from RobustBench [Croce et al., 2020]. The experiments on ImageNet-to-ImageNet-C are performed under ten diverse corruption orders. The relaxation factor δ is set as 0.2, λ1 = 0.1 and λ2 = 1 in our experiments. We set βl = 1-e-5l 1+e-5l and l is the number of layers.

4.3 Baselines

We compare our method with several state-of-the-art continual test-time adaptation algorithms, the details of these methods are as follows: 1) Source directly uses the pre-trained model for adaptation without any specific method for domain adaptation; 2) BN Stats Adapt keeps the pre-trained model weights and uses the Batch Normalization statistics from the input data of the input batch for the prediction [Li et al., 2016; Schneider et al., 2020]; 3) Pseudo-Label [Lee and others, 2013] picks up the class which has the maximum predicted probability as the pseudo-labels to update the model; 4) TENT [Wang et al., 2020] reduces generalization error by reducing the entropy of model predictions on test data, TENTcontinual is a continual learning version of TENT; 5) CoTTA [Wang et al., 2022] reduces the error accumulation by using weight-averaged and augmentation-averaged predictions and avoids catastrophic forgetting by stochastically restoring a small part of the source pre-trained weights; 6) NOTE [Gong et al., ] adopts an Instance-Aware Batch Normalization to correct normalization for out-of-distribution samples; 7) RoTTA [Yuan et al., 2023] presents a robust batch normalization scheme to estimate the normalization statistics; 8) RMT [D¨obler et al., 2023] uses symmetric cross-entropy and contrastive learning to pull the test feature space closer to the source domain; 9) ROID [Marsden et al., 2023] proposes to continually weight-average the source and adapted model, and an adaptive additive prior correction scheme.

4.4 Performance Evaluation

CIFAR10-to-CIFAR10C. Table 1 shows the classification error rate for the standard CIFAR10-to-CIFAR10C task. We compare our method with the seven baseline methods. 'Gain' represents the percentage of improvement in model accuracy compared with the source method. CoTTA considers the error accumulation to improve performance further. As the latest proposed methods, NOTE attempts to improve the performance of the model in different domains from the distribution with BN. Although it performs well in domains such as Gaussian and shot, it performs poorly in some simple domains, such as Brightness and Contrast. ROID has dramatically improved the overall performance of the model. However, the model does not perform well in some difficult domains due to the limited parameters that can be learned. Compared with all the previous methods, our method achieves the best results in the average error value and most of the corruption-type data.

CIFAR100-to-CIFAR100C. Table 2 shows the classification error rate for the standard CIFAR100-to-CIFAR100C task. BN Stats Adapt and NOTE do not bring error accumulation, but there is little room for improvement. CoTTA considers the error accumulation problem and reduces the error to 32.5%. Similarly, Visual Domain Prompt performs well in some domains, but in some relatively complex domains, the limited learnable parameters lead to a limited upper bound of the model. Further, the performance of our method is better than RMT and ROID on several corruption types of data, and the average error value is reduced to 27.8%.

ImageNet-to-ImageNet-C. We also make experiments on the ImageNet dataset. Following [Wang et al., 2022], we conduct ImageNet-to-ImageNet-C experiments over ten diverse corruption type sequences in severity level 5. The average result of ten experiments is shown in Table 3. ImageNet is more complex than CIFAR-100 and CIFAR-10, and the overall average test error is more significant. Our method outperforms other competing methods and reduces the average test error to 50.3%.

4.5 Ablation Studies

In addition, we first conduct ablation experiments with the same supervision signals to prove the effectiveness of the dual-stream network. For the convenience of expression, 'Stream 1' represents the stream in that only the batch normalization layers are tuned, and 'Stream 2' is the stream in which all parameters are tuned. The results are shown in Table 4, where EMA represents the Exponential Moving Average, and WSPA is Weighted Soft Parameter Alignment. The results demonstrate that the proposed modules are helpful for performance gains. Subsequently, we focus on validating the proposed supervision signals module. It can be seen that the pseudo-label after prior calibration can effectively improve the performance of the model. In addition, the proposed label selection strategy can also effectively suppress noisy labels. Finally, the proposed contrastive learning strategy effectively optimizes the sample distribution.

4.6 Parameters Analysis

We explored how the model varies with the parameter λ, and the results are shown in Figure 2. The results demonstrate that our method is not sensitive to λ1 and λ2 at range [0.01, 1].

5 Conclusion

This paper first proposes a dual-stream structure to capture the discriminative ability, maintain generalization, and prevent catastrophic forgetting. We propose continually capturing source knowledge using different strategies in each stream to calibrate the adapted model. Then, we adopt the source predictions to rough calculate the lower and upper bounds of individual class probability, which can calibrate the pseudo-labels and avoid a trivial solution. Moreover, we select an independent threshold for each class through global and local strategies to choose reliable pseudo-labels. Based on such reliable parts, we construct a soft-weighted contrastive learning module, which pulls the potential sameclass samples closer and discriminates against uncorrelated samples. Finally, we evaluate the proposed method on several benchmarks and prove its superiority.

Acknowledgements

This work is supported in part by the National Key Research and Development Program of China (No. 2023YFC3305600), Joint Fund of Ministry of Education of China (8091B022149, 8091B02072404), National Natural Science Foundation of China (62132016, 62171343, and 62201436), Key Research and Development Program of Shaanxi (2024GX-YBXM-127) and Fundamental Research Funds for the Central Universities (ZDRC2102).