Learning Unforgotten Domain-Invariant Representations for Online Unsupervised Domain Adaptation

1 Introduction

Unsupervised domain adaptation (UDA) aims to transfer the knowledge from a labeled source domain to an unlabeled target domain. Due to the ability of deep neural networks, UDA studies [Ganin et al., 2016; Long et al., 2017; Zhang et al., 2019] have achieved good performance in a wide range of applications by learning domain-invariant representations. However, current UDA methods are in an offline manner which heavily relies on sample-rich target domains and adequate training time while real-world applications may involve online requirements, especially in real-time systems. In this paper, we discuss Online UDA (OUDA) which assumes that the target samples are arriving sequentially as a small batch rather than a static sample-rich target domain.

(b) CDAN in OUDA

Figure 1: Catastrophic forgetting in OUDA. The experiments are conducted on Office-31, A-W task. Y-axis represents the accuracy on the whole target domain while X-axis represents the steps in online training. For each step, the algorithms obtain 36 unlabeled target samples and the training epoch is set as 20. Both DANN and CDAN suffer from catastrophic forgetting while SFD (a novel method for reducing forgetting) can significantly improve generalization with little extra time cost, i.e., about 0.15 seconds per epoch.

OUDA [Moon et al., 2020] is composed of continuous learning steps. For each step, OUDA algorithm inherits the model generated in the previous step and updates the model with a small batch of unlabeled target data currently arrived. A crucial challenge in OUDA is to link new knowledge to old, i.e., learning while resisting forgetting. Although UDA has been extensively explored, few studies focus on OUDA. Existing OUDA studies [Moon et al., 2020; Hajifar and Sun, 2021] are majorly driven by applications while lacking theoretical guarantees. Existing UDA theories [Ben-David et al., 2007; Blitzer et al., 2007; Zhao et al., 2019] are also insufficient to cover OUDA which suffers from catastrophic forgetting. There exists a strong need to understand why and how to resist forgetting in OUDA tasks.

To measure the forgetting across different steps, we propose a novel metric, i.e., generator discrepancy. Then a novel generalization bound has been proved where the OUDA target error can be bounded by the proposed generator discrepancy and other measurable items based on the data currently collected. Our bound shows insights on providing theoretical guarantees for OUDA without the access of the data collected in previous steps while illustrating the tradeoffs inherent in learning and remembering representations. For example, in an extreme case when the generators in all steps share the same parameters, then we can ensure a zero generator discrepancy. However, in this case, generators cannot obtain any new knowledge since the parameters are fixed. In another hand, a large generator discrepancy can also lead to poor generalization since the OUDA target error is also bounded by the generator discrepancy. We conduct experiments to verify this conclusion as shown in Figure 1 where both DANN [Ganin et al., 2016] and CDAN [Long et al., 2017] apply no constraints on the generator discrepancy. Both methods suffer from poor generalization or catastrophic forgetting.

To minimize the proposed generator discrepancy, a Source Feature Distillation (SFD) method has been proposed which utilizes the source-only model as a teacher to guide OUDA algorithms. In the experiments, we choose two classical UDA methods, i.e., DANN [Ganin et al., 2016] and CDAN [Long et al., 2017] and one state-of-the-art (SOTA) UDA method [Jin et al., 2020] as the baselines. The experiments are conducted with real-world datasets, i.e., Office-31, Office-home and ImageCLEF-DA. It is observed that all baselines suffer from catastrophic forgetting in OUDA while SFD can significantly reduce forgetting with little extra cost as shown in Figure 1. The contributions of this paper are as follows:

1. We propose a novel metric for measuring forgetting in OUDA and show a novel target error upper bound for OUDA tasks with the proposed forgetting metric.
2. Based on the theoretical results, we propose a novel SFD method that utilizes the source-only model as a teacher to guide the algorithms in OUDA tasks.
3. To verify the effectiveness of the theoretical results and the proposed SFD, we conduct experiments with several SOTA UDA algorithms on real-world OUDA tasks.

2 RealatedWork

The goal for domain adaptation [Pan and Yang, 2009] is to generalize a learner across different domains of different distributions. Ben et al. [Ben-David et al., 2007] and Blitzer et al. [Blitzer et al., 2007] introduce a seminal theoretical model where the target error can be bounded by source risk and a divergence measure, known as the H-divergence. Based on these theoretical results, adversarial domain adaptation methods [Ganin et al., 2016; Long et al., 2017; Tzeng et al., 2017; Mao et al., 2017] learn domain-invariant representations by applying a domain discriminator to predict if the feature is from the source or the target. Then the feature extractor is trained both to maximize the discriminator and minimize the classification loss in the source domain. However, existing UDA studies [Ben-David et al., 2007; Blitzer et al., 2007; Shu et al., 2018; Zhao et al., 2019; Tachet des Combes et al., 2020] are in an offline manner which is insufficient to cover OUDA which suffers from catastrophic forgetting.

Online algorithms [Kirkpatrick et al., 2017; McMahan et al., 2013] assume that the target data are arriving sequentially as a small batch. As the algorithms have no access to the target arrived in previous steps, online algorithms suffer from catastrophic forgetting. EWC [Kirkpatrick et al., 2017] remembers old tasks by selectively slowing down learning on the weights important for those tasks. However, existing online studies [McMahan et al., 2013] or continual learning [Rolnick et al., 2019; Kirkpatrick et al., 2017;

Lopez-Paz and Ranzato, 2017] are in a supervised manner which cannot cover OUDA.

Although both UDA and online learning have been extensively explored, few studies focus on OUDA. Existing OUDA studies [Mancini et al., 2018; Moon et al., 2020; Hajifar and Sun, 2021] are majorly application-driven. [Moon et al., 2020] proposes an intuitive multi-step framework that computes the mean-target subspace by the geometrical interpretation of the euclidean space. [Mancini et al., 2018] focuses on OUDA robotic kitting in unconstrained scenarios while [Hajifar and Sun, 2021] studies the continuous cross-subject liver viability evaluation. The theoretical analysis for OUDA is still vacant.

3 Method

In this section, we formalize the problem of OUDA and describe our method for addressing the problem. Theoretical guarantees for the proposed method are shown in Sec. 4.

3.1 ProblemDefinition for OUDA

In this paper, we focus on the problems concerning k-class classification in OUDA. Basically, we follow the standard notations in this field [Ben-David et al., 2007; Blitzer et al., 2007]. Let X and Y be the input and output space. Let DS (resp. DT ) be the source domain (resp. target domain) with a marginal data distribution PS(x) (resp. PT (x)) and a marginal label distribution PS(y) (resp. PT (y)). The error of a classifier h with a feature generator g in DS is defined as:

where L is a loss function L : Rk ×Rk → R+ and L satisfies the triangle inequality. fS is the label function in DS. In the same way, we define the target error ϵT (h ◦ g).

In OUDA, DT is composed of a sequential target domains. For each step t, the algorithm collects a small batch of samples, i.e., DtT . Assume that both the labeled source samples drown from DS and the unlabeled target samples drown from DtT are i.i.d.. We define gt (resp. ht ) as the feature generator (resp. classifier) learned in step t. Then we define ϵiT (ht ◦ gt ) as the error of gt and ht in the i-th target domain, i.e., DiT where i ≤ t. For each step t, the goal of OUDA is to minimize ϵt-all T where

It's noteworthy that ∀i < t, ϵiT (ht ◦ gt ) is an immeasurable item since OUDA algorithms have no access of the target data in previous steps, i.e., DiT , ∀i < t.

3.2 Domain-Invariant Representation

To perform adaptation in an online manner, we leverage the idea of learning domain-invariant representations with domain adversarial training [Ganin et al., 2016; Long et al., 2017]. Domain-adversarial training methods are based on a three-player game that includes a feature generator g, a classifier h trained on source labels, and a discriminator d which

(g) Offline w/ SFD (h) Step 1 w/ SFD (i) Step n w/ SFD

Figure 2: Motivation of learning stable source features in OUDA. Red and blue points are source samples with two classes. Green points are currently collected target samples while gray points are unavailable target samples, i.e., uncollected or collected in previous steps. Figures (a-c) show the inputs in OUDA where the target data is generated by rotating the source samples. Figures (d-e) show the features generated by offline UDA methods. Figures (g-i) show the features generated by offline UDA methods combined with SFD.

aims to infer the domain index of the features. In the threeplayer game, g aims to learn discriminative features for h while fooling d to prevent it from inferring the domain index. Formally, domain-adversarial training methods perform a minimax optimization with a total loss:

where we have

where yd is domain label, e.g., 0 for source and 1 for target.

In OUDA, algorithms have to learn representations from a sample-rich source domain and sequential sample-poor target domains as shown in Figure 2 (a-c). By playing the three-player game with the whole target inputs, UDA methods could learn domain-invariant representations with good generalization as shown in Figure 2 (d). However, in OUDA, small Lc and large Ld are not sufficient to get such representations as shown in Figure 2 (e-f). Another critical challenge in OUDA is catastrophic forgetting which makes the situation worse when learning representations from Figure 2 (e) to Figure 2 (f). By applying a constraint on stable source representations as shown in 2 (h-i), we can gradually learn unforgotten domain-invariant target representations.

3.3 Learning Unforgotten Representations

In this paper, to get unforgotten representations in OUDA tasks, we apply a constraint on stable source features, i.e., Lsfd. Theoretical explanation for the effectiveness of Lsfd is provided in Sec. 4. Then the total loss for OUDA,

where α is a hyper-parameter and we have,

where gt is the feature generator to be learned in current tth step and gi is the previous feature generator in the i-th step. Computing Lsfd is at a high cost since it's related to the feature generators in all previous steps. To simplify the calculation, we show a upper bound for Lsfd. Let g0 be the source-only model. As L satisfies the triangle inequality,

where δ = ∑t-1 i=1 Ex∼DS [L(gi (x), g0 (x))].

It's noteworthy that δ is a small value since it has been minimized in previous steps and the only concern is the first item which corresponds to a distillation loss between g0 and gt in the source feature space, i.e., the proposed SFD. Let L be an L2 distance function, then we have:

Then the minimization of Lsfd equals to the minimization of Ex∼DS [∥∥gt (x) - g0 (x)∥∥22 ]. As shown in Figure 3, our method can be applied to any UDA method which learns domain-invariant representations, i.e., a feature generator which induces similar distributions across two domains. In the experiment part, we verify the effectiveness of our method with both domain-adversarial training methods [Ganin et al., 2016; Long et al., 2017] and non-domain-adversarial training method, MCC [Jin et al., 2020]. The experimental results in Sec. 5 show that our method can significantly reduce forgetting which contributes to the improvement of the performance in all baselines.

4 Theoretical Results for OUDA

In this section, we show the theoretical results for OUDA.

4.1 Preliminaries

Lemma 1. Let H be a hypothesis space in a input space X with any distribution D. Assume that L satisfies the triangle inequality. Then for any h, h′ , h′′ ∈ H, we have (Proof shown in Appendix A.1)

Definition 1. (Discrepancy Distance H△H ):Let H be a hypothesis space in a feature space Z . Let P1 and P2 be two distributions on Z. Then the discrepancy distance between P1 and P2 is:

Remark. Based on Lemma 1 and H△H, [Ben-David et al., 2007] and [Blitzer et al., 2007] proved the following generalization bound on the target error risk.

Theorem1. [Ben-David et al., 2007; Blitzer et al., 2007] Let H be a hypothesis space. Then ∀h ∈ H,

Remark. The first item is the source error while the second item is the domain discrepancy distance across two domains. The third item λ∗ is assumed a small const.

4.2 AGeneralization Bound for OUDA

In each step t, the goal of OUDA is to minimize ϵt-all T where

Definition 2. (Generator Discrepancy H△H): Let X (resp. H) be a input (resp. hypothesises) space. Let g1 and g2 be two generators,

Remark. gH△H is first proposed in this paper for measuring forgetting across OUDA steps. It's noteworthy that gH△H is measured by a common input distribution P with two generators while dH△H in Theorem 1 is measured by two input distributions with a common generator.

Theorem 2. (Upper bound for OUDA): Let H be a hypothesis space. Let DtT be the t-th target domain with a marginal data distribution of P tT (x). Then ∀gt , ht (Proof shown in Appendix A.2),

Remark. It's noteworthy that although ϵt-all T includes immeasurable items, i.e., ∀i < t, ϵiT (ht ◦ gt ), all items in Theorem 2 are measurable without the access of the data collected in previous steps. For the t-th step, the Source Error in Theorem 2 can be minimized by the source labels. For the Domain Discrepancy, it's noteworthy that ∀i < t, dH△H (PS(gi (x)), P iT (gi (x)))) has been minimized in previous i-th step since such it's about the i-th generator and the i-th target domain. Then the only concern is the measurable domain discrepancy distance between the source domain DS and the target domain currently arrived DtT , i.e., dH△H (PS(gt (x)),P tT (gt (x)))). For the Generator Discrepancy, it requires the current t-th generator gt sharing small gaps with all previous generators gi , i ≤ t. The proposed SFD method is to reduce ∑ti=1 gH△H(gt , gi ), details please refer to Sec. 4.3. For the Shared Error, λ∗t (i) is assumed a small const, ∀i ≤ t.

4.3 Minimizing Generator Discrepancy with SFD

To minimize gH△H(gt , gi ), ∀i ≤ t, a proper way is to minimize the feature movements between gt and gi with the same dataset drown from a distribution as wide as possible. It's noteworthy that the proposed generator discrepancy is not binding with certain distributions. As the target samples are used for obtaining new knowledge, we apply a constraint on stable source features, i.e., Lsfd to minimize generator discrepancy. However, there is still a gap between the Generator Discrepancy, i.e., ∑ti=1 gH△H(gt , gi ) and the simplified Lsfd which minimizes Ex∼DS [∥∥gt (x) - g0 (x)∥∥22 ].

Lemma 2. Let H be a hypothesis space in a feature space Z. Then ∀P1, P2, P3 ∈ Z, we have

Lemma 3. Let X (resp. H) be a input (resp. hypothesises) space. Then ∀g1 , g2 , g3 (Proof shown in Appendix A.3),

According to Lemma 3, we have

By minimizing the simplified Lsfd, we can achieve the reduction of gH△H (gt , g0 ) which also provides guarantees on the reduction of ∑ti=1 gH△H(gt , gi ) as ∀i < t, gH△H (gi , g0 ) has been reduced in the previous i-th step.

4.4 Summary

By introducing the generator discrepancy gH△H, our theory provides theoretical explanations on why catastrophic forgetting leads to poor generalization in OUDA tasks. The proposed Theorem 2 also shows insights on providing theoretical guarantees for OUDA only by utilizing measurable items with the target data currently collected. Furthermore, our theory illustrates the tradeoffs inherent in learning and remembering representations for OUDA tasks since reducing the generator discrepancy gH△H is at a cost of weakening the learning ability. We believe these insights are helpful in guiding the future design of OUDA algorithms.

5 Experiment

In this section, we conduct experiments on a wide range of real-world datasets. We modify three offline UDA algorithms, i.e., DANN [Ganin et al., 2016], CDAN [Long et al., 2017], and MCC [Jin et al., 2020], and evaluate their performance on OUDA tasks. To verify the effectiveness of the proposed SFD, we apply SFD to all baselines. The source code is available in https://github.com/FujitsuResearch/sourcefeature-distillation.

5.1 Datasets

As the most prevalent benchmark dataset in DA tasks, we conduct experiments on Office-Home [Venkateswara et al., 2017], Office-31 [Saenko et al., 2010] and ImageCLEF-DA [Long et al., 2017].

Office-31 [Saenko et al., 2010] is one of the most prevalent benchmark datasets in DA which has 31 classes with 4,652 images. Office-31 includes three domains, i.e., Amazon (A), Dslr (D), and Webcam (W).

Office-Home [Venkateswara et al., 2017] is a harder task compared with Office-31 which has 65 classes with over 15,500 images including four domains, i.e., Art (A), Clipart (C), Product (P), and Real World (R).

ImageCLEF-DA [Long et al., 2017] is a dataset organized by selecting 12 common classes shared by three public datasets, i.e., Caltech-256 (C), ImageNet ILSVRC 2012 (I), and Pascal VOC 2012 (P).

5.2 Setup

Settings andMetrics. We start OUDA tasks with a sourceonly (SO) model which trains from scratch with ResNet-50 [He et al., 2016] implemented by Pytorch. The whole target domain is divided into a sequential of sub-domains with a batch size of 36 by randomly selecting. We follow the settings in the literature for online training [Kirkpatrick et al., 2017; McMahan et al., 2013] where the algorithms have no access to the target data arrived in previous steps. The training epoch for each step is set as 20. We adopt mini-batch SGD with a momentum of 0.9 and the learning rate annealing strategy as [Ganin et al., 2016]. For each step, we use the accuracy of the whole target dataset to verify the generalization of the updated model. We both compare the last step accuracy while the average accuracy of overall steps. All tasks are repeated three times and the mean results are used for comparison.

Baselines. We verify the effectiveness of SFD with both domain-adversarial training methods [Ganin et al., 2016; Long et al., 2017] and non-domain-adversarial training method, MCC [Jin et al., 2020]. All methods are based on domain-invariant representations which learn a single feature generator to achieve good performance in both source and target domains. To achieve SFD, we use an extra source-only model to calculate Lsfd for all baselines. For all tasks, we use the same hyper-parameter where α = 1, Parameter sensitivity analysis is also applied in Sec. 5.3.

(a) Step 0: SO

(b) Step 10: w/o SFD (c) Step 10: w/ SFD (d) Step 21: w/o SFD (e) Step 21: w/ SFD

Figure 5: T-SNE visualization of features on Office-31 A-W task with DANN. Figures (a-e) represent domain alignment in OUDA: blue denotes the source features, red denotes the target features arrived in current step, and gray denotes the other target features.

5.3 Results

Overall Results. According to the data listed in Table 1 and Table 2, all baselines in OUDA suffer from catastrophic forgetting in OUDA tasks which leads to poor performance in most of the tasks. Although all baselines are effective in offline UDA training, the performance in OUDA is even worse than the source-only model in most of the tasks. Data listed in Table 1 and Table 2 shows that SFD can improve the performance of all baselines by 5.5% on the accuracy of the last step while 2.7 % on the average accuracy of all steps. These results have verified the effectiveness of the proposed SFD. However, there still exists a gap between offline training and online training. For example, in the Office-Home task, CDAN [Long et al., 2017] combined with SFD can achieve an accuracy of 59.3% in the last step while according to the results listed in [Long et al., 2017], CDAN can achieve an accuracy of 63.8% in offline training. This is caused by the tradeoffs inherent in learning and remembering representations in OUDA tasks.

Forgetting Analysis. Based on a common input distribution, we use the feature discrepancy generated by two generators to describe forgetting, for example, the forgetting between g1 and g2 over a distribution P can be described as a L2 discrepancy distance, i.e., Ex∼P[∥∥g1 (x) - g2 (x)∥∥22 ]. Figure 4 (a,b,c) show the results of forgetting analysis in the source domain between the current generator and the source-only generator, i.e., Lsfd. The proposed SFD method has a good performance on the convergence of Lsfd in all baselines. As shown in figure 4 (d,e,f), all baselines suffer from catastrophic forgetting in the target domain, i.e., a large target feature discrepancy even in adjacent steps while the proposed SFD can significantly reduce such forgetting by about 70%.

T-SNE Visualization. Figure 5 shows the T-SNE[Maaten and Hinton, 2008] visualization with DANN [Ganin et al., 2016] on task Office-31 A-W. Although DANN has aligned the red points (available target features) and blue points (source features) in a single step, the gray points (unavailable or unseen target features) suffer from catastrophic forgetting which leads to a similar feature distribution with the sourceonly model on the whole target domain. The proposed SFD can significantly reduce such forgetting where the whole target can be gradually aligned to the source domain.

Parameter Sensitivity Analysis. The parameter α in Eq. 5 is to control the weights of Lsfd. A larger α means a harder constraint on stable source features, i.e., a smaller Lsfd. According to Sec. 4.3, a smaller Lsfd is beneficial to a smaller ∑ti gH△H(gt , gi ). However, a smaller ∑ti gH△H(gt , gi ) also means a weaker ability to obtain new knowledge. When ∑ti gH△H(gt , gi ) = 0, the algorithm has no ability to obtain any new knowledge. As the Theorem 2 is bounded by both ∑ti gH△H(gt , gi ) and dH△H (PS(gt (x)), P tT (gt (x)))), there exists a trade-off between the two items. Data listed in Table 3 shows the results of parameter sensitivity analysis where α is at a wide range form 0.1 to 10. The results show that the proposed SFD has stable performance at a wide range of α while we can also observe the trade-offs on minimizing ∑ti gH△H(gt , gi ). When α = 10, the performance is worse than other settings except α = 0, the removal of the SFD.

6 Conclusion

Existing unsupervised domain adaptation (UDA) studies focus on transferring knowledge in an offline manner. However, many tasks involve online requirements, especially in realtime systems. In this paper, we discuss online UDA (OUDA) which suffers from catastrophic forgetting with the drawback of poor generalization. We propose a novel generalization bound for OUDA which illustrates the tradeoffs inherent in learning and remembering representations. To minimize the proposed bound, we provide a novel SFD for OUDA which utilizes the source-only model as a teacher to guide OUDA algorithms. The experimental results show that SFD significantly improves the generalization of all baselines in online iterative procedures.

A Appendix

In this section, we show the proofs.

A.1 Proof of Lemma 1

Let H be a hypothesis space in a input space X with any distribution D. We denote the loss function L : Rk ×Rk → R+ and assume that L satisfies the triangle inequality. Then for any h, h′ , h′′ ∈ H, we have

Proof.

Then we finish the proof.

A.2 Proof of Theorem 2

Let H be a hypothesis space. Let DtT be the t-th target domain with a marginal data distribution of P tT (x). Then ∀gt , ht ,

Proof. ∀i ≤ t, let fS and fT be the label functions in two domains. According to Lemma 1,

A.3 Proof of Lemma 3

Let X (resp. H) be a input (resp. hypothesises) space. Then ∀g1 , g2 , g3 , we have

Proof. According to Lemma 2, we have

Then we finish the proof.