Adversarial Bi-Regressor Network for Domain Adaptive Regression

1 Introduction

Deep neural network has become popular on solving many practical scenarios whether the specific task is classification or regression [Sellami and Tabbone, 2022; Xu et al., 2014]. However, considerable network parameters to be optimized means we have to collect extensive training samples with precise annotation, which tends to be time-consuming and laborious for real applications [Krizhevsky et al., 2012]. The intuition is to utilize the off-the-shelf label-sufficient data to accumulate knowledge and directly adapt it to solve other specific and relevant challenges [Awais et al., 2021; Xia et al., 2021a]. But the knowledge transfer is negatively obstructed by the distribution discrepancy across training and test sets so that the well-learned network suffers from performance degradation [Jing et al., 2021; Xia et al., 2021b]. The dilemma motivates the development of unsupervised domain adaptation (UDA) to bring knowledge from source domain to predict unlabeled target instances by learning domaininvariant representations. [Pan et al., 2019].

∗The work was done during the internship at MERL.

Along with this direction, abundant UDA solutions have achieved promising performance in classification task [Na et al., 2021; Chang et al., 2019; Xia and Ding, 2020]. To the best of our knowledge, there are a few studies exploring the domain adaptation regression while they focus on the theoretical analysis in shallow regime without the benefit of deep neural architecture [Pan et al., 2010; Yamada et al., 2014]. The regression task is of great importance and common in our reality, e.g., indoor wireless localization [Tong et al., 2021; Zhou et al., 2021; Arnold and Schaich, 2021]. Concretely, a robot moves in a multi-room environment as shown in Figure 1 and communicates with multi-modality anchors, e.g., Wi-Fi and ultra-wideband (UWB), placed at fixed locations denoted as red points. By processing the multi-modal radio frequency (RF) signals, one needs to predict the coordinates of robot in the floor map and make subsequent decisions for path planning and obstacle avoidance. Potential applications include smart home and autonomous factory where conducting the asset tracking is crucial.

The current DAR works mainly focus on the visual regression problem, where convolutional network typically detects the boundary information of object beneficial for the domain adaptation [LeCun et al., 1998; Krizhevsky et al., 2012]. Thus, the relatively simple alignment strategies as DANN [Ganin and Lempitsky, 2015] easily learn the domaininvariant information for regressive task. And RSD [Chen et al., 2021] empirically finds the further control of feature scale learned from image data achieves better distribution alignment. However, the multi-modal RF signals are sensitive to environmental changes due to the fact that a subtle change in the environment may impact the RF signal propagation and result in signal variations at these RF anchors. This leads to a significant distribution shift across different scenarios and affecting the model generalization. And the relationship between multi-sensor RF signals and localization output is too implicit to be detected, which increases the difficulty of learning intrinsic representations across two domains. We empirically evaluate the existing UDA methods [Courty et al., 2017; Li et al., 2020], and notice they hardly improve the model generalization when solving indoor localization based on multi-sensor RF signals.

Step 3: Update Extractor & Discriminator

Figure 2: Overview of our proposed adversarial bi-regressors network (ABRNet) including feature generator F(·), two regressors { ˆG, ˆR}, { ˜G, ˜R} and discriminator D(·). The main training process involves three stages. During the first step, ABRNet utilizes all source supervisions to train the network. For the second step, we introduce soft-similarity to maximize the bi-regressor difference. The third one aims to generate domain-invariant representations by updating feature generator and discriminator with the frozen regressors.

To overcome the practical challenges, this paper proposes a robust and effective Adversarial Bi-Regressor Network (ABRNet). Specifically, the ABRNet network consists of one feature generator and two same-architecture regressors. To identify the certain and uncertain target samples, we adopt the adversarial training strategy by maximizing the bi-regressor discrepancy to seek domain-invariant representations. However, it is difficult to directly align source and target domains in the presence of a large discrepancy. An intuitive solution is to synthesize samples in intermediate domains to bridge the original two domains and gradually eliminate their domain mismatch. The main contributions of our work are summarized three folds:

• First, we propose an adversarial bi-regressor architecture to achieve a direct alignment across the source and target domains. Specially, the disagreement of dual regressors is maximized to discover target samples outside the source support and then learn domain-invariant features.
• Second, to fight off a situation where the cross-domain

distribution discrepancy becomes relatively significant, as in the case of multi-modal indoor localization, we attempt to construct two intermediate domains between the source and target domains and gradually eliminate their mismatch to achieve distribution alignment.

• Finally, the experimental performance on multi-model RF signal and image benchmarks illustrates the effectiveness of our method on solving challenging DAR in real applications. Moreover, the empirical studies analyze the contributions of our designed modules.

2 The ProposedMethod

2.1 Problem Statement andMotivation

Domain adaptive regression (DAR) aims to learn transferable knowledge from the label-sufficient source domain Ds = {Xsi , lsi }ns i=1 to predict the continuous output for the target samples Dt = {Xti }nt i=1 , where Xs/t i , lsi corresponds to the instances and annotations sampled from the continuous space, and ns/t is the number of instances for each domain. Moreover, Xsi and Xti are collected from the same sensors but with varying environment settings such as the change of furniture location, but share the identical continuous label space, e.g., 2D coordinate within the same floor map.

Take the fingerprinting-based indoor localization as an example [He and Chan, 2016; Wang et al., 2017; Koike-Akino et al., 2020]. During the offline fingerprinting phase, a robot moves in a pre-defined area, e.g., a multi-room apartment in Figure 1, and registers locations by collecting multi-modal RF signals from a set of fixed (Wi-Fi and UWB) anchors. Denoting the fingerprinted RF signals as Xsi ∈ Rm×d where m is the number of continuously fingerprinted samples during the offline fingerprinting phase, and d is the dimension of the concatenation of multi-modal RF signals. Corresponding 2D coordinates li = (lxi , lyi ) of m fingerprinted samples can be obtained from the robot using more costly on-board sensors such as Lidar and high-precision wheel encoder, where lxi ∈ [0,L], lyi ∈ [0,W] and L, W are the length and width of the floor respectively. Then, during the online test phase, new data Xti are collected under, possibly, changed environments such as the movement of furniture and the close/open of a door. As a result, well-trained localization models using the offline fingerprinted (source domain) data Xsi may suffer from performance degradation when evaluating it on the test (target domain) samples Xti from changed environments. The problem of interest is to predict the continuous 2D coordinates of these new test data Xti by learning intrinsic and domain-invariant features from the offline signals.

Motivated by this consideration, this paper presents a novel method named Adversarial Bi-Regressors Network (ABRNet) to achieve distribution alignment for DAR setting. As shown in Figure 2, ABRNet includes one feature extractor, two same-architecture regressors and one discriminator. The dual-regressor design allows the model to detect target samples outside source distribution and serves as one implicit discriminator to generate domain-invariant features. Moreover, to overcome the considerable distribution shift across source and target domains, ABRNet constructs source-similar and target-similar domains via the linear combination of original inputs and align them with the explicit discriminator to gradually eliminate the original cross-domain discrepancy.

2.2 Bi-Regressor Discrepancy Discovery

The feature generator takes source and target samples as input to yield high-level feature representations, i.e., f s/t i = F (Xs/t i ). For specific tasks, we can consider the CNN or LSTM network architecture as the feature extractor. However, the significant cross-domain difference results in the feature distribution mismatch. For instance, in the indoor RF localization task, source and target features corresponding to the same location may become significantly different due to environment variations nearby. Thus, the regressor learned from the source supervision fails to accurately predict the coordinate of several target instances which are shown as shaded area in Figure 3 and far from the source support. To achieve domain alignment, we first detect these target samples and adjust feature extractor to generate them within the source support. For the convenience of illustration, we name these target samples as source-dissimilar ones.

According to the voting strategy, two regressors easily produce inconsistent prediction on source-dissimilar target instances. Thus, we utilize the prediction disagreement to detect those target samples. Specifically, the introduced two regressors are equipped with the same network architecture, i.e., two fully-connected layers transforming the highlevel feature into the final prediction via ˆgs/t i = ˆG(f s/t i ) or ˜gs/t i = ˜G(f s/t i ) andˆls/t i = ˆR(ˆgs/t i ) or˜ls/t i = ˜R(˜gs/t i ), where { ˆG, ˆR} and { ˜G, ˜R} are the components of two regressors, respectively. If the source and target regressors provide different predictions for the same target samples, we consider them as source-dissimilar ones with high confidence. The prediction discrepancy means that these two regressors involve different predictive abilities for target instances. For this objective, we need to maximize the discrepancy of two regressor. Different from the classification task, the final output of the regressors fail to reflect the probability distribution of samples in the continuous label space. Although the maximization of the final outputs of two regressors makes them different, such an operation easily misleads the regressor to deliberately search for solutions that are far from the groundtruth, leading to degraded performance of the regressor.

On the other hand, it is observed that the first output of regressor gt i (g ∈ {ˆg, ˜g}) indicates the distribution of hidden feature space. Thus, we consider gt i as the prior condition for the final output and concatenate them as ˆhti = [ˆgt i ,ˆlti ] or ˜hti = [˜gt i ,˜lti ]. To quantify the similarity between ˆhti and ˜hti that reflects the bi-regressor discrepancy, one option is to use the Jaccard Similarity Coefficient (IoU score) from the field of object detection using visual sensors [Gupta et al., 2014]. However, the non-differentiable nature of the IoU score leads to the optimization of the network weights problematic. Alternatively, a differentiable soft-similarity is developed here for this purpose:

where ⟨·, ·⟩ is the inner-product suggesting the intersection of ˆhti and ˜hti , ⊗ is the element-wise product, and d is the dimension size of hti (h ∈ {ˆh, ˜h}). Note that gt i is activated with the Sigmoid function before computing this metric. The minimization of Eq. (1) decreases the intersection between ˆgt i and ˜gt i to further increase the difference of dual regressors, and vice verse. With Eq. (1), we can achieve the domain alignment with the bi-regressor in the following three steps.

First, due to the accessibility of source labels, it is simple to train the entire network to learn and adapt the source support set via a supervised manner as:

where L∗r is the mean-square error (MSE) loss. This training manner likely yields two identical regressors. To avoid this issue, the second step is to maximize the difference of the bi-regressor with the aforementioned soft-similarity score to detect the source-dissimilar instances. Meanwhile, we still need to guarantee the predictive ability of model on the source samples. These considerations lead to the following objective function formulated as min ˆG, ˜G, ˆR, ˜R Lr + Ls. It is worth noting that the second step only updates the bi-regressor with the feature generator fixed. To this end, we can explore the bi-regressor discrepancy to discover the target samples outside the source support. The third step optimizes the feature extractor to move these target samples into the source support set by improving the overlap between ˆhti and ˜hti with the formulation as minF -Ls.

2.3 Intermediate Bi-Domain Alignment

According to the above illustration, the conditional biregressor discrepancy maximizes the difference of two regressors to encourage the feature generator to produce domain-invariant representations. Thus, the essence of this module considers the combination of the two regressors as one discriminator to directly distinguish the source features from the target ones. However, the aforementioned training manner means these two regressors should be very similar with smaller difference which makes it invalid for considerable cross-domain shift.

An intuitive solution is to find two intermediate domains between the source and target ones and gradually mitigate the shift of two new introduced domains to align original distributions. To this end, we further explore data augmentation to achieve this motivation. First, we adopt a fixed ratio λ to linearly combine the source and target samples to synthesize source-similar and target-similar instances as:

where λ is set to 0.7 in all experiments. The combined instances gradually form two intermediate domains where one is closer to the source domain and the other is similar to the target domain. Second, we introduce an additional discriminator following the feature generator and utilize their adversarial relationship to mitigate the domain shift between source-similar and target-similar ones. Concretely, the augmented instances flow into the feature generator and two regressors with the corresponding output, i.e., f s/t ∗ = F(X∗s/t ), ˆls∗ = ˆR( ˆG(˜f s ∗ )) and˜lt∗ = ˜R( ˜G(˜f t ∗ )). And then we adopt the adversarial loss as the following:

where D∗ s/t denotes the source-similar or target-similar domain and D(·) is the domain discriminator. Since the augmented samples are derived from the original source and target samples, their adversarial training would trigger the domain alignment of the original two domains. Thus, our augmented domain alignment can compensate the previous biregressor alignment.

2.4 Overall Objectives

So far, we have illustrated the details of our proposed ABRNet to seek domain-invariant feature representation for crossdomain regression. The overall optimization iterates the following three steps until the convergence is achieved or the maximum number of iterations is reached:

3 Experiments

3.1 Experimental Steup

Datasets. 1) SPAWC2021 [Arnold and Schaich, 2021] is a large-scale indoor localization dataset, where a robot moves in a multi-room apartment and uses the received multi-modal RF signals from the deployed devices to predict its coordinates in this floor plan. For each location, the recorded signals for this robot consists of RSSI (R, scalar), CSI (C, 64 values over 64 sub-carrier frequencies) from 11 Wi-Fi anchors, UWB (U) from 3 anchors recording range and power readings, and a 9-dimensional IMU (I) signal. SPAWC2021 consists a well-annotated Dataset1 (source domain) with 750k samples and one unlabeled Dataset2 (target domain) with 650k instances. Their differences are the movement of furniture around the green box in Figure 1 and the change of data collection time, which results in significant distribution difference. 2) dSprites [Higgins et al., 2017] is popular 2D synthetic image dataset suitable to unsupervised regressive domain adaptation and includes three domains as Color (C), Noisy (N) and Scream (S). And each domain involves 737,280 images. The regressive tasks in dSprites are to predict the scale and (x,y) coordinates.

Implementation details. Due to the difference of data type, we adopt distinct backbones to learn feature with corresponding optimizer on two benchmarks. For SPAWC2021, when moving along with the trajectory, the current location of robot is related to the status of previous t time steps. Considering this point, we adopt Bi-LSTM as feature generator to capture the temporal dependency in the high-level representations. The inputs of Bi-LSTM correspond to the signals of 10 continuous time steps and its output is a 512-dimensional vector for each instance. In addition, the varying environmental factors have different impacts on different sensors. To evaluate the effectiveness of our method with various signals, we consider the concatenation of the multi-modal signals as the input. It is worth noting that the dimensions of the processed RSSI, CSI, UWB and IMU are 8, 40, 6 and 9, respectively. The domain-specific regressors and the discriminator both comprise of two full-connection layers. In the implementation, the Adam optimizer with a learning rate of 0.001 is used to update all network components. For dSprites, we following the training strategy of RSD [Chen et al., 2021] utilize the pre-trained ResNet-18 as backbone to extract features and SGD with a momentum 0.95 to optimize the network architecture. The learning rate of the new added network modules is 10 times larger than that of the pre-trained module.

Baselines. To assess the power of our method on solving DAR, we compare it with the state-of-the-art domain adaptation methods which are appropriate for the regressive challenges. Specifically, the competitors are divided into two branches. The first one is metric-based solutions such as Transfer Component Analysis (TCA) [Pan et al., 2010], Joint Distribution Optimal Transport (JDOT) [Courty et al., 2017], Deep Adaptation Network (DAN) [Long et al., 2015], Adaptive Feature Norm (AFN) [Xu et al., 2019] and Enhanced Transport Distance (ETD) [Li et al., 2020] and Representation Subspace Distance (RSD) 1 [Chen et al., 2021]. Another is domain confusion approaches such as Domain Adversarial Neural Network (DANN) [Ganin and Lempitsky, 2015] and Maximum Classifier Discrepancy (MCD) [Saito et al., 2018]. The domain adaptation baselines are deployed in the identical network (LSTM or ResNet-18) and tested on both datasets.

3.2 Comparison Results

Table 1 and Table 2 summarize the results of the mentioned methods on SPAWC2021 dataset with the concatenation of different signals. From both tables, several observations can be made as follows. First, our proposed method outperforms the baselines by a large margin on the average mean square error (MSE). Specially, when only two types of sensor are used, our strategy further improves the precision of localization prediction by comparing it with the second best result obtained by DANN (1.04 v.s. 1.25). It suggests that our method effectively eliminates the distribution discrepancy across Dataset1 and Dataset2 and transfers more source knowledge to target domain. Compared with DANN, our learning strategy explores the data augmentation to construct source-similar and target-similar samples and aligns them to achieve distribution matching. Second, it is simple to find that the model generalization is driven by the combination of different type of sensors. For example, when directly using the well-trained LSTMmodel into target domain, training model with CSI and IMU (CI) obtains better localization than that with UWB and IMU (UI) (0.98 v.s. 2.43). The main reason lies in that the signals UI are sensitive to varying external factors triggering considerable shift across source and target domains. Our proposed approach shows stronger mitigation of cross-domain discrepancy than the selected baselines. Especially, the comparison between our method and MCD with (1.58 v.s. 2.10) illustrates that our designed bi-regressor module better utilizes the maximization of discrepancy to discover the target samples far away from the source support and then minimize the dual-regressor difference to generate domaininvariant representations. Third, our method achieves the best localization precision (MSE:0.54) with the combination of all four type of sensors where the MSE of LSTM without any domain adaptations is about 0.88. The performance improvement is attributed to the efficient alignment of source and target distributions.

1 Note that we attempt to evaluate RSD with the public code on SPAWC2021 dataset but the low-rank operation easily suffers from the vanishing gradient problem within a few iterations. Thus, we fail to report the results of RSD on SPAWC2021 dataset.

Table 3 reports the regressive performance on dSprites dataset. The distribution discrepancy in some tasks is smaller so that the well-learned source model (ResNet-18) shows promising prediction on target domain, e.g., N→C and S→C, where our method achieves comparable performance. With respect to more challenging tasks such as C→S, the prediction ability of our model on target domain becomes much better than that of ResNet-18 with 0.27 v.s. 0.90. Moreover, compared with RSD, our model further reduces the MAE from 0.32 to 0.20 on task C→N, which demonstrates that our model effectively eliminates the domain shift.

(a) LSTM

(b) DANN

(b) Ours

Figure 4: Visualization of average MSEs over 50 × 50 cm2 on the floor map. The x-axis and y-axis are divided several intervals with 0.5

meter, which forms the chessboard. The number of each grid is computed by taking the average MSE of all test samples falling into the grid.

3.3 Empirical Analysis

Visualization. To better comprehend the performance gain of our method on SPAWC2021, Figure 4 specially visualizes the average MSE over 50×50 cm2 grids in the floor map and compare it with other baselines e.g., LSTM and DANN. The data description of SPAWC2021 shows for target domain, several cabinets were newly deployed in the area indicated by red box during the online data collection, leading to a distribution shift across source and target data. Thus, ResNet-18 and DANN give much worse localization performance than the proposed method for the target regressive task in the red box. Our method shows better capacity to distinguish sourcedissimilar samples in the red box and significantly reduces the localization errors.

Ablation Study & Training Stability. Our method includes two important operations: conditional bi-regressor discrepancy (CBRD) and data augmentation driven game (DADG). To analyse the contribution of each module, we first remove one part while keeping the other intact to form two variants: w/o DADG and w/o CBRD. From the results in Figure 5, it is seen that CBRD contributes more to the performance gain than DADG as the amount of localization error increase more w/o CBRD than w/o DADG. In addition, we analyse the performance sensitivity to different parameter (λ)

selections on tasks UI and RI. From Figure 5, the model appears to be insensitive to λ and achieves the best performance with λ = 0.7. Finally, to assess the training stability, we record the MAE of target domain via target regressor over the number of iterations in Figure 5 (c) which suggests the training process achieves convergence.

4 Conclusion

This paper explores domain adaptation regression (DAR) aiming to transfer source knowledge to provide continuouslabel prediction for unlabeled target samples in real-world applications such as indoor localization. To this end, we propose a novel adversarial bi-regressor network (ABRNet) that consists of a feature generator, two regressors and a discriminator. First, we utilize the prediction disagreement of two regressors to identify target samples outside source distribution and then seek domain-invariant features depending on the adversarial relationship between feature generator and dual regressors. Moreover, considering the difficulty of direct alignment across two domains with large domain shift, ABRNet attempts to construct two intermediate domains with smaller discrepancy to gradually eliminate the original domain mismatch. Extensive experiments verify our method outperforms the state-of-the-art on solving DAR issue.