Meta-Tsallis-Entropy Minimization: A New Self-Training Approach for Domain Adaptation on Text Classification

1 Introduction

Text classification plays a crucial role in language understanding and anomaly detection for social media text. With the recent advance of deep learningf [Kipf and Welling, 2017; Devlin et al., 2019], text classification has experienced remarkable progress. Despite the success, existing text classification approaches are vulnerable to domain shift. When transferred to a new domain, a well-performed model undergoes severe performance deterioration. To address such deterioration, domain adaptation, which aims to adapt a model trained on one domain to a new domain, has attracted much attention [Du et al., 2020; Lu et al., 2022].

A direct way to achieve domain adaptation is to build a training set that approximates the distribution of the target

.

domain. For this purpose, self-training [Zou et al., 2019; Liu et al., 2021] uses the unlabeled data from the target domain to bootstrap the model. In specific, self-training first uses the model's prediction to generate pseudo-labels and then uses the pseudo-labeled data to re-train the model. In this process, self-training forces the model to increase its confidence in the confident class, which is a Gibbs entropy minimization process in essence [Lee and others, 2013].

However, Gibbs entropy minimization is sensitive to prediction errors [Mukherjee and Awadallah, 2020]. To handle the intractable label noise (i.e., prediction errors), data selection strategies are designed to select reliable pseudo labels [McClosky et al., 2006; Reichart and Rappoport, 2007; Rotman and Reichart, 2019]. Among them, many qualified achievements [RoyChowdhury et al., 2019; Shin et al., 2020] are grounded on prior knowledge about the tasks (e.g., the temporal consistency on video [RoyChowdhury et al., 2019]), and thus hard to be applied in text classification tasks. Since the Gibbs entropy minimization process in self-training is to minimize the model's uncertainty on the new domain, [Liu et al., 2021] recently proposes to replace the Gibbs entropy with the Tsallis entropy, which is another effective metric for measuring uncertainty.

Tsallis entropy is a generalization of Gibbs entropy, referring to a set of entropy types controlled by the entropy index. Fig. 1 shows the change of Tsallis entropy with different entropy indexes for binary problems. When the entropy index is small (the resultant entropy curve is sharp), the entropy minimization process tends to increase one dimension to 1.0 sharply, thus only being suitable for the scenario where pseudo labels are reliable. Otherwise, Tsallis entropy with a larger entropy index (smoother curve) is more suitable for scenarios with large label noise, e.g., domain adaptation scenarios with a large domain shift. Researchers [Liu et al., 2021] tried to use the Tsallis entropy to improve self-training, but the proposed objective only involves a unified entropy index for all unlabeled data in the target domain. As illustrated in [Kumar et al., 2010; Kumar et al., 2020], different instances in the target domain have different degrees of shifts from the source domain. Thus, a unified entropy index cannot fully exploit the different pseudo instances in the target domain.

In this paper, we propose Meta-Tsallis-Entropy Minimization (MTEM) that uses an instance adaptive Tsallis entropy minimization process to minimize the model's prediction uncertainty on the target domain. Since the best entropy indexes changes along with the training, manually selecting an appropriate entropy index for each unlabeled data is intractable. Thus, we employ meta-learning to adaptively learn a suitable entropy index for each unlabeled data. The meta-learning process iterates over the inner loop on the target domain and outer loop on the source domain. In this process, the parameters optimized on the target domain also achieves a low loss on the source domain, which forces the model to obtain task informations on the target domain. However, the proposed MTEM still faces two challenges.

Firstly, the meta-learning algorithm in MTEM involves a Second-order derivation (i.e., the gradient of the entropy index), which requires much computation cost, especially when the model is large. Hence, it is hard to apply MTEM for prevailing big pre-trained language models. To this end, we propose to approximate the Second-order derivation via a Taylor expansion, which reduces the computation cost substantially.

Secondly, minimizing Tsallis entropy requires the guidance of pseudo labels (see § 2.2 and § 2.3). Previous selftraining approaches generate pseudo labels by selecting the prediction with the largest probability (i.e., greedy selection), which tends to collapse when the model's prediction is unreliable [Zou et al., 2019]. To this end, we propose to sample pseudo labels from the model's predicted distribution instead of a greedy selection. Further, we propose an annealing sampling mechanism to improve the sampling efficiency.

To summarize, our contributions are in three folds 1 :

(i) We propose Meta-Tsallis-Entropy Minimization (MTEM) for domain adaptation on text classification. MTEM involves an approximation technique to accelerate the computation, and an annealing sampling mechanism to improve the sampling efficiency.

(ii) We provide theoretical analysis for the MTEM, including its effectiveness in achieving domain adaptation and the convergence of the involved meta-learning process.

(iii) Experiments on two benchmark datasets demonstrate the effectiveness of the MTEM. Specifically, MTEM improves BERT on cross-domain sentiment classification tasks with an average of 4 percent, and improves BiGCN on cross-domain rumor detection task with an average of 21 percent.

1 As the rest paper involves many mathematic symbols, we provide a symbol list (Tab. 7 in Appendix A) for reading convenience.

2 Preliminary

2.1 Domain Adaptation on Text Classification

Text classification is a task that aims to map a text to a specific label space. On a correct classification case, the process is expressed as yi = argmax k f[k](xi; θ), where xi ∈ X

is an input text, yi ∈ {0, 1}K is the corresponding one-hot label with K classes, and f is a model with parameters θ, f(xi; θ) is the prediction probability. Domain adaptation is to adapt a text classification model trained on the source domain (denoted as DS) to the target domain (denoted as DT ). On the source domain, we have a set of labeled instances, i.e., DS = {(xi, yi)}Ni=1 , which satisfies that DS ⊆ DS. On the target domain, unlabeled text in the target domain is available, which we denote as Du T = {(xm)}Um=1 .

2.2 Tsallis Entropy

In information theory, Tsallis entropy refers to a set of entropy types, where the entropy index is used to identify a specific entropy. Formally, Tsallis entropy with α denoting the entropy index is written as Eq. (1),

where pi is the prediction probability. When α > 1, eα is a concave function [Plastino and Plastino, 1999]. When α → 1, eα recovers the Gibbs entropy, as shown in Eq. (2) 2 :

More intuitively, Fig. 1 exhibits the impact of the entropy index on the curves of the Tsallis entropy type. Specifically, a larger entropy index makes the curve more smooth, while a smaller entropy index exerts a more sharp curve.

Extending from the unsupervised Tsallis entropy, the corresponding Tsallis loss ℓα(pi, yi) is expressed as Eq. (3). When α → 1, the corresponding supervised loss is the widely used cross-entropy loss (see Appendix B.1).

2.3 Self-Training for Domain Adaptation

Self-training aims to achieve domain adaptation by optimizing the model's parameters with respect to the supervised loss on the source domain and the unsupervised loss (prediction uncertainty) on the target domain, as shown in Eq. (4).

2 The second equation is obtained by L'Hˆopital's rule.

Algorithm 1 Meta-Tsallis-Entropy Minimization

where LS is a supervised loss, LT is an unsupervised loss, and λ is a coefficient to balance LS and LT . Due to the simplicity, Gibbs entropy is widely used to measure the prediction uncertainty on the target domain [Zou et al., 2019; Zou et al., 2018], which is expressed as below:

However, as LT (θ|Du T ) in Eq. (5) is a concave function, minimizing LT (θ|Du T ) is hard to converge because the gradients on the minimal are larger than 0 [Benson, 1995]. For this purpose, self-training uses pseudo labels to guide the entropy minimization process, i.e., replacing Eq. (5) with Eq (6) where ˜ yi = argmax k f[k](xi; θ) is the pseudo label.

3 Meta-Tsallis-EntropyMinimization

MTEM inherits the basic framework of self-training, i.e., (i) simultaneously minimizing the supervised loss on the source domain and the unsupervised loss (prediction uncertainty) on the target domain; (ii) generating the pseudo labels to guide the entropy minimization process. The improvements of the MTEM are in three folds. Firstly, we propose an instance adaptive Tsallis entropy to measure the prediction uncertainty (§ 3.1). Secondly, we propose to use a meta-learning algorithm to minimize the joint loss (§ 3.2), which involves an approximation technique to reduce computation cost (§ 3.3). Thirdly, we propose to generate pseudo labels with an annealing sampling (§ 3.4). Fig. 2 exhibits the overview of the MTEM, and Algorithm 1 presents the core process.

3.1 Instance Adaptive Tsallis Entropy

The instance adaptive Tsallis Entropy, i.e., the unsupervised loss on the target domain, is as below:

where ψ[k] indicates the entropy index for unlabeled data xk, eψ[k] is the resultant Tsallis entropy.

Such an instance Tsallis entropy minimization is more effective in exploiting the model's prediction. In general, the prediction correctness is different on different instances. Thus, entropy index should be different on different instances too. For the instances with wrong prediction, we can increase the entropy index to make Tsallis entropy more smooth, then the model is updated more cauciously. Otherwise, for instances with correct prediction, we can set a small entropy index to update the model more aggressively.

However, as we are not aware of the label, setting the appropriate entropy indexes for each unlabeled data is intractable. Furthermore, as prediction errors can be corrected during the model training, the best entropy indexes change along with the updates of the model. To handle the above issues, we propose to use meta-learning to determine the entropy indexes automatically.

3.2 Meta-Learning

Meta-learning can help MTEM to find the appropriate entropy indexes due to the following reasons. Firstly, parameters optimized on a well-determined instance adaptive Tsallis entropy (entropy indexes are determined appropriately) should be more generalizable, which corresponds to the characteristics of meta-learning, i.e., training a model that can be fast adapted to a new task [Finn et al., 2017]. Secondly, the meta-learning process updates the entropy indexes dynamically, thus maintaining the consistency between the model's parameters and the entropy indexes along with the whole training process. In specific, the meta-learning algorithm in MTEM iterates over the Inner-Loop on the target data and the Outer-Loop on the source domain.

In the Inner-Loop, we fix the entropy indexes to optimize the model's parameters with respect to the instance-adaptive Tsallis entropy on the target domain. In specific, we sample a batch of unlabeled data B from Du T and update the model with respect to their instance adaptive Tsallis entropy (LT (θ, ψ|B)), as shown in Eq. (8):

However, as introduced in § 2.2 and § 2.3, eψ in Eq. (7) is a concave function hard to be minimized. Following the way in self-training, we use the equation in Eq. (9) 3 to transform a concave function to a convex function,

where ˜ yi ∈ {0, 1}K is a one-hot pseudo label sampled from the model's prediction probability (i.e., ˜ yi ∼ f(xi; θt)). Then, the objective in the Inner-Loop is as Eq. (10):

3 Deduction is provided in Appendix B.2

In the Outer-Loop, we validate the model updated (i.e., ˆ θt+1(ψt) in Eq. (8)) with labeled data from the source domain. Since different entropy indexes ψ leads to different ˆ θt+1(ψt), we adjust ψ to find the better ˆ θt+1(ψt ) that can be fast adapted to the validation set. For this purpose, we optimize the entropy indexes ψ to minimize the validation loss. In each meta-validation step, we sample a valid batch of labeled data from the source domain, i.e., V ∼ DS, and use the validation loss LS(ˆ θt+1(ψt )|V) to evaluate the model, then update entropy indexes ψ with ▽ψLS(ˆ θt+1(ψt )|V). With the updated entropy indexes ψt+1, we return to update the model's parameters, as shown in line 8 of Algorithm 1.

3.3 Taylor Approximation Technique

The first challenge in the above meta-learning algorithm is the computation cost carried out in the ▽ψLS(ˆ θt+1(ψt )|V). Formally, the computation of ▽ψLS(ˆ θt+1(ψt )|V) is as:

where the second equation is obtained by substituting ˆ θt+1(ψt ) with Eq. (8). Since ∂ 2LT (θ,ψt |B) ∂θ∂ψ in Eq. (11) is a hessian matrix (Second-order derivation), the computation in Eq. (11) is intractable. Although deep learning codebase, i.e. Pytorch and TensorFlow, provide tools for computing the Second-order derivation, the computation cost is quadratic to the model's parameters, which is thus unacceptable for recent big pre-trained language models (e.g., BERT).

Inspired by the research in [Liu et al., 2018; Chen et al., 2021], we propose an approximation technique for computing Eq. (11). In specific, we employ the Taylor Expansion to rewrite the term ∂LS(ˆ θ(ψ)) ∂ ˆ θ ∂ 2LT (ψ) ∂θ∂ψ in Eq. (11) with Eq. (12).

where ϵ is a small scalar, θ+ and θ- are defined as below:

where LT (θ) is the abbreviation of LT (θ, ψt|B). As demonstrated in [Liu et al., 2018], Eq. (12) would be accurate enough for approximation when ϵ is small. However, computing ▽ψLT in Eq. (12) still requires much computation cost as it involves a forward operation (i.e., LT ) and a backward operation (i.e., ▽ψLT ). To this end, we derive the explicit form of ▽ψLT as Eq. (14) (details are in Appendix B.3).

l1(xi, ˜ yi) and lψ[i] inEq. (14) can be computedwithout gradients, thus preventing the time-consuming back-propagation process. Therefore, computing ▽ψLT with the above explicit form can further reduce the computation cost.

3.4 Annealing Sampling

In domain adaptation, the naive sampling mechanism in the Inner-Loop can suffer from the lowly efficient sampling problem. When the domain shift is large, the model usually performs worse in the target domain than in the source domain. As a result, the model's prediction confidence (i.e., the sampling probability) on the true class is small. Considering an extreme binary classification case, where the instance's ground truth label is [0, 1]t but the model's prediction is [0.99, 0.01]t, the probability of sampling the ground truth label is 0.01. In this case, most of the training cost is wasted on the pseudo instances with error labels.

To improve the sampling efficiency, we propose an annealing sampling mechanism. With a temperature parameter κ, we control the sharpness of the model's prediction probability (sampling probability) by p(·; θ, xi, κ) = softmax( score κ ), where p is the sampling probability and score is the model's original prediction score. In the earlier training phase, the model's prediction is not that reliable, so we set a hightemperature parameter κ to smooth the model's prediction distribution. With this setting, different class labels are sampled with roughly equal probability, which guarantees the possibility of sampling the correct pseudo label. Along with the convergence of the training process, the model's prediction is more and more reliable, thus the temperature scheduler will decrease the model's temperature. We design a temperature scheduler as Eq. (15):

where σ denotes the sigmoid function 4 , κmax and κmin are the expected maximum temperature and minimum temperature, s is a manual set positive scalar. t is the index of the current training iteration, Tmax is the maximum of the training iterations. Thus, t Tmax increases from 0.0 to 1.0, and the input s-2s× t Tmax decreases from s to -s. In our implementation, s is large value that satisfies σ(s) ≈ 1.0 and σ(-s) ≈ 0.0, which gurantees that κt will decrease from κmax to κmin.

4 Theoretical Analysis

Proofs of Lemma 1, Theorem 1, Theorem 2, and Theorem 3 are detailed in Appendix A.

4σ(x) = 1 1+ex , which approaches to 0 when x < -5.0 and saturates to 1.0 when x > 5.0

Lemma 1. Suppose the operations in the base model is Lipschitz smooth, then ℓψ[i] (f(xi, θ), ˜ yi) is Lipschitz smooth with respect to θ for ∀ψ[i] > 1 and ∀xi ∈ DS ⋃ Du T , i.e., there exists a finite constant ρ1 and a finite constant L1 that satisfy:

Also, for ∀ψ[i] > 1 and ∀xi ∈ Du T , ℓψ[i] (f(xi, θ), ˜ yi) is Lipschitz smooth with respect to ψ[i], i.e., there exists a finite constant ρ2 and a finite constant L2 that satisfy:

Assumption 1. The learning rate ηt (line 10 of Algorithm 1) satisfies ηt = min{1, k1 t } for some k1 > 0, where k1 t < 1. In addition, The learning rate βt (line 8 of Algorithm 1) is a monotone descent sequence and βt = min{ 1L , k2 3√t2 } for some k2 > 0, where L = max{L1, L2} and 3√t2 k2 ≥ L.

Based on the Assumption 1 and Lemma 1, we deduce Theorem 1 and Theorem 2. Theorem 1 demonstrates that, by adjusting ψ, the model trained on the target domain can be generalized to the source domain immediately. In other words, by adjusting ψ, the learning process on the target domain (i.e., Eq. (10)) has learned the domain agnostic features. At the same time, Theorem 2 guarantees the convergence of the learning process on the target domain.

Theorem 1. The training process in MTEM can achieve E[‖∇ψLS(ˆ θt(ψt)|DS)‖22] ≤ ϵ in O( 1 ϵ3 ) steps:

where C is an independent constant.

Theorem2. With the training process inMTEM, the instance adaptive Tsallis entropy is guaranteed to be converged on unlabeled data. Formally,

We use hypothesis h : X → ∆K-1 to analyze the effectiveness of MTEM in achieving domain adaptation. Formally, hθ(xi) = argmax k f[k](xi; θ). We let RD(h) denote the model's robustness to the perturbations on dataset D. We let ˆR(H|D) denote the Rademacher complexity [Gnecco et al., 2008] of function class H (h ∈ H) on dataset D. Radmacher complexity evaluates the ability of the worst hypothesis h ∈ H in fitting random labels. If there exists a h ∈ H that fits most random labels on D, then ˆR(H|D) is large. With the above definitions, we deduce Theorem 3.

Theorem 3. Suppose DS and Du T satisfy (q, c)- constant expansion [Wei et al., 2021] for some constant q, c ∈ (0, 1). With the probability at least 1 - δ over the drawing of Du T

from DT , the error rates of the model hθ (h ∈ H) on the target domain (i.e., ϵDT (hθ)) is bounded by:

With Theorem 3, we demonstrate that:

1. Theorem 1 and Theorem 2 prove that MTEM can simultaneously optimize the ψ and θ to minimize LS(h|DS)+ LT (h, ψ|Du T ), i.e., the first two term in Eq. (17).
2. With the bi-level optimization process, the learning process on Du T is regularized by supervised loss on the source domain. As DS does not overlap with Du T , fitting the random labels on Du T cannot carry out the decrease of the supervised loss on the source domain (i.e., LS(θ|DS)). Thus, h ∈ H fits less noise on Du T , reducing ˆR( ˜H×H|Du T ). At the same time, as DS is unseen in the training process, it is also hard to fit the random label on DS, thereby reducing ˆR(H|DS ).
3. Instance adaptive Tsallis-entropy is an unsupervised loss. As accessing unlabeled data is easier than accessing the labeled data, MTEM provides the possibility of sampling a larger unlabeled data to make ζ smaller.
4. RDS∪Du T (h) is a term that can be minimized in the training process technically, e.g., adversarial training [Jiang et al., 2020] or SAM (Sharpness-Aware-Minimization) optimizer [Foret et al., 2020].

5 Experiments

5.1 Experiment Settings

Datasets. On the rumor detection task, we conduct experiments with the dataset TWITTER [Zubiaga et al., 2016], which contains five domains: "Cha.", "Ger.", "Fer.", "Ott.", and "Syd.". On the sentiment classification task, we conduct experiments with the dataset Amazon [Blitzer et al., 2007], which contains four domains: books, dvd, electronics, and kitchen. Preprocess and statistics on the TWITTER dataset and the Amazon dataset can be found in Appendix D.

Comparing Methods. We compare MTEM with previous domain adaptation approaches on both semi-supervised 5 and unsupervised domain adaptation scenarios. Under the unsupervised domain adaptation, we compare MTEM with Out [Chen et al., 2021], DANN [Ganin et al., 2016], FixMatch [Sohn et al., 2020], and CST [Liu et al., 2021]. Under the semi-supervised domain adaptation, MTEM 6 is compared with In+Out [Chen et al., 2021], MME [Saito et al., 2019], BiAT [Jiang et al., 2020], and Wind [Chen et al., 2021]. Out and In+Out are two straightforward ways for realizing unsupervised and semi-supervised domain adaptation, where Out means the base model is trained on the out-of-domain data (i.e., DS) and In+Out means the base model is trained on both the in-domain and the out-of-domain data. DANN realizes domain adaptation by min-max the domain classification loss. CST and FixMatch are self-training approaches that generates pseudo instances to augment domain adaptation. Although CST also involves Tsallis entropy, the entropyindex is a mannually set hyper-parameters and is not instanceadaptative. WIND is a meta-reweigting based domain adaptation approach that learns-to-learn suitable instance weights of different labeled samples in the source domain. More details about the baseline methods can be found in the references.

5 A small set of labeled data in the target domain can be accessed, named in-domain dataset.

6 For semi-supervised domain adaptation, we insert the labeled target data into DS to run MTEM.

Implementation Details. The base model on the Amazon dataset is BERT [Devlin et al., 2019] and the base model on the TWITTER dataset is BiGCN [Bian et al., 2020]. Domain adaptation experiments are conducted on every domain on the benchmark datasets. For every domain on the benchmark dataset, we seperately take them as the target domain and merges the rest domains as the source domain. For example, when the "books" domain in the Amazon dataset is taken as the target domain, the "dvd", "electronics" and "kitchen" domains are merged as the source domain. All unlabeled data from the target domain are involved in the training process, meanwhile the labeled data in the target domain are used for evaluation (with a ratio of 7:3). Since the TWITTER dataset does not contain extra unlabeled data, we take 70% of the labeled data on the target domain as the unlabeled data for training the model and preserve the rest ones for evaluation. The experiments on TWITTER are conducted on "Cha.", "Fer.", "Ott.", and "Syd." 7 . For the symbols in Algorithm 1, we set ηt and βt with respect to Assumption 1.

5.2 General Results

We use all baseline approaches (including MTEM) to adapt BiGCN across domains on TWITTER, and to adapt BERT across domains on Amazon. We validate the effectiveness of the proposed MTEM on both unsupervised and semisupervised domain adaptation scenarios. For semi-supervised domain adaptation scenario, 100 labeled instances in the target domain are randomly selected as the in-domain dataset. As the rumor detection task mainly concerns the classification performance in the 'rumor' category, we use the F1 score to evaluate the performance on TWITTER. On the sentiment classification task, different classes are equally important. Thus, we use the accuracy score to evaluate different models, which is also convenient for comparison with previous studies. Experiment results are listed in Table 1, Table 2.

The results in Table 1 and Table 2 demonstrate the effectiveness of the proposed MTEM algorithm. In particular, MTEM outperforms all baseline approaches on all benchmark datasets. Compared with the self-training approaches, i.e., FixMatch and CST, MTEM maintains the superiority of an average of nearly 2 percent on the Amazon dataset and an average of 4 percent on the TWITTER set. Thus, regularizing the self-training process with an instance adaptative is beneficial. Moreover, MTEM also surpasses the meta-reweighting algorithm, i.e., WIND, by an average of nearly 2 percent on the Amazon dataset and nearly 18 percent on the TWITTER dataset. Thus, the meta-learning algorithm in MTEM, i.e., learning to learn the suitable entropy indexes, is a competitive candidate in the domain adaptation scenario.

7 The labeled data in "Ger." domain is too scare to provide extra unlabeled data.

5.3 Ablation Study

We separately remove the meta-learning module (- w/o M), the temperature scheduler (- w/o T), and the sampling mechanism (- w/o S) to observe the adaptation performance across domains on the benchmark datasets. - w/o M means all instances in the target domain will be allocated the same entropy index (determined with manually attempt). - w/o T means removing the temperature scheduler, and the temperature κ is fixed to be 1.0. - w/o S means to remove the sampling mechanism, i.e., generates pseudo labels with greedy strategies as previous self-training approaches. The experiments are conducted under the unsupervised domain adaptation scenarios. We validate the effectiveness with F1 score on TWITTER, and use the accuracy score on Amazon. The experiment results are listed in Tab. 3 and Tab. 4.

From Tab. 3 and Tab. 4, we can find that all variants perform worse than MTEM on two benchmark datasets: (i) MTEM surpasses MTEM - w/o M on the Amazon dataset with an average of 2 percent, and on the TWITTER dataset with an average of 7 percent. Thus, allocating an instance adaptative entropy index for every unlabeled instance in the target domain is superior to allocating the same entropy index. Furthermore, since the unified entropy index in MTEM - w/o M is searched manually, MTEM - w/o M should be better than Gibbs Entropy. Otherwise, the entropy index would be determined as 1.0 (Gibbs Entropy). Thus, the instance adaptive Tsallis-entropy in MTEM is better than Gibbs Entropy. (ii) MTEM surpasses MTEM - w/o S on the Amazon dataset with an average of 1.4 percent, and on the TWITTER dataset with an average of 1.5 percent, which is attributed to the sampling mechanism can directly correct the model's prediction errors. (iii) MTEM surpasses MTEM - w/o T with an average decrease of 0.9 percent on the TWITTER dataset, and with an average of 0.7 percent on the Amazon dataset, which is consistent with our claims that the annealing mechanism is beneficial to align the domains gradually.

5.4 Computation Cost

We conduct experiments on the Amazon dataset to compare the computation cost in the Taylor approximation and the original Second-order derivation. We separately count the time and the memory consumed in computing the gradient of the entropy indexes with different batch sizes. Experiments are deployed on an Nvidia Tesla V100 GPU. From Fig. 3, the time cost in the Second-order derivation is almost two times higher than in the Taylor approximation, and the memory cost in the Second-order derivation is 3-4 times higher than in the Taylor approximation technique.

We also count the different performances of adapting the BERT model to the 'kitchen' domain with respect to different batch sizes. The experiment results are listed in Tab. 5, where '/' means the memory cost is out of the device's capacity. From Tab. 5, we can observe that the Taylor Approximation technique keeps a similar performance with the Second-order derivation. What's more, the best performance is achieved

1.6

by using a batch with more than 50 instances (the setting in § 5.2), which would exceed the memory capacity if we use the Second-order derivation. Thus, the benefit of reducing the computation cost is apparent, as a larger batch size leads to better adaptation performance.

5.5 Case Study

In Tab. 6, we present two cases with different entropy indexes learned in the meta-learning process (more cases are provided in Appendix. E). Experiments are conducted on sentiment classification tasks, and the settings are the same as 'kitchen' in § 5.2. On the sentences with smaller entropy index (updating the model aggressively), the sentiment words are more transferrable across domains in e-commerce , e.g., 'cheaper' and 'free shipping'. Otherwise, sentences with larger entropy index contain more domain discriminative words, e.g., the ngram 'but too small for soup or stew' in the kitchen domain are less relevant to the other domains (electronics, books, dvd). In this case, MTEM uses a large entropy index to update the model more cautiously.

6 RelatedWork

6.1 Domain Adaptation

To adapt a model to a new domain, feature-alignment approaches [Ganin et al., 2016; Saito et al., 2019; Saito et al., 2019] focus on explicitly aligning the feature space across domains. For example, DANN [Ganin et al., 2016] proposes to align the feature space by min-max the domain classification loss. With similar efforts, MME [Saito et al., 2019] min-max the conditional entropy on the unlabeled data. BiAT [Jiang et al., 2020] proposes to decouple the min-max optimization process in DANN, i.e., firstly maximize the risk loss to obtain a gradient-based perturbation on the input space and then minimize the objective on the perturbed input cases. On the other hand, data-centric approaches use the unlabeled data in the target domain or the labeled data from the source domain to align the feature space implicitly. To select labeled data from the source domain, researchers [Moore and Lewis, 2010] design a technique based on topic models for measuring the domain similarity, while [Chen et al., 2021] takes a meta-learning algorithm to implicitly measure the domain similarity. To exploit the unlabeled data from the target domain, pseudo labeling approaches, including self-training [Zou et al., 2019], co-training [Chen et al., 2011], and tri-training [Saito et al., 2017], are widely applied and become an important direction. The difference lies in that self-training [Zou et al., 2018; Zou et al., 2019; Liu et al., 2021] uses the model's prediction to improve the model, while co-training [Chen et al., 2011] and tri-training [Saito et al., 2017] involves more models which learn the task information from each other. In the research of self-training for domain adaptation, many efforts tried to use prediction confidence to reduce the label noise of pseudo instances [Zou et al., 2018; Zou et al., 2019; Liu et al., 2021], i.e., they preserve only the easy examples that have high prediction confidences while discarding the hard examples that have low prediction confidences. However, fitting the model on easy pseudo instances cannot effectively improve the performance, as the model is already confident about its prediction.

6.2 Meta-Learning

Meta-learning is an emerging new branch in machine learning that aims to train a model that can adapt to a new task or new domain quickly given a few new samples. For this purpose, previous studies tried to learn better initial model parameters [Finn et al., 2017], or better learning rates [Li et al., 2017]. Learning to Compare methods, e.g., relation network [Sung et al., 2018] and prototypical learning [Snell et al., 2017], are investigated more widely in text classification tasks[Tan et al., 2019; Geng et al., 2020]. With the recent success of the pre-trained language model, Network Architect Search (NAS) methods, e.g., DARTs [Liu et al., 2018]), are also widely studied in Natural Language Processing (NLP) tasks [Xu et al., 2021; Dong et al., 2021]. Some metalearning algorithm learns to knowledge distillation [Zhou et al., 2022], i.e., increase the number of teacher models to train a meta-teacher model that works better than a single teacher model. Meta reweighting algorithm [Ren et al., 2018], which proposes to dynamically reweight the risk on different instances, has also inspired some NLP tasks [Li et al., 2020; Chen et al., 2021]. Our research is similar to the meta reweighting algorithm, i.e., the training objective is instance adaptive, the difference lies in that the entropy index controls the loss function on different instances while the instance weights do not change the loss function.

7 Conclusion

This paper proposes a new meta-learning algorithm for domain adaptation on text classification, namely MTEM. Inheriting the principle of entropy minimization, MTEM imposes an instance adaptative Tsallis entropy minimization process on the target domain, and such a process is formulated as a meta-learning process. To reduce the computation cost, we propose a Taylor approximation technique to compute the gradient of the entropy indexes. Also, we propose an annealing sampling mechanism to generate pseudo labels. In addition, we analyze the proposed MTEM theoretically, i.e., we prove the convergence of the meta-learning algorithm in optimizing the instance-adaptative entropy and provide insights for understanding why MTEM is effective in achieving domain adaptation. Extensive experiments on two popular models, BiGCN and BERT, verify the effectiveness of MTEM.

Acknowledgements

This work is supported by the following foundations: the National Natural Science Foundation of China under Grant No. 62025208, the Xiangjiang Laboratory Foundation under Grant No. 22XJ01012, 2022 International Postdoctoral Exchange Fellowship Program (Talent-Introduction Program) under Grant No. YJ20220260.

Contribution Statement

Menglong Lu and Zhen Huang contributed equally to this work. Zhiliang Tian and Yunxiang Zhao are the corresponding authors.