Rapid Performance Gain through Active Model Reuse

1 Introduction

In traditional machine learning, a learning model is tailored to the target task. Nowadays, however, with the popularity of machine learning, unlike training data which is often hard to share due to the privacy issue [Albrecht, 2016], a great deal of well-trained machine learning models have been available for use. For example, these models have been trained with a large amount of labeled training data [Parkhi et al., 2015]; with smart optimization techniques [Kingma and Ba, 2014]. All these models work well on the specific tasks, but have to be discarded once the target task changes, which causes the user to retrain a newmodel for the target task.

Model reuse [Zhou, 2016] tries to reduce the learning resources for a new target task with the exploitation of pretrained models (as demonstrated in Figure 1), which has attracted much attention and has shown promising performance when the labeled examples are limited for the target task. For example, Yang et al. [2017b] makes predictions for new instances from a collection of different pre-trained models and shows encouraging results; Ye et al. [2018] adapts the pretrained models to a new environment with different features and shows that the performance will be boosted. However, previous model reuse studies usually assume that the labeled data for the target task are passively collected. This leads to a limited performance improvement for the target task, which may be difficult to meet the demands. Since in many practical applications, it is expected that the performance of the target model can be good enough and quickly improved. It is evident that this scenario is quite different from the classic model reuse studies. We call this kind of model reuse the "rapid performance improvement" problem.

The problem is related to but different to standard active learning [Settles, 2012], which tries to select the most informative instances to be labeled. The informativeness is typically defined as maximal expected improvement in classification accuracy [Huang et al., 2010]. Many sample selection criteria (e.g., uncertainty sampling [Lewis and Gale, 1994], query by committee (QBC) [Seung et al., 1992]) have been devoted. However, they ignore the use of many pre-trained models, which results in a slow performance improvement for target tasks or needing a large amount of query costs.

As illustrated in Figure 2, for a new classification task in 20 Newsgroups, we directly employ active learning method (e.g., QBC [Seung et al., 1992]) or model reuse method (e.g., Safer [Li et al., 2017]). As shown, both of the two approaches yield slow performance gains or require a large number of queries to achieve good enough accuracy.

In this paper, we study the rapid performance improvement problem and propose the ACMR (Active Model Reuse) method. As illustrated in Figure 3, unlike traditional active learning, we consider pre-trained models to help construct queries, facilitating active learner when labeled examples are insufficient on the target task. Moreover, we leverage pretrained models to filter out not very necessary queries so that considerable queries could be saved compared with direct active learning. During the learning process, the relationships between pre-trained models and the target task are continually updated such that they can predict unlabeled samples more accurately. Theoretical analysis verifies that ACMR requires fewer queries than active learning. Experiments show the superior performance of the proposed method.

This paper is structured as follows. We first review related works and then present our proposed method. Next we show the experiments. Finally we conclude this work.

2 RelatedWork

Reusability has been emphasized as a crucial characteristic of the new concept of learnware [Zhou, 2016]. It would be ideal if models can be reused in scenarios that are very different from their original training ones. This is of course a big challenge, whereas reusing models have already been demonstrated very useful. Li et al. [2013] has shown that by starting from a trained model, it is easier to construct a model. FMR [Yang et al., 2017a] integrates the discriminative ability of fixed models into deep network training, and achieves promising performance in various applications.

Active learning tries to query the most informative samples and mainly focuses on representative and uncertain information in the unlabeled data [Settles, 2012]. For example, one of the most common strategies is the uncertainty-based selection [Lewis and Gale, 1994], in which the certainties are measured according to the predictions on new unlabeled samples obtained from the initial classifiers. Chakraborty et al. [2015] combines the uncertainty and representativeness into a convex framework to perform active learning loops. Huang et al. [2010] queries the informative and representative samples based on a min-max framework. Wang and Ye [2015] puts the discrimination and representativeness together via a trade-off parameter to query the i.i.d samples.

Transfer learning is one way of implementing model reuse. However, it typically works on a more general problem setting [Pan and Yang, 2010], such as transferring knowledge through data representation [Pan et al., 2011], model structures [Long et al., 2017] and so on. There are several studies on actively transfer learning [Shi et al., 2008], however, they require additional data rather than strict model reuse setting.

In recent years, robust model reuse based on semisupervised learning [Li et al., 2017; Li and Liang, 2019; Li et al., 2019] has been proposed, which works on deriving performance-safe prediction when labeled examples in target tasks are limited, where safe prediction means that the performance would not be worse than direct supervised learning using only limited labeled data. Although these studies are able to achieve prudent performance, the performance improvement is often relatively limited since they never consider involving more labeled data.

3 The ACMRMethod

3.1 Notation and Setting

Let D = {(x1, y1), (x2, y2), . . . , (xnl , ynl ), xnl+1, . . . , xN} be the training data set of the target task that consists of nl labeled instances and nu = N - nl unlabeled ones. Each instance xt = [xt1, xt2, . . . , xtd]⊤ is a vector of d dimensions. L = {(x1, y1), (x2, y2), . . . , (xnl , ynl )} is the labeled data set and U = {xnl+1, . . . , xN} is the unlabeled one. Table 1 summarizes the notations in the paper.

Formally, suppose that we have obtained k pre-trained models {f1, f2, . . . , fk}. We let fL denote the active learner obtained by training a supervised learner with only labeled data. The goal of model reuse is to derive a better model: f + = g({f1, f2, . . . , fk}, fL), which often outperforms, meanwhile would not be worse than fL. g represents the form of model reuse, specifically, with the help of pre-trained models, f + can be build by structural risk minimization:

where LX = {x1, x2, . . . , xnl }, LY = {y1, y2, . . . , ynl }, L is a loss function, e.g., mean square loss, hinge loss, etc and Ω is a regularizer. The smaller the value of the loss function, the better the performance. The forms of the loss function, regularizer and f + are flexible to design for specific tasks. In this paper, we use the hinge loss and logistic regression [Harrell, 2015] to be the form of loss function and f + respectively.

3.2 Deficiencies of Baseline Approaches

Previous works on model reuse typically assume that when labeled examples for the target task are limited, model reuse could guarantee performance improvement [Ye et al., 2018; He et al., 2018]. However, this facilitation is only effective when the number of labeled samples is small and the performance improvement is often relatively limited. To get a good enough model, we still need a lot of queries.

Assume we have obtained confident weights (denoted by η = [η1, . . . , ηk]) of the pre-trained models about the target task, according to the weighted voting method [Dietterich, 2000], the prediction ˆ y(t) of the unlabeled instance xt is

where I(z) = 1 if z is true and 0 otherwise. This prediction, however, may be risky since we can not get accurate prior weights especially when there are very few labeled examples in target tasks [Pan and Yang, 2010]. As illustrated in Figure 2, we consider that the pre-trained models are helpful while their performance is not good enough.

Thus, rather than fully trusting pre-trained models, we consider trusting the pre-trained models partially and leverage pre-trained models to filter out not very necessary queries generated by direct active learning to facilitate an effective active learner. Moreover, once the label capacity is enhanced, we continually update the confident weights of pre-trained models to improve their confidence for target task. It is worthy that the above two steps promote each other in the active learning iterations. In the following, we introduce them respectively and present the theoretical justification.

3.3 Actively Reuse Pre-trainedModels

When the predictions of pre-trained model are risky, we may have the following observations [Shi et al., 2008]: i) Pretrained models assign xt with a class label that is different from the one given by the facilitated active learner; ii) The posteriors of the pre-trained models are low.

According to the above considerations, we can design a query indicator function θ(xt) to reflect the necessity for unlabeled instance xt to query the label:

Pj (ˆ y(t) |xt) is the predicted probability of pre-trained model fj for instance xt. As it can be seen, α(xt) is related to the posteriori probability P(ˆ y(t) |xt), and α(xt) = 0 if the supervised classifier f (t) L and the pre-trained models have assigned different labels to xt. In general, the larger the value α(xt) is, the less we need to query the label.

Moreover, because mislabeling of the few instances can put significant negative effect on accuracy, we further set θ(xt) = (1+α(xt))-1 so as to guarantee the possibility (necessity) to query is greater than 50%. In other words, we trust the label given by the pre-trained models only when its confidence reflected by α(xt) is very high. Accordingly, with the value of θ(xt), we randomly generate a real number R within 0 to 1, and then the decision function F(xt) is defined as

According to Eq.(6), if F(xt) = 0, then it means that the instance xt should be labeled by the pre-trained models; otherwise, xt should be labeled by the domain experts. In other words, we label the instance xt by the pre-trained models with probability 1 - θ(xt).

We then derive its sampling error bound to demonstrate its ability to mitigate the deficiencies for direct approaches, which utilizes the predictions only when they are accurate enough. Additionally, we show its querying bound to validate the claim that ACMR can reduce labeling cost by querying fewer examples. In the algorithm ACMR, let εp and εa denote the expected error of the pre-trained models and the active learner fL respectively, and let δ = εp + εa. The results of theoretical analysis are shown in Table 2.

Theorem 1. In the algorithm ACMR, we assume that εa ≤ εp, then the sampling error ε for ACMR satisfies:

Proof. According to the analysis of the decision function F(x) described above, ACMR makes wrong decision only when both the pre-trained models and the active learner fL agree on the wrong label. In this case, ACMR has probability 1 - θ(x) to trust the classification result given by the pre-trained models, where θ(x) is defined in Eq.(5). Thus, the sampling error of ACMR can be written as ε = εpεa(1 - θ(x)) ≤ ε2p (1 - θ(x)). Moreover, in this situation, θ(x) = 1 1+(1-εp) . Thus,

Theorem 2. In the algorithm ACMR, for an unlabeled instance, the probability that ACMR queries the label from the experts (with cost) satisfies:

Proof. According to the analysis of the decision function F(x), ACMR will query the experts to label the instance when the pre-trained models and the active learner hold different predictions on the classification result. And when the two classifiers agree on the result, it still has probability θ(x) to query the experts. Thus:

Remark. In contrast to Shi et al. [2008] which suffers from exponential term exp(-N -1 ), the results in Theorem 1 and Theorem 2 show that ACMR avoid this issue and improve the bound evidently. From theoretical analysis, we can find that the sampling error and querying probability of the proposed approach are bounded and related to εp and εa. In the next section we will talk about how to reduce εp to improve performance in the whole learning process.

3.4 UpdateWeights for Pre-TrainedModels

It is well known that ensemble learning often outperforms individual models [Zhou, 2012]. However, simply building a uniformly weighted ensemble of the models is suboptimal, since it does not measure the precise relationships between each pre-trained model and the target task.

Inspired by Murugesan et al. [2016], we employ an errordriven update rule in which the weights of pre-trained models

Algorithm 1 The learning algorithm for ACMR

Input: labeled dataset L, unlabeled dataset U, sampling size N, k pre-trained models {f1, f2, . . . , fk} and λ > 0 Output: the model fL for the target task

• 1: Initialize weight vector η = [1/k, ..., 1/k] and fL.
• 2: for m= 1, 2, · · · ,N do
• 3: Select an instance xt(m) from U by traditional active learning
• 4: Predict xt(m) by pre-trained models, and calculate the decision function F(xt(m)) via Eq.(6)
• 5: if F(xt(m)) = 0 then
• 6: yt(m) ← ˆ y(t(m))
• 7: else
• 8: yt(m) ← query from oracle
• 9: if yt(m) ̸= ˆ y(t(m)) then

10:

update the weights via Eq.(12)

• 11: else
• 12: η(m+1) ← η(m)
• 13: end if
• 14: end if
• 15: L = L∪ (xt(m), yt(m)), U = U/(xt(m));
• 16: Train the learner fL with L
• 17: Facilitate active learner fL via Eq.(1)
• 18: end for
• 19: return fL

are updated only when the prediction of pre-trained models is wrong. Specifically, at time m, the target task receives a training instance xt(m), the pre-trained models make a prediction and suffer a loss after yt(m) is revealed. Following the error-driven update rule in which the weights are updated only when the pre-trained models make a mistake, we consider the following optimization problem at each time:

where DKL(η||η(m) ) denotes the Kullback-Leibler (KL) divergence between current and previous soft-attention distributions, which allows η to evolve smoothly over time. The solution for η(m+1) is given in a closed form:

Proposition 1. (Weight Concentration). During the weight update procedure in the whole learning process, the weights will concentrate on those pre-trained models who suffer a small cumulative loss on the target task.

Proof. Through the update rule, we know that the weight associated with the j-th previous model is equal to ηj = exp(-Lj /λ) ∑kj′=1 exp(-Lj′/λ) , j ∈ [k]. The smaller the loss Lj of the pre-trained model, the higher the weight ηj.

We summarize the pseudo-code of ACMR in Algorithm 1.

4 Experiment

In this section, we first give the experimental setup and then show the evaluation of our proposal compared to several state-of-the-art algorithms on a number of real-world tasks.

4.1 Experimental Setup

In order to better validate our method, the following methods are compared in our experiments. Two sample selection criteria: i) QBC [Seung et al., 1992] selects examples that cause maximum disagreement amongst an ensemble of hypotheses; ii) Random selects examples randomly; one actively transfer learning method AcTraK [Shi et al., 2008]; one baseline method Safer [Li et al., 2017].

For ACMR, we choose QBC and Random to implement the base sampling strategy respectively, and we take the pretrained models as the input to ACMR. For example, in the sentiment analysis task of book, we take the pre-trained models in DVD as the input to ACMR; in the classification task of rec.autos and sci.crypt, we take the pre-trained models in "rec.motorcycles and sci.electronics" as the input to ACMR.

Because ACMR need experts to annotate unlabeled samples, we should compare these methods in different number of samples labeled by experts. For each task, we randomly divide the data into two parts: 75% as the unlabeled pool, and the rest 25% as the test set. Experiments are repeated for 30 times, and the average classification accuracy is reported.

4.2 Text Classification Task

The text classification task is collected from 20 Newsgroups 1 . 20 Newsgroups has a two-level hierarchy, for example, the four sub-categories (sci.crypt, sci.electronics, sci.med, sci.space) belong to the top-category sci. Because the difference among top-categories is relatively large and the subcategories under each top-category are similar, we can design the top-categories classification problem. For example, we generate 6 text classification tasks (comp vs. talk, comp vs. sci, comp vs. rec, rec vs. talk, rec vs. sci, sci vs. talk).

1 https://www.cse.ust.hk/TL/

Results are shown in Table 3, which can be seen that ACMR can achieve better performance than other methods. It can be seen that ACMR obtains quite promising performance as the number of queries increases, for various sample selection criteria. Actively transfer learning methods also obtain good performance but is not that competitive as ACMR. This suggests that the use of pre-trained models is beneficial to improve performance rapidly.

4.3 Sentiment Analysis Task

The second task is a sentiment analysis problem 1 , and the goal is to label the documents with respect to what types of problems they describe. We evaluate our algorithm on product reviews from Amazon on a dataset containing reviews from 4 domains: Book, DVD, Electronics and Kitchen. We consider each domain as a binary classification task: reviews with rating > 3 are labeled positive(+), those with rating < 3 are labeled negative(-). For sentiment analysis dataset we have 4 classification tasks.

Results are shown in Table 4, which can be seen that ACMR can significantly outperform its based active learning methods in most cases. Moreover, ACMR also consistently performs better as the number of labeled examples increases. More importantly, we find that even though the performance improvement of pre-trained models is very limited, our proposal can also use them to improve performance rapidly. It also clearly shows the advantage of our proposed approach which is able to exploit useful pre-trained models.

4.4 SpamDetection Task

The last task is a spam detection problem, and we use the dataset obtained from ECML PAKDD Discovery challenge 2 to verify whether our method can help improve the performance. We use the task B challenge dataset which consists of labeled training data from the inboxes of 15 users. Each task is a binary classification problem: spam or non-spam. For spam detection task, we test the top 4 classification tasks.

Results from Table 5 show that ACMR achieves highly competitive performance with compared methods. Once again, our proposal works better than state-of-the-art active learning and Safer model prediction algorithms. This shows that in the frame of active learning, taking pre-trained models into account can get a rapid performance improvement.

2 http://ecmlpkdd2006.org/challenge.html

5 Conclusion

Reusable model design becomes a desire for the rapid expansion of machine learning applications. However, previous model reuse studies assume that the target task receives labeled data passively. This leads to a slow performance improvement to the target task. In this paper, we study a kind of new model reuse problem, where the goal is that the model performance for the target task can be quickly improved. We propose the ACMR method which constructs queries through pre-trained models when labeled examples are insufficient for the target task, and leverages pre-trained models to filter out not very necessary queries. Extending our work into deep learning problem and applying it to more applications are interesting for future study.

Acknowledgments

This research was supported by the National Key R&D Program of China (2018YFB1004300), the National Natural Science Foundation of China (61772262) and the Fundamental Research Funds for the Central Universities (020214380053).