Augmenting Transfer Learning with Semantic Reasoning

1 Introduction

Transfer learning [Pan and Yang, 2010] aims at solving the problem of lacking training data by utilizing data from other related learning domains, each of which is referred to as a pair of dataset and prediction task. Transfer Learning plays a critical role in real-world applications of ML as (labelled) data is usualy not large enough to train accurate and robust models. Most approaches focus on similarity in raw data distribution with techniques such as dynamic weighting of instances [Dai et al., 2007] and model parameters sharing [Benavides-Prado et al., 2017] (cf. Related Work).

Despite of a large spectrum of techniques [Weiss et al., 2016] in transfer learning, it remains challenging to assess a priori which domain and data set to elaborate from [Dai et al., 2009]. To deal with such challenges, [Choi et al., 2016] integrated expert feedback as semantic representation on domain similarity for knowledge transfer while [Lee et al., 2017] evaluated the graph-based representations of source and target domains. Both studies encode semantics but are limited by the expressivity, which restricts domains interpretability and inhibits a good understanding of transferability. There are also efforts on Markov Logic Networks (MLN) based transfer learning, by using first order [Mihalkova et al., 2007; Mihalkova and Mooney, 2009] or second order [Davis and Domingos, 2009; Van Haaren et al., 2015] rules as declarative prediction models. However, these efforts still cannot answer questions like: What ensures a positive domain transfer? Would learning a model from road traffic congestion in London be the best for predicting congestion in Paris? Or would an air quality model transfer better?

In this paper, we propose to encode the semantics of learning tasks and domains with OWL ontologies and provide a robust foundation to study transferability between source and target learning domains. From knowledge materialization [Nickel et al., 2016], feature selection [Vicient et al., 2013], predictive reasoning [L´ecu´e and Pan, 2015], stream learning [Chen et al., 2017] to transfer learning explanation [Chen et al., 2018], all are examples of inference tasks where the semantics of data representation are exploited for deriving a priori knowledge from pre-established statements in ML tasks.

We introduce a framework to augment transfer learning by semantics and its reasoning capability, as shown in Figure 1. It deals with (i) when to transfer by suitable transferability measurements (i.e., variability of semantic learning task and consistent transferability knowledge), (ii) what to transfer by embedding the semantics of learning domains and tasks with transferability vector, consistent vector and variability vector. In addition to expose semantics that drives transfer, a transfer boosting algorithm is developed to integrate the embeddings with existing transfer learning approaches. Our approach achieves high performance for multiple transfer learning tasks in air quality and bus delay forecasting.

2 Background

Our work uses OWL ontologies underpinned by Description Logic (DL) EL++ [Baader et al., 2005][Bechhofer et al., 2004] to model the semantics of learning domains and tasks.

2.1 Description Logics EL++ and Ontology

A signature Σ, noted (NC , NR, NI ) consists of 3 disjoint sets of (i) atomic concepts NC, (ii) atomic roles NR, and (iii) individuals NI . Given a signature, the top concept ⊤, the bottom concept ⊥, an atomic concept A, an individual a, an atomic role expression r, EL++ concept expressions C and D in C can be composed with the following constructs:

A DL ontology is composed of a TBox T and an ABox A. T is a set of concept, role axioms. EL++ supports General Concept Inclusion axioms (GCIs e.g., C ⊑ D), Role Inclusion axioms (RIs e.g., r ⊑ s ). A is a set of class assertion axioms, e.g., C(a), role assertion axioms, e.g., r(a, b), individual in/equality axioms e.g., a ̸= b, a = b. Given an input ontology T ∪A, we consider the closure of atomic ABox entailments (or simply entialment closure, denoted as G(T ∪A)) as {g|T ∪ A |= g}, where g represents an atomic concept assertion A(b), or an atomic role assertion entailment r(a, b), involving only named concepts, named roles and named individuals. Entailment reasoning in EL++ is PTime-Complete.

Example 1. (TBox and ABox Concept Assertion Axioms) Figure 2 presents (i) a TBox T where Road (1) denotes the concept of "ways which are in a continent", and (ii) concept assertions (8-9) with individuals r0 and r1 being roads.

Figure 2: Sample of an Ontology's TBox T and ABox A.

2.2 Learning Domain and Task

To model the learning domain with ontology, we use Learning Sample Ontology and Target Entailment, as in [Chen et al., 2018]. A learning domain consists of an LSO set (i.e., dataset) and a target entailment set (i.e., prediction task).

Definition 1. (Learning Sample Ontology (LSO))

A learning sample ontology O = 〈〈T , A〉, S〉 is an ontology 〈T , A〉 annotated by property-value pairs S.

The annotation S acts as key dimensions to uniquely identify an input sample of ML methods. When the context is clear, we also use LSO to refer to its ontology 〈T , A〉.

Example 2. (An LSO in Context of Ireland Traffic)

Assume an LSO is annotated by property-value pairs S := {topic: Road, road : C Way, country : UK}. Its TBox T includes static axioms like (1); its ABox A includes facts e.g., hasAvgSpeed(r0, Low) that are observed in C Way in UK.

Definition 2. (Learning Domain and Target Entailment)

A learning domain D = 〈O, G Y 〉 consists of a set of LSOs O that share the same TBox T , and target entailments G Y , each of whose truth in an LSO is to be predicted. Its entailment closure, denoted as G(O) is defined as ∪O∈OG(O).

Definition 3 revisits supervised learning within a domain. In a training LSO, a target entailment is true if it is entailed by an LSO, and false otherwise. In a testing LSO, the truth of a target entailment is to be predicted instead of being inferred.

Definition 3. (Semantic Learning Task)

Given a learning domain D = 〈O, G Y 〉, whose LSOs O are divided into two disjoint sets O′ and O′′ , a semantic learning task, denoted by T, within D, is defined as: 〈D, O′ , O′′ , f(·)〉 i.e., the task of identifying a function f(·) with O′ and G Y to predict the truth of G Y in each O in O′′ . Here, O′ is called a training LSO set, while O′′ is called a testing LSO set.

Example 3. (Semantic Learning Task)

Given a domain composed of LSOs annotated by {topic : Road, country : UK} and target entailments Cleared(r0) and Disrupted(r0), the LSOs are divided into a training set O′ and a testing set O′′ according to the type of roads involved, the objective is to identify a function from O′ to predict the condition of road r0, namely the truth of Cleared(r0) and Disrupted(r0) in each LSO in O′′ .

2.3 Transfer Learning Across Domains

Definition 4 revisits transfer learning where DS and DT are called source domain and target domain and their entailment closures are denoted as GS and GT .

Definition 4. (Transfer Learning)

Given two learning domains DS = 〈OS, G Y S 〉 and DT = 〈OT , G Y T 〉, where the LSOs of DT are divided into two disjoint sets O′T and O′′T , transfer learning from DS to DT is a task of learning a prediction function fT|S (·) from OS, G Y S , O′T and G Y T to predict the truth of G Y T in each LSO in O′′ β .

Example 4. (Transfer Learning)

Assume DT is the domain in Example 3, DS is a domain with LSOs annotated by {topic: Road, country: IE}, an example of transfer learning is to identify a function using all the LSOs of Dublin traffic and the training LSOs of London traffic (O′T ) for predicting the traffic condition of road r0 in each testing LSO of London traffic (O′′T ).

We demonstrate how ontology-based descriptions can drive transfer learning from one domain to another. To this end, similarities between domains are first characterized. We adopt the variability of ABox entailments [L´ecu´e, 2015] in Definition 5, where (10) reflects variant knowledge between two domains while (11) denotes invariant knowledge.

Definition 5. (Entailment-based Domain Variability)

Given a source learning domain DS and a target learning domain DT , let G = GS ∪GT , the variability from DS to DT , denoted as ∇(OT , OS) are ABox entailments:

Example 5. (Entailment-based Domain Variability)

Let Figure 3, which capture the contexts in IE and UK, be ontologies of DS and DT respectively. Table 1 illustrates some variabilities of DS and DT through ABox entailments. For instance r1 as a disrupted road in DS is new (variant) w.r.t. knowledge in DT and axioms (1), (9) and (12-15).

3 Transferability

We present (i) variability of semantic learning tasks, (ii) semantic transferability, as a basis for qualifying, quantifying transfer learning (i.e., when to transfer), together with (iii) indicators (i.e., what to transfer) driving transferability. They are pivotal properties, as any change in domains, their transfer function and consistency drastically impact the quality of derived models [Long et al., 2015; Chen et al., 2018].

3.1 Variability of Semantic Learning Tasks

Definition 6 extends entailment-based ontology variability (Definition 5) to capture the learning task variability, where (·)[YS],[YT ] represents using target entailments in (10) (11).

Definition 6. (Variability of Semantic Learning Tasks)

Let TS and TT be semantic learning tasks of source learning domain DS and target learning domain DT . The variability of semantic learning tasks ∇(TS, TT ) is defined by (22), where | · | refers to the cardinality of a set.

The variability of semantic learning tasks (22), also represented by (∇(TS, TT )|O,∇(TS, TT )|Y) in [0, 1], captures the variability of source and target domain LSOs as well as the variability of target entailments. The higher values the stronger variability. The calculation of (22) is in worst case polynomial time w.r.t size OS, OT , YS, YT in EL++. Its evaluation requires (i) ABox entailment, (ii) basic set theory operations from Definition 5, both in polynomial time [Baader et al., 2005].

Example 6. (Variability of Semantic Learning Tasks)

The variability of learning task between TS and TT in Example 4 is (2/3, 0) as the number of variant and invariant ABox entailments are respectively 6 and 3, and YS = YT . i.e., moderate variability of domains, none for target variables.

3.2 Semantic Transferability - When to Transfer?

We define semantic transferability from a source to a target semantic learning task as the existence of knowledge that are captured as ABox entailments [Pan and Thomas, 2007] in the source and have positive effects on predictive quality of the prediction function of the target semantic learning task.

Definition 7. (Semantic ε-Transferability)

Let TS, TT be source, target semantic learning tasks with entailment closures GS, GT . Semantic ε-transferability TS ε ↦→ TT occurs from TS to TT iff ∃S ⊆ OS :

where fT|S (·) is the predictive function fT (·) w.r.t. OT ∪ S. GS is the ABox closures of S.

S is knowledge fromOS, to be used for over-performing the predictive quality of fT (·) with a ε ∈ (0, 1] factor (23) while being new with respect to ABox entailments in GT (24).

Example 7. (Semantic ε-Transferability)

Let TS, TT be semantic learning tasks in DS, DT in Example 4, S be ABox entailment closure of (12-15) in OS, and m(fT|S (·)) > m(fT (·)). Semantic ε-transferability occurs from TS to TT as (i) an ε > 0, satisfying condition (23), exists, and (ii) (24) is true cf. Table 1 w.r.t. S. Thus, knowledge S in IE traffic context (DS) ensures transferability from DS to DT for traffic prediction in UK.

ABox entailments S satisfying Definition 7 are denoted as transferable knowledge while those contradicting (23) i.e., m(fT |S (·)) - m(fT (·)) ≤ ε are non-transferable knowledge as they deteriorate predictive quality of target function fT (·).

Example 8. (Transferable Knowledge)

Consider entailments in S: (i) Disrupted(r4), derived from (13) (19-21), (ii) Cleared(r0), derived from (8) (12) (17-18). As part of knowledge S positively impacting the quality of the prediction task, they are also separate ε-transferable knowledge with max ε: .1, .07 (computation details omitted).

3.3 Consistent Transferable Knowledge

Transferring knowledge across domains can derive to inconsistency. Definition 8 captures knowledge ensuring transferability while maintaining consistency in the target domain.

Definition 8. (Consistent Transferable Knowledge)

Let S be ABox entailments ensuring TS ε ↦→ TT . S is consistent transferable knowledge from TS to TT iff S ∪OT ̸|= ⊥.

ABox entailments S satisfying S ∪ OT |= ⊥ are called inconsistent transferable knowledge. They are interesting ABox entailments as they expose knowledge contradicting the target domain while maintaining transferability. Evaluating if S is consistent transferable knowledge is in worst case polynomial time in EL++ w.r.t. size of S and OT .

Example 9. ((In-)Consistent Transferable Knowledge)

Disrupted(r4) in S of Example 8 is consistent transferable knowledge in TT as {Disrupted(r4)} ∪OT ̸|= ⊥. On contrary Cleared(r0) and Disrupted(r0) in S, derived from (16-18) are inconsistent (7). Thus, Cleared(r0) in OS is inconsistent transferable knowledge in TT .

4 Semantic Transfer Learning

We tackle the problem of transfer learning by computing semantic embeddings (i.e., how to transfer) for knowledge transfer, and determining a strategy to exploit the semantics of the learning tasks (Section 3) in Algorithm 1.

4.1 Semantic Embeddings - How to transfer?

The semantics of learning tasks exposes three levels of knowledge which are crucial for transfer learning: variability, transferability, consistency. They are encoded as embeddings through Definition 9, 10 and 11.

Definition 9. (Transferability Vector)

Let G = {g1, . . . , gm} be all distinct ABox entailments in OS ∪OT . A transferability vector from TS to TT , denoted by t(G), is a vector of dimension m such that ∀j ∈ [1,m]: tj .= εj if gj is εj-transferable knowledge, and tj .= 0 otherwise, with εj | ∄ε∗j , εj < ε∗j and g is ε∗j -transferable knowledge.

A transferability vector (Definition 9) is adapting the concept of feature vector [Bishop, 2006] in Machine Learning to represent the qualitative transferability from source to target of all ABox entailments. Each dimension captures the best of transferability of a particular ABox entailment.

Example 10. (Transferability Vector)

Suppose G .= {Disrupted(r4),Cleared(r0)}. Transferability vector t(G) is (.1, .07) cf. ε-transferability in Example 8.

A consistency vector (Definition 10) is computed from all entailments by evaluating their (in-)consistency, either 1 or 0, when transferred in the target semantic learning task. Feature vectors are bounded to only raw data while transferability and consistency vectors, with larger dimensions, embed transferability and consistency of data and its inferred assertions. They ensure a larger, more contextual coverage.

Definition 10. (Consistency Vector)

Let G = {g1, . . . , gm} be all distinct ABox entailments in OS ∪ OT . A consistency vector from TS to TT , denoted by c(G), is a vector of dimension m such that ∀j ∈ [1,m]: cj = 1 if {gj} ∪ OT ̸|= ⊥, and cj = 0 otherwise

The variability vector (Definition 11) is used as an indicator of semantic variability between the two learning tasks. It is a value in [0, 1] with an emphasis on the domain ontologies and / or label space depending on its parameterization (α, β). We characterize any variability weight above 1/2 as inter-domain transfer learning tasks, below 1/2 as intra-domain.

Definition 11. (Variability Vector)

Let G = {g1, . . . , gm} be ABox entailments in OS ∪ OT . A variability vector v(G, α, β) from TS to TT is a vector of dimension m with α, β ∈ [0, 1] such that vj, j∈[1,m] is:

Example 11. (Variability Vector)

Applying (25) on the variability of semantic learning tasks between TS and TT : (2/3, 0) in Example 6 results in v(G, α, β) = 1/3, which represents moderate variability.

4.2 Boosting for Semantic Transfer Learning

Algorithm 1 presents an extension of the transfer learning method TrAdaBoost [Dai et al., 2007] by integrating semantic embeddings. It aims at learning a predictive function fT|S (·) (line 20) using 〈TS,OS〉, OT for TT . The semantic embeddings of all entailments in GS ∪ GT are computed (lines 7-8). They are defined through transferability, consistency, variability effects from source to target domain. Then, their importance / weight w are iteratively adjusted (line 9) depending on the evaluation of f t (lines 13-14) when comparing estimated prediction f t (ei) and real values YT (gi).

The base model (lines 11-12), which can be derived from any weak learner e.g., Logistic Regression, is built on top of all entailments in source, target tasks. However, entailments from the source might be wrongly predicted due to tasks variability (Definition 6 - line 8) TS, TT . Thus, we follow the parameterization of γ and γt [Dai et al., 2007] by decreasing the weights of such entailments to reduce their effects (lines 17-19). In the next iteration, the misclassified source entailments, which are dissimilar to the target ones w.r.t. semantic embeddings, will affect the learning process less than the current iteration. Finally, StAdaB returns a binary hypothesis (line 20). Multi-class classification can be easily applied.

Algorithm 1: StAdaB(〈DS , TS 〉, 〈DT , TT 〉, O′T , G, L, N, α, β)

• 1 Input: (i) Source/target learning domains and tasks 〈DS, TS〉, 〈DT , TT 〉, (ii) a training LSO set of the target learning domain O′T , (iii) distinct ABox entailments G = {g1, . . . , gm} of OS ∪O′T , (iv) a base learning algorithm L, (v) max. iterations N, (vi) α, β ∈ [0, 1].
• 2 Result: fT |S : A predictive function by DS, TS, O′T , GY T for TT .

3 begin

4 % Initialization of weights for transferability, consistency, 5 % and variability vectors of all mABox entailments in G.

6

Initialization of w1 .= (w1 1, · · · ,w1 3×m);

7 %Computation of semantic embeddings for all gi ∈ G.

8

9

10

11

12

13

14

15

16

17

18

19

20

ei ← (t(gi), c(gi), v(G, α, β)), ∀i ∈ {1, · · · , m};

foreach t = 1, 2, ..., N do % Weight computation iteration pt ← wt/∑3m i=1 wt i ; % Probability distribution of wt .

% Predictive function f t over OS ∪ O′T .

% Error computation of f t on 〈TT , O′T 〉.

%Weights for reducing errors on TT over iteration.

γt ←ψt/(1 -ψt); γ ← 1/(1 + √2 ln(|GS|/N));

%Weight update of source and target entailments in G.

% using γt, γ, and results from previous iteration: wt i .

return Hypothesis ensemble:

A brute force approach would consist in generating an exponential number of models with any combination of entailments from source, target. StAdaB reduced its complexity by only evaluating atomic impact and (approximately) computing the optimal combination. As a side effect, StAdaB exposes entailments in the source which are driving transfer learning (cf. final weight assignment of embeddings).

5 Experimental Results

StAdaB is evaluated by two Intra-domain transfer learning cases: (i) air quality forecasting from Beijing to Hangzhou (IBH), (ii) traffic condition prediction from London to Dublin (ILD), one Inter-domain case: (iii) from traffic condition prediction in London to air quality forecasting in Beijing (ILB). Accuracy with cross validation is reported. All tasks are performed with a respective value of .3, .4, .7 for variability v(G, α, β). α and β are set to .5.

In IBH 1 , air quality knowledge in Beijing (source) is transferred to Hangzhou (target) for forecasting air quality index, ranging from Good (value 5), Moderate (4), Unhealthy (3), Very Unhealthy (2), Hazardous (1) to Emergent (0). The observations include air pollutants (e.g., PM2.5), meteorology elements (e.g., wind speed) and weather condition from 12 stations. The semantics of observations is based on a DL ALEH(D) ontology, including 48 concepts, 15 roles, 598 axioms. 1, 065, 600 RDF triples are generated on a daily basis. 18 (resp. 5) months of observations are used as training (resp. testing). Even though the ontologies are from the same domain, the proportion of similar concepts and roles are respectively .81 (i.e., 81% of concepts are similar) and .74. For instance, no hazardous air quality concept in Hangzhou.

In ILD, bus delay knowledge in London (source) is transferred to Dublin (target) for predicting traffic conditions classified as Free (value 4), Low (3), Moderate (2), Heavy (1), Stopped (0). Source and target domain data include bus location, delay, congestion status, weather conditions. We enrich the data using a DL EL++ domain ontology (55 concepts, 19 roles, 25, 456 axioms). 178, 700, 000 RDF triples are generated on a daily basis. 24 (resp. 8) months of observations are used as training (resp. testing). The concept and role similarities among the two ontologies are respectively .73 and .77.

In ILB, bus delay knowledge in London (source) is transferred to a very different domain: Beijing (target) for forecasting air quality index. Data and ontologies from IBH and ILD are considered. Both domains share some common and conflicting knowledge. Inconsistency might then occur. For instance, both domains have the concepts of City, weather such as Wind but are conflicting on their importance and impact on the targeted variable i.e., bus delay in London and air quality in Beijing. The concept and role similarities among the two ontologies are respectively .23 and .17.

5.1 Semantic Impact

Table 2 reports the impact of considering semantics (cf. Sem. vs. Basic) and (in)consistency (cf. Consistency / Inconsistency) in semantic embeddings on Random Forest (RF), Stochastic Gradient Descent (SGD), AdaBoost (AB). "Basic" models are models with no semantics attached. "Plain" models are modelling and prediction in the target domain i.e., no transfer learning, while "TL" refers to transferring entailments from the source. As expected semantics positively boosts accuracy of transfer learning for intra-domain cases (IBH and ILD) with an average improvement of 13.07% across models. More surprisingly it even over-performs in the inter-domain case (ILB) with an improvement of 20.03%. Inconsistency has shown to drive below-baseline accuracy. On the opposite results are much better when considering consistency for intra-domain cases (63.55%), and inter-domain cases (187.89%).

1 Air quality data: https://bit.ly/2BUxKsi. See more about the application and data in [Chen et al., 2015].

Figure 4 reports the impact of consistency and inconsistency on transfer learning by analysing how the ratio of consistent transferable knowledge in [0, 1] is driving accuracy. Accuracy is reported for methods in Table 2 on intra- (average of IBH and ILD) and inter-domains (ILB). Max. (resp. min.) accuracy is ensured with ratio in [.9, .7) (resp. [.3, .1)). The more consistent transferable knowledge the more transfer for [.9, .1). Interestingly having only consistent (resp. inconsistent) transferable knowledge does not ensure best (resp. worst) accuracy. This is partially due to under- (resp. over-) populating the target task with conflicting knowledge, ending up to limited transferability.

5.2 Comparison with Baselines and Discussion

We compare StAdaB (L = Logistic Regression, N = 800) with Transfer AdaBoost TrAB [Dai et al., 2007], Transfer Component Analysis (TCA) [Pan et al., 2011], TrSVM [Benavides-Prado et al., 2017] and SemTr [Lv et al., 2012] (cf. details in Section 6). We considered intra-domains: IBH, ILD and inter-domains: ILB and ILB∗ (i.e., ILB with same level of semantic expressivity covered by SemTr). Results report that transfer learning has limitations in the Beijing - Hangzhou context cf. Figure 5(a). Although our approach over-performs other techniques (from 10.29% to 50%), accuracy does not exceed 74%. The latter is due to the context, which is limited by the (i) semantic expressivity and (ii) data availability in Hangzhou. The results show that TrSVM and TCA reach similar results (average difference of 9.1%) in all the cases. However our approach and TrAB tend to maximise the accuracy specially in inter-domains ILB and ILB∗ in Figures 5(c) and 5(d) as both favour heterogenous domains by design. Interestingly the semantic context of ILB∗ in Figure 5(d) (i) does not favour SemTr much (+7.46% vs. ILB), (ii) does not have impact for StAdaB compared to ILB, and more surprisingly (iii) does benefit TrAB (+9.15% vs. ILB). This shows that expressivity of semantics is crucial in our approach to benefit from (in-)consistency in transfer.

Adding semantics to domains for transfer learning has clearly shown the positive impact on accuracy, specially in context of inter-domains transfer. This demonstrates the robustness of models supporting semantics when common / conflicting knowledge is shared. The expressivity of semantics has also shown positive impacts, specially when (in)consistency can be derived from the domain logics, although some state-of-the-art approaches benefit from taxonomy-like knowledge structure. Our approach also demonstrates that the more semantic axioms the more robust is the model and hence the higher the accuracy cf. Figure 5(a) vs. 5(b). Data size and axiom numbers are critical as they drive and control the semantics of domain and transfer, which improve accuracy, but not scalability (not reported in the paper). It is worst with more expressive DLs due to consistency checks, and with limited impact on accuracy. Enough training data in the source domain is required. Indeed logic reasoning could not help if important data or features are not mapped to the ontology. This is crucial for training and validation of semantics in transfer learning. Our approach is as robust as other transfer learning approaches, it only differentiate on valuing the transferability at semantic level.

6 RelatedWork

We briefly divide the related work into instance transfer, model transfer and semantics transfer. Instance transfer selectively reuses source domain samples with weights [Dai et al., 2007]. [Tan et al., 2017] select data points from intermediate domains to obtain smooth transfer between largely distant domains. Model transfer reuses model parameters like features in the target domain. For example, [Pan et al., 2011] introduced a transfer component analysis for domain adaption; [Benavides-Prado et al., 2017] selectively shares the hypothesis components learnt by Support Vector Machines. These methods however usually ignore data semantics.

Semantics transfer incorporates external knowledge to boost the above two groups, by using semantic nets [Lv et al., 2012] or knowledge graph-structure data [Lee et al., 2017] to derive similarity in data and features, with no reasoning applied. There are efforts on Markov Logic Networks (MLN) based transfer learning, by using first [Mihalkova et al., 2007; Mihalkova and Mooney, 2009] or second order [Davis and Domingos, 2009; Van Haaren et al., 2015] rules as declarative prediction models. However, these approaches do not address the problem of when is feasible to transfer. Our approach uses OWL reasoning to select transferable samples (addressing 'when to transfer'), then enriching the samples with embedded transferability semantics. It can support different machine learning models (and not just rules).

7 Conclusion

We addressed the problem of transfer learning in expressive semantics settings, by exploiting semantic variability, transferability and consistency to deal with when to transfer and what to transfer, for existing instance-based transfer learning methods. It has been shown to be robust to both intraand inter-domain transfer learning tasks from real-world applications in Dublin, London, Beijing and Hangzhou. As for future work, we will investigate limits and explanations of transferability with more expressive semantics (e.g, based on approximate reasoning) [Pan et al., 2016; Du et al., 2019].

Acknowledgments

This work is partially funded by NSFC91846204.