Enabling Abductive Learning to Exploit Knowledge Graph

1 Introduction

The integration of machine learning and logical reasoning is a long-standing holy grail problem of Artificial Intelligence (AI). It is also argued that the next generation of AI calls for integrating the power of machine learning and logical reasoning [Bengio, 2017]. In recent years, Neuro-Symbolic (NeSy) Learning [Garcez et al., 2019; Raedt et al., 2020] and Statistical Relational AI (StarAI) [Raedt et al., 2016] are representative progress. NeSy systems usually aim to build an explainable neural network structure with external domain knowledge. StarAI [Koller et al., 2007] shares a similar idea, where a probabilistic graphical model is constructed based on domain knowledge expressed in first-order logic (FOL). Probabilistic Logic Program (PLP) [De Raedt and Kimmig, 2015] combines these two paradigms by extending FOL to accommodate probabilistic groundings such that probabilistic inference can be conducted.

Abductive Learning (ABL) [Zhou, 2019; Zhou and Huang, 2022] is a flexible framework that integrates a machine learning model with a FOL reasoning model while preserving the full expressive power of both sides: the machine learning model learns to convert input data into primitive logic facts, named pseudo-labels, which serve as input to symbolic reasoning; the reasoning model tries to infer the truth-value of these facts to update the machine learning model. The integration of the two systems adopts abduction, a.k.a. abductive reasoning, to reason about the pseudo-labels based on background FOL knowledge in the reasoning model.

In order to exploit logical reasoning in machine learning, most of the above systems assume a specifically-designed background knowledge base (KB) containing FOL rules for the learning task, which could be unrealistic in real-world tasks. Although recent progress on ABL [Dai and Muggleton, 2021] has shown its capability of learning with incomplete background knowledge, the knowledge bases are still manually prepared by human experts. A specific expert knowledge base would undoubtedly benefit machine learning tasks, however, it could be hard to obtain in reality due to the following reasons: (1) Domain expertise may be very expensive and developing a large knowledge base consisting of abundant and correct rules is time-consuming. (2) These knowledge bases are highly task-specific and can hardly be reused in different tasks.

Recently, knowledge graphs (KGs), as a structured form of human knowledge, have drawn great attention from both academia and industry [Nickel et al., 2015; Wang et al., 2017; Ji et al., 2022; Hogan et al., 2022]. KGs model information in the form of a graph, consisting of entities (nodes) and relations between them (edges). A large number of KGs have been created, including YAGO [Rebele et al., 2016], DBpedia [Lehmann et al., 2015], NELL [Carlson et al., 2010], Freebase [Bollacker et al., 2008], ConceptNet [Speer et al., 2017], etc. They contain millions of nodes and billions of edges [Nickel et al., 2015]. If existing KGs could be used as resources for domain knowledge, from which we extract a segment of knowledge related to the machine learning task, the problem of lacking knowledge can be alleviated to a certain extent. Note that the words knowledge base (KB) and knowledge graph (KG) may share the same meaning in some papers. In this paper, we distinguish between KB and KG, where KG contains triplets and KB consists of FOL rules.

Three challenges hinder machine learning systems from exploiting knowledge in KGs. First, large-scale and generalpurpose KGs often contain much information irrelevant to the machine learning task, causing inefficiency or inconsistency in reasoning. Second, a KG that is not tailored for the learning task may use different expressions for entities and relations, bringing semantic challenges in entity matching, e.g., "father" in the KG but "dad" in the machine learning task. Third, largescale KGs are usually mined from data, the noise within could bring obstacles to rule extraction and logical inference.

This paper proposes to tackle the above challenges under the abductive learning (ABL) framework and presents an approach to facilitate ABL systems with knowledge graphs. Given a learning task with a large-scale and general-purpose KG, the proposed ABL-KG (ABductive Learning with Knowledge Graph) first mines logic rules from a knowledge sub-graph relevant to the learning task. Then, it adapts the rules for the learning task by aligning their names. We propose a remembering algorithm to extract and transform the mined rules to a weighted logic program while resolving the contradictions caused by KG noise. Finally, ABL-KG leverages the logic program as background knowledge in ABL by utilizing a consistency measure to handle noisy and uncertain FOL rules.

We assess the effectiveness and generalization of ABL-KG on a diverse range of tasks, including a classification task on tabular data, two representation learning tasks on graph data, and a classification task on image data. Experiments show that ABL-KG can exploit the knowledge in KGs without manual efforts, and improve machine learning performance by automatically extracting task-relevant domain knowledge from large-scale and general-purpose KGs.

2 RelatedWork

Probabilistic Logic Program (PLP) [De Raedt and Kimmig, 2015] and Statistical Relational Learning (SRL) [Koller et al., 2007; Raedt et al., 2016] are two typical paradigms for integrating logical reasoning and machine learning. PLP extends FOL to accommodate probabilistic groundings and conduct probabilistic inference. SRL tries to construct a probabilistic graphical model based on domain knowledge expressed in FOL. Various novel approaches have been proposed in recent years, including DeepProbLog [Manhaeve et al., 2018], Abductive Learning [Zhou, 2019] and NGS [Li et al., 2020].

A knowledge graph (KG) has a multi-relational graph structure, with each node representing an entity and each edge indicating the relation between two connected entities. Such structured commonsense or factual knowledge has been widely used in intelligent applications [Ji et al., 2022]. There are two typical approaches to exploiting KGs. The conventional symbolic methods either mine logic rules to infer new facts [Gal´arraga et al., 2013; Gal´arraga et al., 2015; d'Amato et al., 2016], or use keyword or SPARQL queries executed on a KG to retrieve answers [Unger et al., 2012]. Recently, representation learning can capture KG semantics in vector space and use the acquired embeddings to improve downstream tasks such as link prediction [Wang et al., 2017] and entity alignment [Sun et al., 2020].

Some approaches attempt to combine deep neural networks with KGs [Marino et al., 2017; Wang et al., 2018] by building an end-to-end graph neural network (GNN), which often demand a large number of labeled data for training and the expressiveness of knowledge is limited. Instead of using GNN, Scallop [Huang et al., 2021a] scales up DeepProblog by using approximate probabilistic logic inference, and it can use small-scale KGs as knowledge resources. Different from these approaches, ABL-KG could leverage reasonably large-scale KGs with unlabeled data by abductive learning and preserve the expressiveness of FOL at the same time.

The ability to discard irrelevant information is a key feature for an intelligent agent, which is referred to as forgetting [Lin and Reiter, 1994]. Forgetting preserves all logical consequences of relevant symbols while removing the irrelevant ones, which enables a logic program to adequately and efficiently handle reasoning tasks such as query answering. Various forgetting operations [Lin and Reiter, 1994; Wang et al., 2005; Delgrande, 2014] have been proposed for different types of logic programs. For a comprehensive overview of forgetting, please refer to [Eiter and Kern-Isberner, 2019]. While these works usually focus on forgetting a small subset of irrelevant information in a logic program, our work would discard a large proportion of them efficiently.

3 Preliminaries

3.1 Abductive Learning

Abductive Learning (ABL) [Zhou, 2019; Zhou and Huang, 2022] is a framework that integrates a machine learning model and a logical reasoning model. The machine learning model f maps the unlabeled input data x ∈ Xu into discrete symbols y ∈ Y, which are called pseudo-labels since no supervision on y. The reasoning model receives the symbols y and verifies their consistency with a knowledge base KB of FOL rules. If inconsistent, the reasoning model would correct the pseudo-labels y to abduced labels ¯ y by abductive reasoning. For instance, given a crayfish's features x, an under-trained classifier f may output the pseudo-label y = fish, then the reasoning model is expected to identify the inconsistency and revise it to ¯ y = crustacean.

Abductive reasoning (abduction) is a form of logical inference that seeks grounded facts explaining observations based on background knowledge. To illustrate clearly, this paper denotes logical symbols as follows: "∧" is conjunction (and); "∨" is disjunction (or); "←" is implication, which means that if premises on the right of "←" hold, then the conclusion on the left holds. For example, consider a KB containing the rules:

where the first two rules describe the characteristics of fish, insect and crustacean, and the last rule specifies that an animal cannot be both an insect and a crustacean simultaneously. Given the observation that an animal (e.g., a crayfish)

.

has no backbone, Rule (1) implies that it should not be a fish (inconsistent with the pseudo-label). Continuing this reasoning process, based on Rules (2) and (3), both insect and crustacean could be two possible explanations. If other rules in KB indicate that it is not an insect, then a crustacean would be the only explanation (abduced label).

Usually, there are multiple candidate abduced labels (denoted by ¯ Y), and ABL minimizes inconsistency to choose the best ¯ y ∈ ¯ Y based on a consistency measure. After revising y to ¯ y, ABL treats them as ground-truth labels to update the machine learning model f. The above process repeats iteratively and f improves its performance by performing abduction on the unlabeled data Xu with knowledge base KB.

3.2 Knowledge Graph

In this work, we focus on the RDF (Resource Description Framework) KGs [Schreiber and Raimond, 2014]. It is a structured representation of facts regarding entities and relations. Each fact is a triplet of the form (h, r, t), e.g., (bird, HasA, feather). Subject h and object t are entities, which can be real-world instances or concepts, and r is their relation. In predicate logic, (h, r, t) is equivalent to r(h, t).

3.3 Forgetting and Remembering

Forgetting, originally proposed in [Lin and Reiter, 1994], aims at removing irrelevant information from a knowledge base KB. In this paper, we denote the signature (i.e., all the predicates) of KB by Σ, and Σ′ ⊆ Σ is the part to forget. forget(KB,Σ′ ) is the result of forgetting Σ′ from KB. The dual of forgetting is remembering [Lin and Reiter, 1994], denoted by remember(KB, Σ′ ) = forget(KB, Σ \ Σ′ ), which means forgetting the remaining signature while preserving all logical consequences in KB w.r.t. Σ′ .

For instance, assume that KB consists of two rules backbone(X) ← vertebrate(X) and vertebrate(X) ← fish(X). The predicate "vertebrate" is irrelevant to the learning task and we want to forget it. In this case, signature Σ = {f ish, vertebrate, backbone} and Σ′ = {vertebrate}, and the result of forget(KB,Σ′ ) is {backbone(X) ← fish(X)}. Obviously, forgetting vertebrate not only keeps information of the remaining signature, but also leads to a new knowledge base that could perform reasoning more efficiently.

4 The ABL-KG Framework

4.1 Problem Setting

We are given unlabeled data Xu, a large-scale knowledge graph KG, and an initial machine learning model f. To utilize the semantics of data, a task specification S of Xu is available, e.g., the set of class names and attribute names such as bird or backbone in a classification task. The goal is to leverage Xu and KG to improve the performance of f.

4.2 Framework

The ABL-KG framework can be roughly divided into three main steps: rule mining, remembering, and abductive learning with KG rules. Figure 1 and Algorithm 1 show its outline.

Rule Mining. Given a large-scale knowledge graph KG and a machine learning task, KG contains numerous irrelevant information to the task. Therefore, ABL-KG first extracts a sub-graph KG ′ ⊂ KG relevant to the machine learning task, where the number of triplets |KG ′ | ≪ |KG|. The relevance is measured by the semantics of the relations and entities in KG and S. Then, a rule mining algorithm is employed to mine FOL rules from the sub-graph KG ′ , which forms the initial knowledge base KBmine. The upper part of Figure 1 shows an illustration, further details are introduced in Section 4.3.

Algorithm 1 Abductive Learning with Knowledge Graph

Input: Knowledge graph KG; Unlabeled data Xu; Task spec-

ification S; Initial model f ; Labeled data Xl

Output: Machine learning model f

• 1: KG ′ ← SubGraph(KG, S, d) // Sub-graph depth d
• 2: KBmine ←MineRules(KG ′ )
• 3: Σ′ ← SigMatch(KBmine, S)
• 4: KB ←Remember(KBmine,Σ′ )
• 5: KB ←PostProcess(KB,Xl)
• 6: for t = 1 to T do
• 7: Yu ← f(Xu)
• 8: ¯ Yu ←Abduce(Yu,Xu,KB) // Solve Eq. (4)
• 9: f ←Update(f,Xu, ¯ Yu) // Update f with ¯ Yu
• 10: end for

Remembering. To preserve the connectivity of the extracted sub-graph, the FOL rule set KBmine may still contain some predicates not matched to the names in the task specification S, and hence, it could further be reduced. As shown in the top right of Figure 1, the predicates backbone, fish, legs, bird and tails should be directly mapped to S, while the predicate vertebrate has no matched words in S and can be removed. Thus, we propose a remembering algorithm to do this. The new KB only consists of the target predicates (signature) Σ′ without the loss of other information, e.g., keeping the relation between predicates fish and backbone as shown on the right side of Figure 1. In addition, since KG inevitably contains noise, we design a strategy to handle contradictory rules in KB during the remembering process. More details are introduced in Section 4.4.

Abductive Learning withKGRules. The lower part of Figure 1 and Lines 6-10 in Algorithm 1 show the basic procedures in abductive learning (ABL). Here a key challenge is how to utilize the noisy KB in ABL. If a small amount of labeled data is available, ABL-KG can use it to post-process and filter unreliable rules in KB. Besides, in abductive learning, one needs to minimize the inconsistency between abduced labels and KB. Due to the inherent noise in KB, a fully consistent abduced label may not even exist. Therefore, we propose to formulate KB as a weighted logic program and introduce a new consistency measure to handle the uncertainty in the abductive reasoning process. Details can be found in Section 4.5.

4.3 Rule Mining

Sub-graph Extraction

Sub-graph extraction aims at extracting a sub-graph KG ′ of KG relevant to the given machine learning task. Let R = πrelation(KG) denote the set of relations in KG and E = πentity(KG) the set of entities. ABL-KG first finds a set ER ⊆ R ∪ E of entities and relations in KG that have the same names as elements in task specification S. Then, a sub-graph KG ′ = {(h, r, t) ∈ KG | {h, r, t} ∩ ER ̸= ∅} consists of the triplets that contain the relations or entities in ER. Furthermore, we could construct a larger KG ′ based on current KG ′ by iteratively performing the above procedure. ABL-KG limits the size of the sub-graph KG ′ by a parameter d, which controls the depth of graph expansion.

Rule Mining for Knowledge Graph

ABL-KG could use any rule mining algorithms designed for KG to mine association rules from the extracted subgraph KG ′ . The outputs (knowledge base KBmine) are typically weighted first-order Horn rules. Each rule's literals are triplets with variables, e.g., isCitizenOf(C, S) ← hasChild(P, C) ∧ isCitizenOf (P, S) with confidence 0.9 estimated from data provided by the rule mining algorithm.

We also propose a simple strategy that directly converts triplets in KG ′ into rules. We first define a subset of relations in KG ′ related to the learning task, and the corresponding direction of implication and confidence in logic rules. Then, the entities are directly converted into predicates with the same name. If a number exists in an entity, it serves as the second argument in its corresponding atom. For example, the triplets (fish, IsA, vertebrate) and (bird, HasA, two legs) can be transformed into rules vertebrate(X) ← fish(X) and legs(X, 2) ← bird(X), respectively. The confidence of the mined rules is generated by a user-defined tolerance w.r.t. the relation in the triplets to reflect the estimated noise level in KG ′ , and details are provided in the experiment section.

4.4 Remembering

Signature Matching

Let Σ be the signature in the mined knowledge base KBmine. Then, the target signature (signature to remember) is Σ′ = {σ ∈ Σ | MaxSimilarity(σ, S) > τ}, where MaxSimilarity() returns the maximum similarity between predicate σ and the elements in the task specification S, estimated fromword embeddings, and τ is a threshold.

Remembering Algorithm

We propose a remembering algorithm that produces a new knowledge base KB from KBmine that only contains the target signature Σ′ without any loss of other information. The signature to remember in ABL-KG is often much smaller than the entire signature of KBmine, i.e., |Σ′ | ≪ |Σ|.

Algorithm 2 shows an outline. The algorithm starts from a subset Rnew of input KBmine, where the signature Sig(r) of each rule r contains at least one element of the target signature Σ′ . In the outer loop (cf. Lines 3-16), it derives a set of new clauses Rres by resolving every clause in Rnew and every clause in KB w.r.t. the signature to forget (Σ \ Σ′ ), i.e., ResolutionOn(rnew, r, Σ\Σ′ ) = {α∨β | rnew = α∨p, r = β ∨ ¬p, p ∈ Σ \ Σ′ }. If resolution succeeds and the resolvent is not in Rres, it is added to Rres and the current KB. The resolvent's confidence would be set as the product of the confidence of the two resolved clauses. In the next iteration, the set of new resolvents Rres will be used as Rnew and resolves with the current KB. The above routine conducts reasoning related to Σ′ on KB until saturated, i.e., there exists no "hidden" information about Σ′ that is entailed by KB but not directly asserted in it. Finally, a new knowledge base is returned by retrieving the clauses in the current KB that only contain Σ′ .

The basic idea of our remembering algorithm is to avoid unnecessary resolutions by restricting one of the clauses in resolution to contain target signatures. For example, given a knowledge base KBmine = {a ← b, b ← c, d ← e, e ← d}, we want to remember literals a and c, and the result is {a ← c}.

Algorithm 2 Remembering

Input: Knowledge base KBmine; Entire signature Σ; Target signature Σ′ Parameter: Iteration limit T Output: Reduced knowledge base KB 1: Rnew ← {r ∈ KBmine | Sig(r) ∩Σ′ ̸= ∅} 2: KB ←KBmine 3: while t ≤ T and Rnew ̸= ∅ do 4: Rres ← ∅ 5: for rnew ∈ Rnew do 6: for r ∈ KB do 7: rres ← ResolutionOn(rnew, r, Σ \ Σ′ ) 8: if rres ̸= ∅ and rres ̸∈ KB then 9: Rres ← Rres ∪ {rres} 10: end if 11: end for 12: end for 13: Rnew ←Rres 14: KB ←KB ∪Rres 15: t ← t + 1 16: end while 17: KB ← {r ∈ KB | Sig(r) ⊆ Σ′ }

If resolving on b, d, e succeeds, the resolution on d and e would be unnecessary because they are completely irrelevant to a and c. In short, our algorithm improves the efficiency of forgetting. Moreover, even if irrelevant symbols are unsatisfiable, e.g., KBmine becomes {a ← b, b ← c, d, ¬d}, it would not influence the ABL-KG framework, since we only care about the relations within the target signature Σ′ = {a, c}.

Theorem 1. Let KBmine be any finite consistent propositional or grounded first-order answer set program [Lifschitz, 2002] consisting of Horn clauses, if the iteration limit T is sufficiently large, Algorithm 2 always terminates and returns a result of forgetting about Σ \ Σ′ in KBmine.

We defer the proof to Appendix. Theorem 1 shows that the algorithm can output a correct logic program of forgetting.

Contradiction Handling

The rules mined from KG ′ may contain contradictory rules. For example, animal(X) ← bird(X) and ¬animal(X) ← bird(X) may both exist in KB (due to (bird, IsA, animal) and (bird, Antonym, animal) in ConceptNet [Speer et al., 2017]). They would result in an incorrect rule ¬bird(X) after remembering, which means that everything is not a bird. We define contradictory rules as follows: if both p ← α and ¬p ← α exist in KB, then they are contradictory rules. ABLKG detects contradictions during the remembering process. If r1 and r2 are contradictory rules, it removes r1, r2 and their resolvents from the current KB.

4.5 Abductive Learning with KGRules

Rule Post-processing

If a small amount of labeled data are available, they could be used not only for training the initial machine learning model f , but also for rule post-processing. By evaluating the quality of rules, ABL-KG drops low-quality rules and adjusts the rules'

confidence to β ·confrule+(1-β)·confdata, where confrule is the rule's confidence given by KG, and confdata given by labeled data.

Consistency Measure for Noisy and Uncertain Rules

Previous ABL methods adopt a consistency measure for abduction to find the best abduced labels, which usually requires that the abduced labels should be consistent with KB, i.e., all the rules in KB are satisfied. However, faced with inevitably noisy KG, even though the mined rules are post-processed, they would not always be satisfied (unless all of them have confidence 1.0), in which case the abduced labels that are consistent with KB may not even exist. Therefore, we propose a new consistency measure for noisy and uncertain rules, where the optimization problem can be formalized as:

where ¯ Y is the set of candidate abduced labels, and x and ¯ y are unlabeled data and abduced labels, respectively.

ModelScore(f, x, ¯ y) represents the consistency between abduced labels and model. For example, it can be defined as:

where Conf(xi, ¯ yi) denotes the confidence by model f that sample xi belongs to label ¯ yi. Alternatively, other consistency measures, e.g., [Huang et al., 2020; Cai et al., 2021; Huang et al., 2021b; Huang et al., 2023], could also be used.

KBScore(KB, x, ¯ y) represents the consistency between abduced labels and knowledge base KB, which is defined as:

where r is a ground rule in KB which is inconsistent with x, ¯ y, i.e., r ∪ x ∪ ¯ y is inconsistent. A ground rule is a rule where all variables are replaced by specific instances. Take the example in Figure 1, if abduced label ¯ y is fish, the ground rule backbone(x) ← fish(x) is violated and therefore inconsistent with abduced labels. Weight(r) is the weight of rule r, which can be calculated based on the rule's confidence, e.g., 2 · confrule - 1 in our framework.

The measure of ModelScore in Eq. (5) prefers abduced labels not far from the model's current prediction. The measure of KBScore in Eq. (6) shares a similar form of weighted maximum satisfiability (Weighted MAX-SAT) problem, where it prefers abduced labels that have a small combined weight of inconsistent rules. The weighting coefficient α in Eq. (4) combines these two scores to form the final consistency measure. When optimizing Eq. (4), if the search space is large, we can solve with derivative free optimization [Yu et al., 2016]. Otherwise, as in this work, we can directly search all candidates.

Our proposed consistency measure is general, which can be regarded as a generalization of previous measures:

Proposition 1. The consistency measure of previous abductive learning approaches [Dai et al., 2019; Cai et al., 2021; Huang et al., 2021b] is a special case of Eq. (4) where KBScore = 0.

This conclusion is clear because previous methods require the abduced labels to be consistent with the whole knowledge base, and therefore there are no inconsistent ground rules.

5 Experiments

This section presents the experimental results on three different types of tasks, including a task on tabular data, two tasks on graph data and a task on image data, to demonstrate that ABL-KG is a general framework that can automatically extract domain knowledge rules from large-scale KGs and leverage them for improving the performance of machine learning models. The code is available for download 1 .

5.1 Animal Classification

The zoo animal classification dataset [Dua and Graff, 2017] contains animals' attributes (e.g., backbone, legs) and their categories (e.g., bird, fish), along with the names of each attribute and class (task specification). In our experiment, we use only one labeled sample of each class for model initialization. The remaining is randomly split into 70% unlabeled data and 30% test data. We use ConceptNet [Speer et al., 2017], a largescale semantic network with 34 million edges, as our KG. We use GloVe [Pennington et al., 2014] for signature matching (cf. Section 4.4). Rules are mined using our algorithm in Section 4.3, where d is set to 2, and we set the confidence of IsA relation to be 1.0 and others 0.9.

The left part of Table 1 shows the test accuracy. Here, the random forest (RF) [Breiman, 2001] is used as the classifier in all methods. ABL-KG is compared with two types of baselines, i.e., RF using only labeled data and semi-supervised learning methods leveraging unlabeled data, including Pseudo-Label (PL) [Lee, 2013] and Tri-training (Tri) [Zhou and Li, 2005]. It is obvious that the performance of ABL-KG is significantly superior to other approaches. Intriguingly, the performance of ABL-KG is close to that of ABL with manually designed rules, which take an expert about one hour to write. Since all rules come from the mining of ConceptNet, it demonstrates that ABL-KG is able to leverage a large-scale and noisy KG for the training of a machine learning model.

The ablation studies are presented on the right part of Table 1. The performance of ABL-KG drops if skipping any of the "remembering", "contradiction handling" and "postprocessing" steps, indicating that all procedures play a significant role in ABL-KG. Specifically, without "contradiction handling", ABL-KG reaches the lowest performance. We check the generated rules in this case, and find that they contain some false rules, e.g., "¬bird(X)" ("nothing is a bird"), which would result in great performance degradation. Further analysis reveals that this is caused by forgetting animal on two contradictions from KG, i.e., "animal(X) ← bird(X)" and "¬animal(X) ← bird(X)". Contradiction handling would remove these contradictory rules and avoid this error.

1 https://github.com/AbductiveLearning/ABL-KG

We analyze the rules after each step, as shown in Table 2. Starting from 34 million triplets, ABL-KG gradually mines useful rules and drops irrelevant ones, leading to the improvement of abduced label accuracy. We compare the mined rules with the expert's ones. Although the former lacks some domain knowledge due to KG incompleteness, most of the rules written by the expert have been discovered by ABL-KG.

5.2 Entity Alignment

The task seeks to find the identical entities from two KGs (input data). The entity alignment model learns embeddings for the two KGs to measure entity similarity. In our experiment, we consider the widely-used cross-lingual dataset DBP-ENFR, which was proposed in the OpenEA benchmark study [Sun et al., 2020]. It aims to align the entities in the English and French DBpedia [Lehmann et al., 2015]. It has 15000 entity alignment pairs and we use 20% of them as training data. We choose the popular entity alignment model AlignE [Sun et al., 2018] as the basic aligner in our experiment, and implement two semi-supervised variants, i.e., AlignE+ and AlignE++, as baselines. AlignE+ employs self-training [Yarowsky, 1995] and selects the predicted entity alignment pairs whose embedding similarity is greater than 0.9 to augment training data. AlignE++ improves AlignE+ by using entity dependencies to reduce the noise in augmented training data [Liu et al., 2023]. We use Wikidata [Vrandecic and Kr¨otzsch, 2014] and YAGO3 [Rebele et al., 2016] as the background KGs to acquire a KB for entity alignment, and use GloVe [Pennington et al., 2014] for relation matching during remembering.

We report the Hits@1, Hits@10, and MRR results of fivefold cross-validation in Table 3. Our ABL-based methods, ABL-KG and ABL-Expert, bring significant improvement to AlignE. They also outperform the semi-supervised variants. This is because by leveraging mined/manual knowledge base, abductive reasoning can identify and resolve the alignment inconsistency issue, which improves the quality of new training data. An example of the automatically mined rules in ABLKG is "sameAs(X, Y ) ← spouse(X, A) ∧ f ather(C, B) ∧

mother(C, Y ) ∧ sameAs(A, B)", which means that a person's father and mother are spouses. In the abductive learning stage, these rules form a logic program that can check the dependency of predicted entity alignment pairs and resolve alignment inconsistency. ABL-Expert achieves better performance than ABL-KG, because the expert rules are superior to our automatic rule mining method in terms of rule noise.

5.3 Link Prediction

Link prediction is the task of predicting the missing entity of an incomplete triple like (IJCAI-2023, location, ?). We consider the benchmark dataset FB15K-237 [Toutanova and Chen, 2015] and choose the most popular model, TransE [Bordes et al., 2013], as the basic learning model in the experiment. Given a triplet, TransE interprets the relation as a translation vector between the subject and object. The translation error is defined as the energy of the triplet. We also use self-training to implement a semi-supervised baseline, denoted by TransE+. We mine rules from DBpedia [Lehmann et al., 2015] and use GloVe [Pennington et al., 2014] for relation matching in the remembering process to build a KB targeted to FB15K-237.

We follow the training/validation/test data splits of FB15K237, and report the average test results of five runs in Table 4. We find that the semi-supervised method TransE+ even underperforms TransE. This is because link prediction is a difficult task and the low accuracy results in much noise in the abduced data, which hurts the model training. By contrast, both ABL-KG and ABL-Expert offer performance improvement. The rules extracted from other background KGs can provide additional capability for accurately inferring new triplets and correcting incorrectly predicted triplets. It is noteworthy that the performance of ABL-KG is close to that of ABL-Expert, showing that the automatically mined rules in ABL-KG have a similar quality to the expert's rules, which could reduce the human efforts on knowledge engineering.

5.4 Image Classification

The task's input is images fromADE20K [Zhou et al., 2017]. We reform it as a multi-label classification task, which requires a model to determine the objects (e.g., bed, sidewalk) in images. From totally 150 categories of objects, we randomly select 12 frequently appearing objects as labels. In our task, only 5% or 10% of all 20k images are labeled. Each image, labeled or not, comes with a scene description (e.g., bedroom, street), with 1055 possible scenes. ABL-KG uses ConceptNet [Speer et al., 2017] as the KG, and the confidence of mined rules is set to 1.0 for the IsA relation and 0.8 for the rest. In addition, we use an initial knowledge base that contains only 40 rules to cover the incompleteness of ConceptNet, and the final KB contains 590 rules. All methods use a ResNet-50 [He et al., 2016] pre-trained on ImageNet as the learning model, in which the last layer is replaced to fit the multi-label task, and the model is fine-tuned on the given labeled and abduced labels.

Table 5 presents the F1-scores (macro-averaging) of compared approaches, including supervised (ResNet [He et al., 2016]) and semi-supervised methods (Pseudo-Label (PL) [Lee, 2013] and Virtual Adversarial Training (VAT) [Miyato et al., 2018]). With different label rates, ABL-KG achieves the highest F1. When there are fewer labeled data, the performance gain of ABL-KG compared with supervised methods is greater. An ablation study only using the initial knowledge base (ABLInit) indicates that the performance gain mainly comes from the mined KG rules. The learning curves of various approaches are shown in Figure 2. Starting with the same performance, ABL-KG achieves higher accuracy than other methods.

6 Conclusion

Previous works that integrate machine learning and logical reasoning usually require a manually engineered knowledge base. To reduce this burden, we propose to exploit existing large-scale knowledge graphs as knowledge resources. In this paper, we propose the ABL-KG framework which can extract an interpretable knowledge base automatically in the form of logic rules and use them for abductive learning.

Experiments on various tasks validate that the extracted knowledge contributes to an improvement of machine learning models under the abduction strategy that employs the consistency measure designed for noisy and uncertain rules. The limitation of ABL-KG is that it assumes the existing knowledge graphs contain the needed knowledge, which may not always be sufficient due to the limited expressive power of triplets. In future work, it would be intriguing to exploit other knowledge representation forms such as ontology.

Acknowledgments

This research was supported by NSFC (61921006, 62272219, 62206124).