LLM-based Multi-Level Knowledge Generation for Few-shot Knowledge Graph Completion

1 Introduction

Knowledge Graphs (KGs) are structured databases representing information through entities and their interrelations, typically organized as triples comprising a head entity, a relation, and a tail entity [Chen et al., 2024]. KGs play a pivotal role in diverse NLP tasks, including question answering [Chen et al., 2021b; Chen et al., 2022], entity search [Gerritse et al., 2022], and recommendation systems [Du et al., 2022]. Despite their extensive utility, KGs frequently confront the challenge of incompleteness, manifested in missing or underrepresented entities and relations. Knowledge Graph Completion (KGC), synonymous with link prediction in the context of this work, focuses on identifying and inferring potential missing triples in KGs to address existing gaps. However, the long-tail problem, a common issue in real-world KGs, complicates this task. It arises from the significant variance in the frequency of relationship types and the disparity in the number of neighbors among different entity types. This results in many relations being sparsely represented, leading to inadequate training data for traditional KGC methods. To overcome this, the focus has shifted towards Few-shot Knowledge Graph Completion (FKGC).

∗Corresponding Author.

FKGC is designed to enhance KG completion in scenarios with limited data availability. It involves predicting missing entities in triples with rare relations (the query set) using a minimal number of example triples containing the same relation (the support set). This approach aligns with fewshot learning methodologies in KGs, addressing the critical challenge of data sparsity in KG augmentation. Current methods mainly focus on model-level enhancements. This includes strategies such as utilizing paths between entities to capture complex relations and their interactions, which leverages the local sub-graph structure for for enriched entity learning [Xu et al., 2021]. Additionally, employing a modelagnostic meta-learning framework is also proved to be helpful in distilling relation-specific information [Chen et al., 2019; Niu et al., 2021].

In this paper, amidst the rapid advancement of Large Language Models (LLMs), we break free from conventional thought patterns and introduce a generation-based FKGC paradigm through LLM distillation (Figure 1), aiming to revisit and reassess the future development and potential value of FKGC. Specifically, we propose a LLM-based Multilevel Knowledge Distillation framework for few-shot KG Completion, termed as MuKDC 1 . MuKDC generates additional knowledge for sparse entities and relations, addressing the challenges of data scarcity in few-shot multi-modal learning environments and supporting further FKGC tasks. As a pioneering study in this domain, we have endeavored to keep our approach as straightforward and effective as possible.

This framework includes two main components: Multilevel Knowledge Generation (MKG) and Consistency Assessment (CA). (i) MKG is designed to enrich and expand the KG by generating additional knowledge at multiple levels, including Triplet Generation (TG), Attribute Generation (AG), and Decision Path Generation (DPG). These components collectively enhance the KG, offering a more comprehensive base for few-shot learning applications. TG augments the relational structure of the KG; AG enriches entities with additional descriptive attributes; and DPG creates logical pathways to facilitate inferencing and robust knowledge discovery. (ii) CA assesses the coherence and reliability of the generated knowledge, comparing existing data with the new knowledge to ensure the expanded graph's integrity and consistency.

Note that the LLM distillation method we propose can naturally serve as a means to complete and construct KGs. This method can be viewed as a data augmentation strategy for downstream FKGC, effectively distilling parametric knowledge from LLMs into structured knowledge. This assists not only in knowledge verification but also in enhancing the explainability of other downstream tasks. To validate the generality and robustness of MuKDC, we extend our analysis to multi-modal FKGC scenarios, demonstrating that our model achieves SOTA results in both FKGC and multi-modal FKGC benchmarks.

2 RelatedWork

2.1 Few-Shot Knowledge Graph Completion

Numerous studies [Niu et al., 2021; Liang et al., 2023] in few-shot relational modeling for KGC have concentrated on long-tail relations, broadly categorizing into three approaches: (i) Metric-basedMethods: These methods focus on learning metrics to determine similarity between support and query triples. Initial models like GMatching [Xiong et al., 2018] utilize one-hop neighbors to refine entity embeddings. Later, FSRL [Zhang et al., 2020] and FAAN [Sheng et al., 2020] integrate attention mechanisms for handling relation-specific neighborhoods more effectively. Advancements like REFORM [Wang et al., 2021b] and YANA [Liang et al., 2022b] employe GNNs for latent relation detection, while P-INT [Xu et al., 2021] and MetaP [Jiang et al., 2021] innovate with directed subgraphs and convolutional layers to extract relation patterns. (ii) Optimization-based Methods: These methods [Chen et al., 2019; Niu et al., 2021; Lv et al., 2019] adapt swiftly to new relations using ModelAgnostic Meta-Learning (MAML). Meta-KGR further [Lv et al., 2019] combines MAML with multi-hop path-finding for enhanced entity selection. (iii) Cognitive Graph-based Methods: An example is CogKR [Du et al., 2021], which applies cognitive science principles to construct and update a cognitive graph, facilitating KGC.

1 The source code is available at https://github.com/ xiaoqian19940510/MuKDC.

Building on these foundations, our paper, set against the backdrop of the rapid advancement of LLMs [Zhang et al., 2023c], introduces a generation-based FKGC paradigm through LLM distillation, as shown in Figure 1. Our approach aims to revisit and reassess the future development and potential value of FKGC, marking a departure from conventional methodologies.

2.2 Multi-modal Knowledge Graph Completion

Multi-modal Knowledge Graph Completion (MMKGC) models primarily focus on incorporating visual information to augment structural-only or text-only KGC tasks [Chen et al., 2024; Fang et al., 2022]. Recent MMKG-based models [Liang et al., 2022a; Zhang et al., 2023a] often process visual and structural data separately [Chen et al., 2023b; Chen et al., 2023c], employing general KG embedding (KGE) methods like TransE [Bordes et al., 2013] for unified modeling. Various multimodal context embedding fusion methods, such as simple concatenation [Sergieh et al., 2018], DeViSE [Frome et al., 2013], and Imagined [Collell et al., 2017], have been explored to integrate these different modalities.

Moreover, TransAE [Wang et al., 2019] introduces an autoencoder mechanism for seamless visual and structural integration; RSME [Wang et al., 2021a] focuses on evaluating different image encoders, highlighting the Visual Transformer [Dosovitskiy et al., 2021] as particularly effective; VBKGC [Zhang and Zhang, 2022] and MANS [Zhang et al., 2023b] suggest fine-grained visual negative sampling to better align visual embeddings with structural embeddings, representing a novel approach for fine-grained comparison training; Additionally, MACO [Zhang et al., 2023d] proposes an adversarial training method to complete missing modal information. The current trend in MMKGC primarily utilizes image information as attributes for task design, facilitating the solution of FKGC and Multi-modal FKGC under a unified framework.

3 Preliminaries

Here we define the FKGC task with potential multi-modal contexts as follows:

Definition 1. (Multi-modal) Few-shot Knowledge Graph Completion. Given an incomplete KG G = (E, R, T , A, V) with T = {TA, TR}, where E, R and T , A are the sets of entities, relations, triples, attributes, and values, respectively. The FKGC task aims to complete G by identifying a set of missing triples T ′ = {(h, r, zt)|(h, r, t) /∈ T , h, t ∈ E, r ∈ R}, given only few-shot entity pairs (h, t) and their potential multi-modal attributes for each relation r. To differentiate modal characteristics of attributes, we define text attributes as vl and visual attributes as vv . For instance, in a MMKG, the attribute triple (e, a, vv ) in TA may associate an entity e from E with a visual value vv under the attribute a, designated as hasImage.

Definition 1 also refers to the K-shot KGC task when K training examples are provided for each relation. Unlike prior studies that assume abundant training triples, FKGC addresses scenarios with limited training data. Specifically, the objective is to rank the correct tail entity higher than other candidates, given only K example triples (h′i , r, t′i )Ki=1 for relation r. The candidate set is formed based on entity type constraints.

For a given relation r ∈ R, a dataset Dtrain consists of support (Dsr ) and query (Dqr ) sets: Dr = Dsr , Dqr . Each of the support set Dsr contains K triples for K-shot tasks. The query set Dqr = {hi, r, ti, Chi,r} includes query triples for relation r with ground-truth tail entities ti for each query (hi, r), and a candidate set Chi,r = {tij} where each tij is an entity in G. The metric model is then evaluated on this dataset by ranking the candidate set Chi,r, given the test query (hi, r) and the support triples in Dsr . The datasets Dval and Dtest comprise similar structures.

After sufficient training, the model predicts facts for new relations r′ ∈ R′ . The relation label spaces in the datasets are disjoint, i.e., R∩R′ = ∅, to adhere to the K-shot learning assumption. Otherwise, the model would have access to more than K-shot labeled data during testing, violating the few-shot learning premise.

4 Framework

As illustrated in Figure 2, our MuKDC framework comprises two primary components: Multi-level Knowledge Generation and Consistency Assessment.

(i) Multi-level Knowledge Generation (MKG) aims to enrich and expand the KG by generating multi-level knowledge. This includes: Knowledge Generation: Generating additional triples and attributes to densify the original KG. This deepens entity characterization by enriching entities with descriptive attributes and triplets, providing a more comprehensive understanding of each entity; Decision Path Generation:

Constructing logical pathways linking entities and relations. These paths are pivotal for understanding relationship dynamics, facilitating inference, and enhancing knowledge discovery, while also providing interpretability for graph completion.

(ii) Consistency Assessment (CA): A crucial part of MuKDC, it involves comparing the newly generated knowledge against existing data and predictions. This component uses predicted tail entities and known head entities to ensure the consistency and coherence of relationships in the expanded graph.

Overall, MuKDC is designed to overcome the challenges of sparse data in few-shot multi-modal learning environments. By generating multi-level knowledge, MuKDC not only enriches the KG but also ensures its integrity and consistency, providing a holistic understanding of the graph.

4.1 Multi-level Knowledge Generation

Knowledge Generation

Knowledge Generation in our framework can be likened to a brainstorming session by the LLM on the training set Dtrain. This process involves divergent thinking based on the existing triples and their attributes in the support set, thereby transforming parameter knowledge into structured knowledge to address sparse data challenges.

Specifically, Triplet Generation (TG) focuses on creating new relational triples, leveraging known entities and relations in a sampled one-hop sub-graph. Its goal is to enhance the KG's relational structure, thus broadening and deepening the understanding of entities and their interconnections. Attribute Generation (AG) engages LLMs to generate attributes for nodes in both textual and potential visual forms. This enriches entities with descriptive attributes, leading to a more comprehensive characterization of each entity within the graph. Note that these two processes, TG and AG, operate in parallel and do not directly influence each other.

Triplet Generation. Let Tg = {TA, TR} represent the set of existing triplets in Dtrain, with Eg and Rg denoting the sets of existing entities and relations, respectively. We define Ge as a one-hop subgraph centered around entity e, sampled from T g, where each entity in the sub-KG includes textual and potential visual attributes. The triplet generation process can be formulated as TRnew = TG(Ge), where TRnew is the set of newly generated triplets. The function TG, realized by the LLM, generates new triplets based on the existing sub-KG. In practice, this is achieved by defining appropriate instruction templates for input into the LLM. Specifically, we have developed an "Instruction Pool", of which only a select few examples are presented below:

1. For knowledge graph completion please generate triplet containing the [ENT]: 2. Please generate appropriate triplet containing the [REL]:

For each instruction, we replace the placeholders [ENT] and [REL] with the sets of entities and relations contained in Ge, respectively, to form an instance Inst. We then serialize Ge as a collection of triplets (including relational and attribute triples) to construct the final Template Seq(Ge) || Inst. Here, "||" denotes the concatenation of the serialized triplet set with the instantiated instruction.

This process continues for X rounds, where X equals the number of entities in Eg, effectively iterating through the entire Dtrain for data augmentation. note that for potential visual value vv inputs that may occur, a trainable projection is utilized for transformation.

Attribute Generation. Similar to Triplet Generation, the function AG, realized by the LLM, generates new attributes based on the existing sub-KG to generate new attribute set TAnew. A corresponding "Instruction Pool" has been defined, with a few selected examples shown below:

1. Please generate the important attributes for the [ENT]: 2. Generate key attributes for knowledge graph completion of the [ENT]:

Other settings for AG are consistent with those of TG.

Decision Path Generation

If Knowledge Generation represents brainstorming within a set framework (creating triples and attributes from existing sub-KG data), Decision Path Generation (DPG) expands on this by extrapolating relationships more freely. DPG analyzes existing relationships and patterns to generate decision paths, uncovering deeper insights into the KG. For instance, from the facts "Hangzhou is in Zhejiang" and "Zhejiang is in China", it deduces that "Hangzhou is in China". Similarly, if "LeBron James" and "Anthony Davis" are known to play for the "Lakers", DPG infers that they are teammates.

Note that both Knowledge Generation and DPG can involve and be assisted by images, providing additional information inputs. This process can be facilitated by multi-modal LLMs (e.g., LLava [Liu et al., 2023]). Furthermore, the input for DPG is based on the output of Knowledge Generation combined with the existing Dtrain. We refer to this combined set as ˆDtrain = TRnew ∪ TAnew ∪ T . The one-hop subgraph sampled from ˆDtrain is denoted as ˆ Ge.

The decision path generation process can be formulated as TRdpg = DPG( ˆ Ge), where TRdpg is the set of newly generated decision paths. A corresponding "Instruction Pool" has been defined, with a few selected examples shown below:

1. Based on sampled rules, please generate important logical rules for relation for [ENT]: 2. Please generate the important decision path for the knowledge graph completion for [REL]:

Other settings for DPG are consistent with TG and AG.

4.2 Consistency Assessment

MuKDC incorporates a crucial Consistency Assessment (CA) process to verify the coherence and reliability of the newly generated knowledge, including triples and attributes. This process evaluates the compatibility of predicted tail entities with known head entities against the existing KG data. By ensuring that these predictions align with established relationships, the integrity and accuracy of the expanded graph are maintained.

Specifically, we employ a Knowledge Graph Embedding (KGE) model pre-trained on Tg (using TransE [Bordes et al., 2013]) as the base scoring model for CA. This model is used to conduct consistency checks on all triplets in the newly generated sets TRnew and TAnew. Each triplet is sequentially input into the CA model, and those scoring below a certain threshold are discarded. Considering the likelihood of encountering many unseen entities and relations in this process, as highlighted in various zero-shot KGC works [Chen et al., 2023a; Chen et al., 2021a; Geng et al., 2021], we train the KGE model where all entity and relation representations are derived from their embeddings obtained via the LLM. This designation facilitates the inductive aspect of the CA process.

4.3 Loss Function

Given a query relation r and its associated support triples (h′i , r, t′i )Ki=1 , we adopt negative sampling to construct query triples. Specifically, we gather a set of valid positive query triples {(hi, r, t+i ) | (hi, r, t-i ) /∈ G} and corrupt the tail entities to construct another group of negative query triples {(hi, r, t-i ) | (hi, r, t-i ) /∈ G}. In line with established fewshot learning paradigms, our model is equipped with a hinge loss function:

where score+θ and score-θ are the scalar values derived from comparing the query triple (hi, r, t+i /t-i ) with the support triples (h′i , r, t′i )Ki=1 via a metric learning model from [Zhang et al., 2022], and γ represents a tunable hyperparameter margin. During each training episode, we commence by sampling Dr from the designated training set Dtrain. Subsequently, K triples are selected to act as the support set Dsr , and additional triples are chosen to formulate the positive query/test set Dqr , compiled from the entirety of known triples within Dr.

50 to 499 triples as few-shot tasks. NELL includes 67 fewshot relations, and Wiki comprises 183. We divide NELL into 51/5/11 and Wiki into 133/16/34 relations for training, validation, and testing, respectively.

Multi-Modal FKGC Datasets. We employ another two public benchmark datasets, MM-FB15K and MM-DBpedia, tailored for few-shot multi-modal KG completion [Zhang et al., 2022]. These datasets feature entities accompanied by images and textual descriptions of at least 15 words. For a robust evaluation, we consider only those relations with 50 to 500 triples as few-shot tasks. MM-FB15K comprises 356 few-shot relations, and MM-DBpedia includes 69. Relations exceeding 500 triples serve as background information, enhancing the knowledge topology. The splits for training, validation, and testing are allocated as 267/18/71 task relations for MMFB15K and 51/6/12 for MM-DBpedia, following the 15:1:4 ratio, in line with prior studies [Zhang et al., 2022].

Evaluation Metrics. We assess our model's performance using two standard metrics: Hits@N and MRR. Hits@N quantifies the proportion of correct entities ranked within the top N predictions, for N set to 1, 5, or 10. MRR computes the mean of the reciprocal ranks assigned to the correct entities across all test triples.

5 Experiments

5.1 Detailed Setting

Implementation Details Our FKGC is implemented using PyTorch and trained on a Tesla V100 GPU. In the experiments, all entity embeddings are represented using the embedding of the last token from the final layer's output of the LLava model [Liu et al., 2023], a common entity encoding approach in PLMs [Chen et al., 2023d; Geng et al., 2023]. For MMKGbased FKGC tasks, the standard visual projection head in LLava is employed, whereas for typical KG tasks, this module is not utilized. During Knowledge Generation and Decision Path Generation, all triple sequences are simply concatenated in the order of h, r, and t, with delimiters used to separate different triples. The threshold for the TransE [Bordes et al., 2013] model during the Consistency Assessment process is set to 1.0. All other experimental settings not mentioned here, including the training procedures for FKGC, are kept consistent with those reported in [Zhang et al., 2022].

FKGCDatasets. We employ two public benchmark datasets for FKGC: NELL and Wiki [Mitchell et al., 2018; Vrandeˇci´c and Krötzsch, 2014]. NELL is an evolving KG dataset containing a broad spectrum of knowledge, while Wiki is derived fromWikipedia content. For both, we select relations with underscore the model's robust representation learning from multi-level information, affirming its capability for few-shot multi-modal KG completion tasks. (ii) Our model surpasses KG embedding baselines with substantial gains of 25.9% and 36.9% on average for the NELL and Wiki datasets, respectively. This performance showcases the benefit of generating multi-level knowledge for entities and relations, particularly apt for few-shot multi-modal KG completion. (iii) Moreover, our model outperforms few-shot KG completion baselines on all metrics, evidencing the efficacy of enriching the KG by generating additional knowledge and employing consistency assessment loss to minimize performance gaps in generated knowledge scores. (iv) Against the best-performing baselines GANA and YANA on NELL and Wiki, our model shows a minimum average improvement of 8.5% and 4.9%, respectively. This is attributed to our method's comprehensive understanding of the multi-modal KG.

In the realm of multi-modal FKGC, our model maintains a considerable lead over all baselines. Against the topperforming baseline MULTIFORM on the MM-FB15K and MM-DBpedia datasets, it registers a minimum average enhancement of 7.4% and 4.1%, respectively. Such results further confirm the model's proficiency in handling multi-modal FKGC tasks.

5.4 Discussion for Model Variants

To assess the contribution of each module in our model, we performed variant experiments on the NELL and Wiki datasets, as shown in Tables 3 and 4. These experiments compare our complete model against versions with key modules removed. The observations from these comparisons are as follows:

(i) The Multi-level Knowledge Generation module shows a notable impact, likely because it enhances the utilization of limited samples by generating entity and relation representations. (ii) Components such as Triplet Generation, Attribute Generation, and Decision Path Generation positively influence the results. They contribute to identifying valuable entity

5.2 ComparisonMethods

Knowledge Embedding Models. We benchmarked against four KGE models: TransE [Bordes et al., 2013], DistMult [Yang et al., 2015], ComplEx [Trouillon et al., 2016], and RotatE [Sun et al., 2019]. These models vectorize relations and entities with constraints reflecting relational structures in KGs, capturing essential structural features for effective representation given ample data.

FKGC. Seven FKGCmodels were evaluated: GMatching (MaxP) [Xiong et al., 2018], MetaR [Chen et al., 2019], FSRL [Zhang et al., 2020], FAAN [Sheng et al., 2020], GANA [Niu et al., 2021], YANA [Liang et al., 2022b], and P-INT [Xu et al., 2021]. These approaches utilize metric or optimization-based meta-learning with pre-trained embeddings, focusing on local structures and relational semantics for enhanced relation and entity pair embeddings, demonstrating strong results on NELL and Wiki datasets.

Multi-modal FKGC. Three baseline methods were considered: TransAE [Wang et al., 2019], RSME [Wang et al., 2021a], and MULTIFORM [Zhang et al., 2022]. TransAE employs an autoencoder for multimodal integration, RSME assesses different image encoders and endorses ViT for multimodal KGC, and MULTIFORM presents a multimodal fewshot relation learning framework that leverages multimodal context for entity representation and learns a metric for matching queries to few-shot examples.

5.3 Main Results

We present the overall 5-shot FKGC results in Tables 1 and 2, with all comparative model results sourced from their respective original papers:

(i) Our model consistently outperforms baseline models across all four metrics on the NELL and Wiki datasets, achieving average improvements of 3.8% and 4.9%, respectively. Notably, it shows at least a 5.8% increase in MRR on Wiki and a 7.5% increase in Hits@10 on NELL. These improvements Consistency Assessment w/o. Consistency Assessment

1-shot 3-shot 5-shot

Few-shot situation

(b) Impact on consistency assessment.

Path 1: Bill Maher -> award -> Primetime Emmy Award for Individual Performance -> ceremony -> 60th

Path 2: Santa Barbara -> award_nominee -> Sheri Anderson

Path 3: Bill Maher -> award_nominee -> Brad Grey -> film -> Charlie and the Chocolate Factory

(Bill Maher, award_nominee, Bernie Brillstein)

Path 1: Bolton Wanderers F.C. -> player-> Keith Andrews-> team -> Wolverhampton Wanderers F.C. Path 2: EFL Championship-> team-> Bolton Wanderers F.C. -> sport -> association football

Path 3: Leicester City F.C. -> league -> EFL Championship -> award_winner-> Reading F.C.

(Bolton Wanderers F.C., league, EFL Championship)

• (c) Interpretable path analysis.

Figure 3: Analysis for the few-shot multi-modal knowledge graph completion on the few-shot situations.

features with greater certainty, highlighting the importance of generating multi-dimensional knowledge at various levels. (iii) Removing the Consistency Assessment module leads to a decrease in performance, underscoring its role in evaluating the consistency and coherence of relationships within the expanded KG. These findings collectively validate the significance of each component in our model's overall effectiveness.

5.5 Discussions on Knowledge Generation

We conducted experiments to assess the impact of the multilevel knowledge generation module, focusing on the number of prompts used, as illustrated in Figure 3 (a).

The findings indicate: (i) Optimal Performance: The model performs best when using three attribute prompts, four triplet prompts, and three decision path prompts. This suggests a balance is necessary to prevent overfitting, which may occur with an excessive number of prompts. Overfitting leads to reduced generalizability as the model becomes overly reliant on specific patterns in the prompts. (ii) Effect of Excessive Prompts: Increasing the number of prompts beyond this optimal range leads to a notable decline in performance. Too many prompts can introduce confusion and noise, impairing the model's effectiveness and decision-making capacity.

These results highlight the importance of a balanced approach to the number of prompts in the knowledge generation module for effective model performance.

5.6 Discussion on Consistency Assessment

We evaluated the impact of the consistency assessment across 1-shot, 3-shot, and 5-shot scenarios on the MM-FB15K and MM-DBpedia datasets, as depicted in Figure 3 (b).

Key findings include: (i) Models lacking the consistency assessment module ("w/o. Consistency Assessment") show a notable decrease across all metrics, highlighting the critical role of evaluating generated facts in enhancing model performance. (ii) Our model demonstrates a greater sensitivity in the 1-shot multi-modal KG completion tasks compared to other baselines. This sensitivity can be attributed to our model's reliance on consistency to refine the generated KG. (iii) In various few-shot contexts, our model consistently outperforms others, underscoring its aptitude for the task by rigorously considering triplet consistency and evaluating the coherence between original and generated knowledge.

These insights confirm that our model effectively leverages multi-level prompts, making it particularly adept at consistency assessment in few-shot MMKGC tasks.

5.7 Discussion on Interpretable Path

To underscore the role of decision paths in boosting interpretability and supporting predictions, we analyzed their application using the MM-FB15K and MM-DBpedia datasets, as depicted in Figure 3 (c).

Key observations include: (i) Decision paths effectively guide predictions. For instance, consider the relation "award_nominee" between the head entity "Bill Maher" and the predicted tail entity "Bernie Brillstein". A possible decision path is: "Bill Maher → award → Primetime Emmy Award → ceremony → 60th". This path not only clarifies the prediction's rationale but also highlights Bill Maher's connection with the Primetime Emmy Award. (ii) Decision paths reveal the crucial roles of various relations and entities in the prediction process. Examining the relation "league" between "Bolton Wanderers F.C." and the predicted tail entity "EFL Championship", a plausible path is: "EFL Championship → team → Bolton Wanderers F.C. → sport → association football". This demonstrates the model's reliance on specific relationships for its predictions.

These insights confirm that decision path generation facilitates interpretable paths for KGC, offering a clearer understanding of model's reasoning and enhancing interpretability.

6 Conclusion

In this work, we introduced the MuKDC framework for FKGC, addressing data scarcity in few-shot scenarios through LLM distillation. Comprising two integral components, Multi-level Knowledge Generation and Consistency Assessment, MuKDC goes beyond merely expanding and deepening KGs; it ensures their coherence and reliability through rigorous evaluation and alignment with existing knowledge structures. This framework has demonstrated SOTA performance in both FKGC and multimodal FKGC tasks, highlighting its efficacy in enriching KGs in long-tail scenarios. The success of MuKDC in advancing KG completion and its potential in leveraging LLMs for NLP tasks sets a new benchmark in the field and paves the way for future research in KG augmentation.

Acknowledgements

We thank the anonymous reviewers for their insightful comments and suggestions. The corresponding author is Jianxin Li. The authors of this paper were supported by the NSFC through grant No.62225202.