LLMs as Parametric Knowledge Sources for Knowledge Graph Completion

1 Introduction

Knowledge graphs (KGs) are widely used to store factual knowledge as triples (subject, relation, object). By organizing entities and relations in a structured form, KGs support a variety of application scenarios such as recommender systems [Zhang et al., 2024a], question answering [Ma et al., 2025], and diagnostic assistance [Li et al., 2025]. However, large real-world KGs are inevitably incomplete, which limits their effectiveness in downstream tasks. This makes knowledge graph completion (KGC) an essential task, aiming to predict missing facts based on existing triples.

The emergence of pre-trained language models (PLMs) and LLMs has significantly changed KG representation learning. Traditional KGC methods embed entities and relations into a continuous space to predict missing facts with geometric scoring functions. More recently, PLM-enhanced KGC approaches leverage the rich linguistic and world knowledge embedded in pre-trained models and can be broadly categorized into two paradigms. The first line of work treats KGs as textual corpora and fine-tunes PLMs/LLMs with KG data [Yao et al., 2019; Lv et al., 2022]. This can be effective, but it is often computationally expensive and may underutilize inherent structural information of KGs due to the inconsistency between language modeling and graph learning objectives. The second paradigm uses prompt design to introduce semantic priors from LLMs into the KGC learning process [Wei et al., 2023; Jiang et al., 2024]. Such methods typically provide only shallow semantic guidance and rely on surface-level prompts, leaving the internal parametric factual knowledge encoded within LLMs insufficiently utilized. Figure 1 illustrates these two paradigms and our proposed elicitation-based paradigm. Rather than adapting PLMs/LLMs to KG triples or relying on guidance from LLM

∗Corresponding author.

textual outputs, LarKS treats LLMs as parametric knowledge sources and uses their internal representations to provide representation-level guidance for KGC.

Recent advances in understanding how factual knowledge is stored in Transformer-based language models [Geva et al., 2021; Meng et al., 2022; Hernandez et al., 2024] have provided new insights into their internal representations. These studies suggest that PLMs implicitly encode a substantial amount of factual knowledge in their parameters. This observation motivates us to explore how such parametric knowledge can be effectively elicited and leveraged to enhance KGC. However, two critical challenges arise: First, factual knowledge may be implicitly stored across different layers and neurons of LLMs, rather than being directly accessible from their responses. Second, there is an objective mismatch between next-token prediction in LLMs and the embeddingbased learning objective commonly used in KGC methods.

To address the above issues, we propose a novel PLMenhanced KGC paradigm, LarKS, that leverages LLMs as parametric knowledge sources to enhance the KGC task. Our framework elicits factual representations from LLM internal representations and aligns them with the representation space learned by KGC models. Specifically, we introduce an LLM knowledge elicitation mechanism to extract representations from intermediate hidden states of existing pre-trained Transformer language models. To bridge the gap between the sentence-level representations of LLMs and the entitylevel embeddings used in KGC, we design a cross-granularity representation alignment method that transforms and aligns them. Finally, to ensure stable and effective joint optimization, a dynamic learning schedule is adopted to balance the alignment and representation learning objectives throughout training.

The main contributions of our work are summarized as follows:

• We propose a novel learning paradigm for KGC methods, where LLMs are treated as parametric knowledge sources to enrich the representation learning of KGs. This paradigm provides a promising way to bridge the implicit parametric knowledge in LLMs and the explicit symbolic facts in KGs.
• We design a PLM-enhanced KGC framework, LarKS. It includes a knowledge elicitation mechanism for extracting factual representations from LLMs, a crossgranularity alignment method for bridging sentencelevel and entity-level representations, and a dynamic learning schedule for balancing joint objectives.
• We conduct extensive experiments on several benchmark datasets under both standard and lifelong settings. The results demonstrate the effectiveness of our method and highlight the potential of integrating parametric knowledge into KGC methods.

2 RelatedWork

2.1 PLM-enhanced KGC

Traditional KGC methods learn vector representations of entities and relations, and define a scoring function over triples to predict missing facts. Prior work has explored increasingly expressive scoring functions, ranging from geometric transformations [Bordes et al., 2013; Sun et al., 2019] to factorized bilinear interactions [Yang et al., 2015; Trouillon et al., 2016], and further to convolutional interaction modules [Dettmers et al., 2018].

The rapid development of PLMs/LLMs, which encode rich world knowledge and semantic priors, has motivated growing interest in combining PLMs with KGC. Existing PLMenhanced KGC methods can be broadly grouped into two categories: (1) Fine-tuning methods convert KG triples and contexts into input sequences for PLMs/LLMs and optimize them for link prediction. KG-BERT [Yao et al., 2019] and PKGC [Lv et al., 2022] fine-tune PLMs for triple scoring. KoPA [Zhang et al., 2024b], MKGL [Guo et al., 2024], and SSQR [Lin et al., 2025] further augment tuning with KG-derived structural information or specialized KG codes. However, fine-tuning methods are costly and easily disrupt KG structural information. (2) Prompt-based methods use designed prompts to obtain guidance from frozen LLMs. KICGPT [Wei et al., 2023] and ARR [Chen et al., 2024] use LLMs for candidate re-ranking. CSProm-KG-CD [Li et al., 2024] distills LLMs' generated triple descriptions into smaller KGC models. KG-FIT [Jiang et al., 2024] uses LLMguided refinement to build a hierarchical structure of entity clusters for improving KGC training. Nevertheless, promptbased methods provide shallow semantic augmentation and may produce hallucinated and underspecified results.

2.2 Factual Representations in Language Models

A growing line of research investigates how factual knowledge is stored and represented inside LLMs 1 . These studies primarily aim to explore the internal mechanisms through which Transformer language models encode factual associations and how such representations influence model predictions. Feed-forward layers in Transformers have been found to act as key-value memories encoding input patterns and their associated output distributions [Geva et al., 2021; Geva et al., 2022]. ROME [Meng et al., 2022] reveals that factual knowledge is stored in highly causal "early-site" MLP activations at the last subject token, which is later leveraged by MEMIT [Meng et al., 2023] to enable largescale model updating. Further studies [Sakata et al., 2025; Morand et al., 2025] explore the finer-grained correlation between entities and their mentions within LLM hidden representations. LRE [Hernandez et al., 2024] shows that many relational associations in TLMs exhibit an approximately affine mapping from subject to object representations. Inspired by these efforts, we explore whether such internal factual representations can be elicited and leveraged to enhance KGC.

3 Methodology

We propose LarKS, a PLM-enhanced KGC framework that leverages the parametric knowledge in pretrained LLMs for KG representation learning. As shown in Figure 2, LarKS

1 Most analyses relied on Transformer language models (TLMs). Since LLMs predominantly adopt this architecture, we do not strictly distinguish between "TLM" and "LLM".

elicits factual representations from the LLM's internal representations, bridges the granularity mismatch to obtain entitylevel guidance, and jointly optimizes objectives of different stages under a dynamic learning schedule.

3.1 LLMKnowledge Elicitation

The knowledge elicitation module extracts factual representations from pretrained LLMs' intermediate hidden states to obtain elicited entity representations and relational transformations, serving as prior semantic guidance from the LLM's parametric general knowledge. We design prompt templates to induce the LLM to produce the expected factual prediction, and take the corresponding hidden states as elicited representations for subsequent modules.

Elicited Representations Extraction

Recent works [Geva et al., 2021; Meng et al., 2022; Hernandez et al., 2024] have explored how factual knowledge is stored within TLMs.

Building on causal intervention analysis [Meng et al., 2022], prior work identifies strongly causal states at an early site in the middle layers of autoregressive TLMs. These states occur at the last subject token and have been shown to encode entity-level factual information that influences the model's predictions. Motivated by this, we extract elicited entity representations from the early-site hidden state of the last subject token. Specifically, given an input sequence of length T, a TLM fθ with L layers produces layer-wise hidden states H(l) = f (l) θ (H(l-1) ) for l = 1, . . . ,L, where H(l) = [h(l) 1 ; . . . ; h(l) T ] ∈ RT×d and d is the hidden size. For a subject entity e tokenized as m tokens with its last token position pm, we select the hidden state of the last subject token from an early layer ln (typically ln ≤ L/2) as its elicited entity representation:

where e denotes the elicited entity representation of e. To maintain consistency in elicited entity representations, we use a simple fixed prompt template, {} is a ..., where {} is replaced with the entity name, which reduces contextdependent variation and yields hidden states that primarily reflect entity-level semantics. We elicit entity representations and relation transformations separately to decouple their estimation dependencies and better align with embedding-based KGC methods.

Our relation extraction is grounded in the linear relational embedding (LRE) [Hernandez et al., 2024], which shows that, for a subset of relations, the transformation from subject to object representations can be well-approximated by a single relation-specific affine mapping. Formally, given a relation-specific prompt context cr, the induced mapping can be written as: o = F(s, cr), where s is the intermediatelayer hidden representation at the last subject token and o is the hidden state used to decode the object's first token. We approximate F with a relation-specific affine transformation:

where Wr and br are the relation-specific transformation matrix and bias term, and λcorr (> 1) is a scalar constant that corrects the magnitude underestimation caused by layer normalization. Following LRE, we use a first-order approximation with Jacobian J(s, c) = ∂F(s, c)/∂s, and estimate Wr and br by taking expectations over examples (si, ci) of relation r:

In practice, we approximate the expectations by averaging over the sampled examples.

Prompt Design and Guidance Strategy

Since large-scale KGs contain many different relations, we design a prompt template generation strategy based on incontext learning, avoiding manual prompt template design for each new relation. Here, prompts are used to control the elicitation of internal representations and relation transformations, rather than to query the LLM for final KGC prediction. Specifically, we adopt a set of seed relations with wellcrafted prompt templates, containing several typical relation categories. For an unseen relation, we encode its relation text and each seed relation text with an embedding model, compute their semantic similarity in the embedding space, and use prompt templates of top-M seed relations as in-context demonstrations. With these demonstrations, the LLM generates N candidate prompt templates for the new relation. Figure 2 provides an example of the generated templates for the relation /music/artist/origin. The generated prompt templates provide multiple prompt patterns for the same relation, enabling parameter estimation from diverse contexts and yielding more stable results.

However, this process cannot ensure the model produces factual predictions, and incorrect outputs will distort the estimated relation transformations. To address this issue, we adopt distracted prompts as model inputs, which induce the model to produce the expected response while having no significant impact on the resulting estimates [Hernandez et al., 2024]. We first apply the instruction-distracted form, and use the repetition-distracted form as a fallback when the model still fails to produce the expected prediction. Figure 2 also illustrates the two distracted prompt designs for the triple (Beethoven, /music/artist/origin,Germany). Empirically, this two-stage prompting strategy suffices for all samples in our experiments. For rare cases where the model still produces an incorrect prediction, which typically correspond to longtail or difficult samples, we resample a new instance instead. These designs help stabilize the elicitation process for relation transformation estimation.

3.2 Cross-Granularity Representation Alignment

A key challenge in PLM-enhanced KGC is the misalignment between sentence-level representations from LLMs and entity-level embeddings learned by KGC methods. This granularity discrepancy prevents these representations from effectively guiding KGC optimization. We address this issue with cross-granularity representation alignment, combining a granularity transformation module with a representation alignment module.

Granularity Transformation

LLMs trained under the next-token prediction objective primarily yield sentence-level contextual representations, which are mismatched with the entity-level embeddings learned by KGC methods. Recent work on the injectivity and invertibility of language models suggests that hidden representations can retain recoverable information about input sequences [Nikolaou et al., 2025]. This motivates us to treat the granularity gap as a learnable transformation problem. Specifically, we introduce a lightweight granularity adapter to convert sentence-level representations into entity-level embeddings. For a triple (s, r, o), we employ a layer normalization followed by a linear transformation to stabilize representations and perform granularity transformation:

where esent o is the sentence-level representation of o obtained by transforming the elicited subject representation es with the relation transformation in Eqs. 3 and 4, eent o is the corresponding entity-level representation produced by the granularity adapter, Wgran and bgran are learnable parameters, and LN(·) denotes layer normalization.

To enable elicited representations to better guide embedding-based KGC training, we encourage consistency between the predictive distributions produced by the granularity adapter and the embedding-based KGC method:

where KL(·∥·) denotes the Kullback-Leibler divergence, which promotes adaptation of the LLM-derived predictions to the granularity and semantic space of the KGC method under joint optimization. We construct the two distributions with the elicited entity representation matrix Eeli and the entity embedding matrix Ekgc of the KGC method:

where r denotes the relation embedding in the KGC method, and ϕ(·) is the entity-relation composition function that composes the subject and relation embeddings into a predicted object representation for link prediction. This composition is consistent with the scoring function of the KGC method. For example, TransE uses the score f(s, r, o) = -∥ekgc s + r - ekgc o ∥, and thus its composition function is ϕ(ekgc s , r) = ekgc s + r.

Representation Alignment

We further enforce their consistency with a representation alignment objective, complementing distribution-level guidance. This alignment mechanism encourages representations of the same entity to be close and suppresses incorrect ones, consistent with CLIP-style representation learning [Radford et al., 2021; Brannon et al., 2024; Zhu et al., 2025]. Accordingly, we measure the distance between the transformed entity representation and the predicted object representation produced by the KGC method as follows:

To optimize the distance with negative triples, we adopt the widely used negative sampling loss [Mikolov et al., 2013]:

where γ is the margin hyperparameter, σ(·) is the sigmoid function, n is the number of negative samples, and (s′i , r, o′i ) denotes the i-th negative sample corresponding to (s, r, o). For prediction, the final representation is obtained by a weighted fusion: eo = ηϕ(e kgc s , r) + (1 - η)e ent o , where η ∈ [0, 1] is a hyperparameter.

3.3 Dynamic Learning Schedule

During the joint optimization of LLM elicited representations and KGC methods, the optimization focus evolves over different training stages. In the early stage of training, we emphasize granularity transformation to make the two representations comparable. This reduces the risk of suboptimal local minima early on, allowing both sides to develop useful structures before stronger semantic alignment. As training proceeds, once the two representations become comparable in granularity and distribution, the optimization focus should shift toward representation alignment. This stage enforces semantic consistency by bringing representations of the same entity closer and suppressing mismatched ones, thereby enabling effective knowledge sharing and mutual reinforcement.

To this end, we adopt a dynamic learning schedule that gradually shifts the training emphasis across stages. Specifically, we introduce two time-varying weights to balance the contributions of granularity transformation and representation alignment:

where t denotes the current training step. In practice, we observed that a simple clipped linear schedule can be effective as follows:

where ts and te denote the start and end epochs, while ws and we are the corresponding initial and final weights. Here, clip(x, 0, 1) = max(0,min(1, x)). The weight of granularity transformation λα(t) decreases linearly as training proceeds, while the weight of representation alignment λβ(t) increases, gradually shifting the training emphasis from granularity reconciliation to representation consistency.

3.4 Total Training Objective

Our overall training objective integrates the above modules for joint optimization. The total loss is formulated as:

where LKGC adopts the same form as Eq. 10, with the model-specific distance term defined as the negative score d(s, r, o) = -f(s, r, o). LarKS jointly leverages KGC representations and elicited representations from LLMs, allowing them to complement each other during training.

4 Experiments

4.1 Experimental Settings

We evaluate our framework in two settings: standard KGC and lifelong learning [Cui et al., 2023]. For the latter, we leverage LLMs' broad world knowledge and generalization ability to continually learn from evolving KG snapshots while avoiding forgetting previously acquired facts.

Datasets. For standard KGC, we use three benchmark datasets, FB15k-237 [Toutanova and Chen, 2015], FB15k237N [Lv et al., 2022], and Wiki27k [Lv et al., 2022], sampled from Freebase and Wikidata. Table 1 reports the statistics of these standard KGC datasets. For the lifelong setting, we adopt seven benchmark datasets constructed under different incremental strategies (ENTITY, RELATION, FACT, HYBRID) and speeds (GraphEqual, GraphHigher, GraphLower), following previous works [Cui et al., 2023; Liu et al., 2024a].

Baselines. In the standard KGC, we compare our method with: (1) Traditional embedding-based methods: TransE [Bordes et al., 2013], DistMult [Yang et al., 2015], ComplEx [Trouillon et al., 2016], ConvE [Dettmers et al., 2018], and RotatE [Sun et al., 2019]; (2) PLM-enhanced methods: KG-BERT [Yao et al., 2019], PKGC [Lv et al., 2022], and KG-FIT [Jiang et al., 2024]. For the lifelong setting, we adopt twelve baselines including: (1) Non-continual methods: Snapshot and Fine-tune; (2) Dynamic architecturebased methods: PNN [Rusu et al., 2016], CWR [Lomonaco and Maltoni, 2017], and FastKGE [Liu et al., 2024b]; (3) Rehearsal-based methods: GEM [Lopez-Paz and Ranzato, 2017], EMR [Wang et al., 2019], and DiCGRL [Kou et al., 2020]; (4) Regularization-based methods: EWC [Kirkpatrick et al., 2017], SI [Zenke et al., 2017], LKGE [Cui et al., 2023], and IncDE [Liu et al., 2024a].

EvaluationMetrics. Following common practice, we evaluate link prediction using mean reciprocal rank (MRR) and Hits@N (H@N) under the filtered setting [Bordes et al., 2013]. In the lifelong setting, we evaluate the final model on the union of all test sets, restricting candidate entities of each query to those observed up to its snapshot.

ImplementationDetails. All experiments are implemented in PyTorch [Paszke et al., 2019], and training uses the Adam optimizer [Kingma and Ba, 2015]. We follow prior implementations [Lv et al., 2022; Jiang et al., 2024; Liu et al., 2024a] and their main settings when applicable. We tune the batch size in {512, 1024, 2048}, and set the hidden dimension to 1024 in the standard setting and 200 in the lifelong setting. For LLM components, we use Qwen3-8B (default) and Llama 3.1-8B (analysis in Section 4.4) as backbones, and adopt Qwen3-Embedding-4B (prompt template generation in Section 3.1) and OpenAI text-embedding-3-large (ablation variant in Section 4.3) as embedding models. During training, we evaluate validation MRR every 3 epochs and apply early stopping with patience 5. For step-based runs, we treat one dataset round as one epoch.

4.2 Main Results

Standard Setting. Table 2 shows results on three standard KGC datasets. Overall, LarKS consistently improves over several baselines and achieves the best performance when built upon KG-FIT. For traditional methods, LarKS improves MRR over TransE, DistMult, ComplEx, ConvE, and RotatE by 12.6%, 6.8%, 14.3%, 3.4%, and 6.5%, respectively. On FB15k-237, FB15k-237N, and Wiki27k, LarKS improves MRR with average relative improvements of 11.1%, 7.1%, and 8.0%, respectively. These results demonstrate the effectiveness and adaptability of our proposed method. For PLMenhanced baselines, LarKS (KG-FIT) achieves average relative MRR improvements of 18.5%, 30.2%, and 18.7% on FB15k-237, FB15k-237N, and Wiki27k, respectively. Compared with KG-FIT, our extended framework yields an average relative MRR gain of 4.4% across the three datasets. We observe that PKGC achieves higher Hits@1 on FB15k-237N and Wiki27k, since we report its definition-augmented variant that injects support prompts and fine-tunes a PLM-based classifier, which yields a clear Hits@1 improvement over the base PKGC. This suggests that fine-tuning PLMs with support prompts can provide a more targeted boost to semantic prediction, but requires higher training and computation costs. Moreover, KG-FIT has been shown to complement PKGC in a recall and re-rank pipeline [Jiang et al., 2024], indicating that LarKS not only achieves better overall performance but also remains promising and extensible to such hybrid architectures.

Lifelong Setting. We integrate LarKS into the representative lifelong baseline LKGE for evaluation and report the results in Table 3. Overall, LarKS achieves the best average performance across the seven benchmarks. Compared with Snapshot and Fine-tune, it yields substantial gains, indicating effective continual learning capability under evolving snapshots. Compared with ten continual learning baselines, LarKS achieves average relative improvements of 62.6% and 48.6% in MRR and H@10, respectively. By integrating LarKS into LKGE, we obtain average relative improvements of 7.5% in MRR and 5.3% in H@10 over the original LKGE, while achieving overall performance comparable to the stateof-the-art baseline IncDE, without introducing any additional lifelong-specific components. Although LarKS is not the best on a few metrics, such as MRR on FACT, it does not incorporate additional designs specialized for the lifelong scenario. Nevertheless, the results still show that LarKS can effectively enhance KGC in the lifelong setting, benefiting from the expressive and transferable parametric knowledge of the LLM.

4.3 Ablation Study

To evaluate the contribution of each component in LarKS, we construct four variants: (1) w/o GT removes the granularity transformation module, (2) w/o RA removes the representation alignment module, (3) w/o DLS disables the dynamic learning schedule, and (4) w/ Emb replaces our elicited entity representations with embeddings obtained from entity texts/descriptions via OpenAI text-embedding-3-large and regularizes the model to stay close to these embeddings using a distance-based constraint. As shown in Table 4, LarKS yields 4.4% and 2.5% relative improvements in MRR and H@10 over the three removal variants. The improvements are averaged across three datasets and both TransE and RotatE, demonstrating the effectiveness and necessity of each module. Moreover, compared with w/ Emb, LarKS further improves MRR and H@10 by 5.9% and 2.1% on average, indicating that our elicited representations are more effective than directly relying on LLM-based embedding models.

4.4 Further Analysis

Effect of Elicitation Layers. We investigate how the choice of elicitation layer affects LarKS. Figure 3 shows the MRR of TransE and RotatE on FB15k-237 and Wiki27k when extracting LLM internal representations from four representative layers. Overall, eliciting from earlier layers consistently yields the best results. On FB15k-237, the performance peaks at layer 5 (our default setting) for both TransE and RotatE and tends to decrease as we move to deeper layers, while a similar trend is observed on Wiki27k. Although the performance is not strictly decreasing in all cases, the early layer setting is consistently superior. This suggests that the guidance leveraged by LarKS is more effective in early internal states, which preserve more localized semantics and are less affected by deeper layer contextualization.

Effect of Backbone LLMs. We further evaluate the effect of backbone LLMs on LarKS. Figure 4 shows the MRR of TransE and RotatE on FB15k-237, FB15k-237N, and Wiki27k when using Qwen3-8B (our default setting) and Llama 3.1-8B to elicit internal representations. LarKS achieves comparable performance across the two backbones. On FB15k-237 and FB15k-237N, Qwen3 yields slightly higher MRR for TransE and achieves comparable results to Llama 3.1 for RotatE. On Wiki27k, the two backbones perform similarly, with Llama 3.1 being marginally better for RotatE. These results indicate that, under comparable model scale, LarKS is not overly sensitive to backbone architecture details, and can effectively leverage parametric knowledge from different LLMs to provide guidance for KGC training.

5 Conclusion

In this paper, we propose a new learning paradigm for PLMenhanced KGC that treats LLMs as parametric knowledge sources. Instead of fine-tuning or prompting LLMs, we elicit factual representations from their internal states and use them to guide the training of KGC methods. This design avoids the cost of LLM updating and shallow prompt-level usage, while leveraging the parametric knowledge implicitly encoded in the model to improve link prediction. Extensive experiments across multiple benchmarks in both standard and lifelong settings demonstrate that our framework consistently improves performance with different KGC methods.