Double-Calibration: Towards Reliable LLMs via Calibrating Knowledge and Reasoning Confidence

1 Introduction

The reliability of Large Language Models (LLMs) is critically undermined by their tendency to hallucinate [Huang et al., 2024], a problem rooted in both intrinsic epistemic uncertainty (knowledge gaps) and extrinsic aleatoric uncertainty (data ambiguity) [H¨ullermeier and Waegeman, 2021]. To mitigate this, Knowledge Graph-augmented RetrievalAugmented Generation (KG-RAG) has emerged as a leading paradigm [Zhang et al., 2025a]. By augmenting LLM with structured evidence retrieved from external Knowledge Graphs (KGs), KG-RAG is helpful to reduce the model's internal knowledge gaps and improve the factual accuracy of its responses [Xiang et al., 2025].

However, this KG-augmentation mechanism introduces a new yet critical dependency: the certainty of the retrieved evidence itself. Prevailing KG-RAG methods [Luo et al., 2024;

∗Corresponding Author.

An extended version of this paper can be found at https://arxiv. org/abs/2601.11956.

Li et al., 2025a; Liu et al., 2026] often rely on an idealistic assumption that the retrieved evidence is always both sufficient and certain to support correct reasoning for a given query. This assumption is routinely violated in practice due to ambiguous queries, the intrinsic incompleteness of KGs, and imperfections in the retrieval process. Consequently, when provided with partial evidence, LLMs may still produce confidently stated but incorrect predictions [Kalai et al., 2025]. For example, as illustrated in Figure 1, given the partial evidence "Belle is a sibling of Snoopy", an LLM might incorrectly infer "Belle is Snoopy's brother". Thus, the current KG-RAG lacks the ability to assess and control uncertainty at the very source of the reasoning chain.

Concurrently, research on Uncertainty Quantification (UQ) for LLMs aims to calibrate prediction confidence but focuses predominantly on the final output [Xia et al., 2025]. For instance, verbalized UQ methods elicit confidence estimates from black-box LLMs, yet these estimates remain opaque and non-traceable [Tian et al., 2023; Xiong et al., 2024]. It is impossible to discern whether the expressed uncertainty stems from flawed evidence, deficiencies in the model's own reasoning, or the intrinsic difficulty of the task. Therefore, existing UQ methods cannot synergize effectively with KGRAG to provide a stepwise-calibrated view of the complete evidence-to-prediction chain.

In summary, a principled solution for systematically managing the propagation of uncertainty in KG-augmented LLMs is lacking. To bridge this gap, we propose a novel doublecalibration paradigm. Its core lies in moving beyond basic evidence retrieval to the construction of a calibrated reasoning chain, where confidence is explicitly estimated and made traceable from the retrieved KG evidence to the final LLM prediction. As shown in Figure 1, this enables the LLM to weigh alternative answers (e.g., correctly favoring "Spike" over "Belle") based on the calibrated confidence of the supporting evidence, rather than making an overconfident guess.

We instantiate this principle in DoublyCal, a framework that implements double-calibrated KG-RAG. DoublyCal grounds the LLM's reasoning on verifiable KG evidence and performs dual calibration: it first calibrates the confidence of the retrieved evidence, then uses this calibrated evidence to guide and further calibrate the final LLM prediction. Specifically, we formalize KG evidence as constrained relational paths extracted from a KG. We then train a lightweight proxy model under Bayesian supervision to generate relevant KG evidence alongside a calibrated confidence score for each query. During inference, the primary LLM is prompted with both the KG evidence and its confidence estimate, leading to more accurate and better-calibrated predictions. Crucially, because the evidence confidence explicitly estimates the expected reasoning uncertainty of the LLM when utilizing the provided evidence, the final confidence becomes traceable to the verifiable KG evidence and its calibrated confidence, rather than remaining an opaque global estimate. Our main contributions are summarized as follows:

• We establish the principle of double-calibration for reliable KG-augmented LLMs, which mandates explicit confidence calibration for both the KG evidence and the final LLM predictions.
• We propose DoublyCal 1 , a framework that implements this principle via a Bayesian-calibrated proxy model, providing the primary LLM reasoner with KG evidence accompanied by evidence confidence.
• We empirically demonstrate that DoublyCal significantly and consistently improves the accuracy and calibration of diverse black-box LLMs on knowledgeintensive benchmarks in a cost-effective manner.

2 RelatedWork

2.1 Knowledge-Augmented Generation for Reliable LLMs

Retrieval-Augmented Generation (RAG) reduces the inherent knowledge gaps of LLMs by providing external information, thereby improving the factual accuracy of their responses [Zhang et al., 2025a]. The choice of knowledge source defines a spectrum of RAG variants, ranging from (i) unstructured text in Vanilla RAG [Guo et al., 2025; Sun et al., 2025], to (ii) textual graphs that model latent connections in GraphRAG [He et al., 2024; Li et al., 2025b], and finally to (iii) formal Knowledge Graphs (KGs) with explicit relations in KG-RAG [Luo et al., 2024; Li et al., 2025a]. By providing precise and structured knowledge, KG-RAG offers a rigorous foundation for complex reasoning and has demonstrated superior performance on knowledge-intensive tasks [Xiang et al., 2025; Pan et al., 2024].

1 The source code of DoublyCal is available at https://github.com/ luyy9apples/DoublyCal.

However, the retrieved knowledge itself may be noisy or insufficient, posing a persistent challenge to the reliability of LLM reasoning. To address this, some research dynamically selects external knowledge when it conflicts with the LLM's internal parametric knowledge [Liu et al., 2026; Zhang et al., 2025b]. Unlike prior work that focus on resolving knowledge conflicts, we introduce a double-calibration principle which explicitly quantifies confidence for both the retrieved evidence and the final LLM prediction, thereby systematically identifying epistemic boundaries.

2.2 Uncertainty Quantification for LLMs

Uncertainty Quantification (UQ) for LLMs aims to calibrate the confidence of model predictions to identify their epistemic boundaries [Huang et al., 2024]. While some studies incorporate uncertainty awareness during training [Stangel et al., 2025], most practical UQ methods often operate post-hoc and are categorized by model access [Xia et al., 2025].

For open-source LLMs, confidence is typically derived from internal states, such as the feature-space distribution of hidden embeddings [Chen et al., 2024; Vazhentsev et al., 2025] and the predictive entropy of the output distribution [Malinin and Gales, 2021]. For black-box LLMs, methods rely on API-based probing. A prevalent strategy generates multiple responses and evaluates their semantic consistency [Manakul et al., 2023; Kuhn et al., 2023; Lin et al., 2024]. A more efficient alternative is verbalized UQ, which directly prompts the LLM to verbalize its own confidence, eliciting introspective uncertainty estimates [Tian et al., 2023; Xiong et al., 2024; Tanneru et al., 2024]. Its plug-and-play nature and low cost make verbalized UQ readily integrable with KG-RAG, forming a strong single-calibration baseline that calibrates only the final output.

A fundamental limitation across prior UQ paradigms is their exclusive focus on the final output, deriving confidence solely from the LLM's internal states or self-assessment. This makes them vulnerable to the model's overconfidence biases due to the lack of an objective external anchor [Xiong et al., 2024]. Our double-calibration principle addresses this by first calibrating externally verifiable KG evidence, thereby establishing a traceable foundation for the entire reasoning chain.

3 Preliminaries

3.1 Uncertainty and Confidence

We extend the conceptualization of uncertainty and confidence for LLM outputs [Lin et al., 2024] to general predictive systems. Given an input x, a predictive system f produces a probability distribution over possible outputs Pf (o | x). The uncertainty of f regarding x is quantified by the dispersion (e.g., entropy) of this distribution. Conversely, the overall confidence of f can be defined inversely to this uncertainty. For a specific output oi, its confidence is directly associated with its assigned probability Pf (o = oi | x).

3.2 Knowledge Graph

A Knowledge Graph (KG) is a graph-structured database representing factual knowledge as a set of triples [Bollacker et al., 2008]. Formally, a KG is denoted as G := (V,R, E), where V is a set of entities, R is a set of relations, and E := (h, r, t) ⊆ V ×R× V is a set of factual triples. Each triple (h, r, t) represents an atomic fact, stating that relation r holds between head entity h and tail entity t. To mitigate the knowledge gaps of LLMs, KG-RAG provides them with structured evidence extracted from KGs [Xiang et al., 2025].

3.3 Knowledge Graph Question Answering

Knowledge Graph Question Answering (KGQA) is a canonical knowledge-intensive reasoning task [Yih et al., 2016]. Given a natural language question Q involving query entities VQ, a reasoning system is expected to retrieve relevant evidence from a KG G and reason over it to produce the correct answer set A. A standard knowledge-augmented pipeline (e.g., KG-RAG) involves two stages: (i) a retriever g that fetches a set of relevant evidence ZQ = g(Q; G), and (ii) a reasoner f (e.g., an LLM) that predicts answers ˆ A = f(Q; ZQ, G). This decomposition naturally highlights two distinct sources of uncertainty that our framework aims to calibrate: the evidence uncertainty in ZQ, and the reasoning uncertainty in generating ˆ A given ZQ.

4 Methodology

This section introduces DoublyCal, a framework designed to establish a calibrated reasoning chain by jointly calibrating both verifiable KG evidence and the final LLM predictions. As illustrated in Figure 2, the proposed DoublyCal framework operates through three core components. Firstly, we formalize KG evidence as constrained relational paths and employ a Bayesian model to estimate a statistically grounded confidence for each evidence (Sec. 4.1). Then, a lightweight proxy model is trained under the supervision of these Bayesian confidence scores to generate KG evidence alongside its calibrated confidence (Sec. 4.2). Finally, the calibrated evidence-confidence pair serves as an objective signal integrated into any black-box LLM to mitigate its inherent overconfidence, thereby enhancing both the calibration and traceability of its final predictions (Sec. 4.3).

4.1 Bayesian Calibration of KGEvidence

KG Evidence Formulation. Effective KG evidence must balance informativeness for accurate reasoning with interpretability for reliable confidence estimation. While relational paths [Luo et al., 2024] offer step-by-step interpretability, they may lack sufficient context. In contrast, subgraphs [Li et al., 2025a] provide broader context but often introduce redundancy. To resolve this trade-off, we introduce constrained relational paths as our primary evidence form. This formulation augments a core relational path with an optional constraint derived from the neighborhood of the candidate answer, thereby enhancing informativeness while preserving interpretability.

Formally, given a KG G and a question Q, a constrained relational path Pc is defined as the conjunction of a relational path Pr and an optional constraint C:

where Pr := ∃v1, . . . , vl-1.r1(q, v1) ∧ r2(v1, v2) ∧ · · · ∧ rl(vl-1, ˆa) denotes a directed relational path of length l from the query entity q ∈ VQ to a candidate answer ˆa. Here, each ri ∈ R is a relation, and vi is an existential variable. The optional constraint C := rc(ˆa, c) represents a one-hop triple from the candidate answer ˆa to a constraint entity c ∈ V, which serves to filter or refine the candidate set.

Example. Consider the question "What is the name of Snoopy's brother?" with the query entity q = Snoopy and the true answer a = Spike. Figure 2 illustrates a relational path evidence z3 and its constrained counterpart z2:

where |=G denotes grounding the evidence in G to obtain candidate answer entities. The auxiliary constraint on the candidate's gender effectively identifies ˆa = Spike, yielding a more precise and informative evidence for reasoning.

Confidence Estimation with Beta-Bernoulli Model. To estimate a statistically grounded confidence for a given KG evidence zQ, we model it as a predictive system in accordance with the definition in Sec. 3.1. Formally, the system takes as input a candidate answer ˆa drawn from the candidate set [[zQ]] obtained by grounding zQ in G (i.e., zQ |=G [[zQ]]), and produces a binary output indicating whether ˆa is correct (ˆa ∈ A). This defines a Bernoulli distribution for the correctness of a uniformly sampled candidate, characterized by the parameter p ∈ [0, 1], which is precisely the probability that the candidate is correct.

To obtain a robust estimate of p that accounts for KG incompleteness, we impose a conjugate Beta prior p ∼ Beta(α, β), with hyperparameters α, β > 0. Given Q and its answer set A, we use the posterior mean as the calibrated confidence score p∗ , which has the following closed form:

where ∣∣[[zQ]] ∩ A∣∣ counts the number of correct candidates, and ∣∣[[zQ]]∣∣ is the total number of grounded candidates. p∗ blends the empirical accuracy of the evidence with prior belief, mitigating the impact of sparse KG grounding.

Example (cont.). With a weakly informative prior set to α = β = 0.5 [Jeffreys, 1998], the statistical confidence for KG evidence z3 and z2 is estimated as p(A | z3) = 0.5 and p(A | z2) = 0.75.

4.2 Proxy for Evidence Generation &Calibration

The Beta-Bernoulli model-based confidence provides a statistically grounded but retrospective measure of evidence quality. To enable prospective evidence retrieval and confidence estimation during inference, we introduce a lightweight reasoning proxy. This proxy model is designed to generate high-quality KG evidence alongside well-calibrated confidence estimates for any input question, thereby approximating a reliable reasoning path before the LLM's final prediction. The proxy model is implemented by an LM-based Evidence Generator and trained using the Bayesian confidence scores as its supervisory signal.

Supervised Fine-Tuning (SFT) for Evidence and Confidence Generation. We formalize the dual task of evidence generation and confidence estimation as a sequenceto-sequence problem and conduct the initial training of the proxy model via SFT. This stage equips the proxy with the fundamental ability to identify relevant KG evidence and estimate confidence by mimicking the Bayesian signal.

The SFT training dataset is constructed from triples (Q, zQ, p(A | zQ)). Each triple is formatted into a structured sequence using a predefined template: the question serves as the instruction, and the target output is the evidence path enclosed in XML-style tags, with the Bayesian confidence score included as an attribute (e.g.,

<PATH confidence= . . . > . . . </PATH>). The proxy model fθ is trained via standard autoregressive language modeling to generate this target sequence.

Reinforcement Learning (RL) for Evidence Decision. We further refine the proxy by framing evidence generation and calibration as a sequential decision process optimized via RL. This stage transitions the proxy from imitation to strategic decision-making that jointly maximizes both inferential quality and confidence calibration.

For each question Q with a gold evidence set ZQ, we compute a reward for the generated evidence ˆ zQ and its predicted confidence ˆ c. Firstly, we define a match score m(ˆ zQ, zQ) ∈ [0, 1] for each zQ ∈ ZQ, which combines Jaccard similarity [Jaccard, 1901] with an order-sensitive Levenshtein ratio [Levenshtein and others, 1966]. The overall reward R is a weighted combination of an inferential quality reward Rinf and a calibration alignment reward Rcal:

Here, F1(zQ) is a precomputed F1 score assessing the reasoning capability of the gold evidence. Intuitively, a lower match score m(ˆ zQ, zQ) reduces both the inferential quality of ˆ zQ and its target confidence c. The weight λ ∈ (0, 1) balances the two objectives, and ξ > 0 is a tolerance coefficient. The final reward per generation is the maximum R over ZQ, followed by transformations to ensure a smooth training signal. The policy πθ of the proxy model is optimized to maximize the expected reward under the Group Relative Policy Optimization (GRPO) [Shao et al., 2024] objective.

4.3 LLMReasoning with Calibrated Evidence

The trained proxy equips any black-box primary LLM with double-calibration capability in a plug-and-play manner.

For a given question Q, the proxy model generates a set of candidate evidence-confidence pairs, i.e., ˆ ZQ = {(ˆ z(i) Q , ˆ c(i) )}i=1...K. Each ˆ z(i) Q is grounded in G, yielding factual reasoning paths that share the same confidence score ˆ c(i) . These paths and their confidences are verbalized into a natural-language context and integrated into prompts following prior verbalized UQ methods [Tian et al., 2023; Xiong et al., 2024]. Processing this enriched context, the LLM produces a final answer along with a well-calibrated prediction confidence. This design establishes a traceable chain of confidence and achieves double calibration: the proxy first calibrates the external evidence confidence, which then informs and refines the LLM's final prediction calibration through its own verbalized uncertainty estimation.

5 Experiments

5.1 Experimental Settings

Datasets and Evaluation Metrics. Following [Luo et al., 2024; Li et al., 2025a], we evaluate our framework on two widely-adopted Knowledge Graph Question Answering (KGQA) benchmarks: WebQSP [Yih et al., 2016] and CWQ [Talmor and Berant, 2018]. To measure prediction accuracy, we report Hits, Recall, and macro-averaged F1 score. To assess the reliability of confidence estimates, we report the Expected Calibration Error (ECE) [Guo et al., 2017] and the Adaptive Calibration Error (ACE) [Nixon et al., 2019]. ECE and ACE measure the expected absolute difference between empirical accuracy and predicted confidence using equal-width and adaptive equal-size bins, respectively.

Baselines. To rigorously evaluate our Double-Calibration mechanism, we construct Single-Calibration (SingleCal) baselines by extending reasoning paradigms with verbalized Uncertainty Quantification (UQ) methods, which elicit a selfreported confidence score alongside the predicted answer. We select three state-of-the-art reasoning frameworks: (i) The base LLM Reasoner without KG access; (ii) RoG [Luo et al., 2024], a KG-RAG method that grounds reasoning in retrieved relational paths; (iii) SubgraphRAG [Li et al., 2025a], which retrieves and reasons over KG subgraphs. Each framework is combined with three representative UQ prompting techniques: Vanilla [Tian et al., 2023], CoT [Kojima et al., 2022], and Self-Probing [Xiong et al., 2024].

Implementation Details. All evaluated methods employ GPT-3.5-turbo [Floridi and Chiriatti, 2020] as the primary reasoner unless otherwise specified, ensuring that performance differences are directly attributable to the calibration mechanism rather than the base LLM capability. The evidence proxy in our DoublyCal is implemented with Llama27B-Chat [Touvron et al., 2023], which is trained via the SFT then RL pipeline described in Sec. 4.2.

5.2 Main Results

Table 1 summarizes the comparative performance of our DoublyCal against all SingleCal baselines.

Superiority of Double-Calibration. DoublyCal achieves the best overall performance, consistently leading in all prediction metrics (Hits, Recall, F1) and the lowest ECE and ACE. Notably, it establishes a new standard for reliability, reducing the ECE and ACE to levels significantly lower than all SingleCal baselines. This result demonstrates that while KGRAG methods can enhance LLM factuality, calibrating both the external KG evidence and the final prediction is necessary for achieving reliable reasoning. Furthermore, the observed gains in prediction metrics indicate that evidence confidence helps improve the quality of the retrieved evidence and further unlocks the LLM's reasoning potential.

Calibrated Evidence as an Anchor for Verbalized UQ. The efficacy of verbalized UQ methods varies across reasoning backbones, with none proving universally dominant. This inconsistency arises because these methods may be subject to LLMs' inherent overconfidence. DoublyCal addresses this by supplying KG evidence with calibrated confidence, providing a reliable external anchor that refines the LLM's uncertainty expression. Consequently, DoublyCal stabilizes all three UQ techniques and achieves the lowest ECE and ACE in nearly every configuration, showing that externally calibrated evidence improves confidence elicitation.

Controlled Enhancement via RL. The RL stage yields a significant performance gain, improving average F1 by ∼3.0 percentage points over the SFT-only. While prior work notes RL's risk of harming calibration [Kalai et al., 2025], our Bayesian confidence-aligned reward successfully mitigates this trade-off, resulting in only a minor and controlled variation in ECE and ACE. This confirms that our reward design effectively balances accuracy and calibration.

5.3 Efficiency Analysis

We further analyze the input token efficiency of each method, with results shown in Figure 3.

Superior Cost-Effectiveness of DoublyCal. DoublyCal substantially outperforms RoG (F1 +8.0, ECE -12.4) with only a marginal increase in input token cost, while consuming only about 39% of the input tokens required by SubgraphRAG. This efficiency stems from the high information density of the evidence provided by DoublyCal. Specifically, compared to the simple relational paths in RoG, our constrained relational paths incorporate an optional constraint that yields more precise and informative evidence without compromising conciseness. Moreover, our proxy model is trained through evidence confidence calibration to select more discriminative KG evidence, further enhancing retrieval precision. In contrast, while SubgraphRAG's subgraphs offer broader context, their lower information density leads to disproportionately high token costs.

Efficiency Across Different UQ Methods. Self-Probing incurs approximately twice the input token cost of Vanilla or CoT due to its two-step prompting design. However, because DoublyCal and RoG retrieve concise evidence, they can effectively leverage Self-Probing's reflective "second-thought" process without excessive overhead. Notably, even when equipped with Self-Probing, their total input token cost remains below that of SubgraphRAG+Vanilla.

5.4 Ablation Analysis

We ablate DoublyCal against two variants (Table 2) to examine the contribution of each component: (i) SingleCal, which removes the calibrated evidence confidence and applies calibration only to the final LLM output; (ii) Evidence, which removes the LLM reasoner and directly outputs the terminal entity of the factual path ([[zQ]]) as the answer, using the evidence confidence as the final confidence score.

Evidence Confidence is Crucial for Final Prediction Calibration. Ablating from DoublyCal to SingleCal reveals a stark outcome: while predictive accuracy remains stable (e.g., F1 changes within ±1 point), the calibration error (ECE and ACE) increases drastically from ∼4 to >20. This indicates that externally calibrated evidence confidence is essential, as it provides the LLM with a reliable anchor for its selfassessment. Without this first-stage calibration, the black-box LLM cannot reliably judge its own certainty, even when it can identify correct answers using high-quality KG evidence.

The LLMReasoner Enables Integrative Reasoning. The significant performance gap of the Evidence variant underscores a pivotal design insight: the evidence proxy and the LLM reasoner play distinct yet complementary roles. The proxy specializes in evaluating individual KG evidence, while the LLM reasoner excels at synthesizing an ensemble of such evidence to perform complex reasoning. Consequently, the final prediction confidence is not a simple pass-through of any single evidence confidence, but rather the result of the LLM's holistic reasoning over the entire set of calibrated evidence. This efficient proxy-reasoner synergy is essential to the framework's performance.

5.5 Cross-model Compatibility Analysis

To assess generalizability of DoublyCal, we evaluate it across diverse black-box LLMs, including GPT-3.5-turbo, GPT-4omini [Achiam et al., 2023], DeepSeek-V3 [Liu et al., 2024], and Gemini-2.5-flash [Comanici et al., 2025].

Performance-Reliability Trade-off in LLM Reasoners. Figure 4 reveals a clear trade-off between accuracy and calibration among standalone LLMs. While GPT-family models and DeepSeek achieve comparable accuracy with moderate calibration errors (F1: 43.3-44.6; ECE: 23.9-27.7), Gemini attains a notably higher F1 (51.8) at the cost of a significantly worse ECE (41.4). This pattern highlights a common pitfall where optimizing purely for accuracy often degrades reliability in standalone LLMs. Confidence distributions (Figure 5) confirms that all models exhibit systematic overconfidence. This issue is most acute in Gemini, where roughly 80% of predictions are made with maximal confidence (1.0), yet the accuracy within this high-confidence group is only about 0.6.

DoublyCal Systematically Decouples the Trade-off.

DoublyCal delivers consistent and substantial improvements across all models, effectively decoupling this trade-off. As shown in Figure 4, it raises the average F1 from 45.8 to 76.5 while reducing the average ECE from 29.4 to 7.6. Crucially, DoublyCal mitigates the overconfidence patterns observed in black-box LLMs (Figure 5), shifting confidence distributions toward well-calibrated and high-accuracy regions. By grounding confidence in externally calibrated evidence, DoublyCal provides a generalizable solution that enhances both accuracy and reliability of diverse black-box LLMs.

Sample

Question: Where did George W. Bush live as a child?

Query Entity (q): George W. Bush Answers: NewHaven.

Retrieval:

• q → people.person.place of birth → New Haven [Confidence: 0.8]
• q → people.person.place of birth → New Haven →

location.location.containedby→Connecticut [Confidence: 0.5]

• q → people.person.place of birth → New Haven →
• location.location.containedby → United States of America [Confidence: 0.5] Predictions: {Connecticut: 0.3}

Retrieval:

• (q, people.person.place of birth, NewHaven)
• (q, people.person.nationality, United States of America)
• (m.03prwzr, people.place lived.location, Midland)
• (m.02xlp0j, people.place lived.location, Washington, D.C.) · ... Predictions: {Midland: 1.0}

Table 3: Case study: DoublyCal vs. SingleCal baseline.

Sample

Question: Where was Martin Luther King, Jr. raised?

Query Entity (q): Martin Luther King, Jr. Answers: Atlanta.

Retrieval:

• q people.person.place of birth→Atlanta [Confidence: 0.8]
• q →people.deceased person.place of death→Memphis [Confidence: 0.8] Predictions: {Atlanta: 0.8, Memphis: 0.1}

RL-DoublyCal (SingleCal) + Vanilla

Retrieval: Same factual paths as above, without confidence scores

Predictions: {Atlanta: 0.8, Memphis: 0.2}

Table 4: Case study: DoublyCal vs. its SingleCal variant.

5.6 Case Studies

This section presents two case studies that qualitatively illustrate how DoublyCal enhances reliability of LLM predictions. As shown in Table 3, for a question lacking a direct KG relation ("lived as a child") to provide accurate support, both methods retrieve the related birthplace fact. However, while SubgraphRAG retrieves scattered evidence leading to an overconfident error (confidence 1.0), DoublyCal presents concise paths with calibrated confidence scores. Its birthplace path receives high confidence (0.8), while less relevant expansions are scored lower (0.5). Consequently, the LLM correctly assigns low confidence (0.3) to the incorrect answer "Connecticut". Furthermore, the ablation study in Table 4 demonstrates that explicitly providing evidence confidence makes the LLM's predicted confidence more concentrated on the correct answer. Together, these cases show DoublyCal's ability to mitigate overconfidence and improve calibration.

6 Conclusion

This paper establishes the principle of double-calibration for constructing a calibrated reasoning chain from KG evidence retrieval to final LLM prediction. We implement this principle in DoublyCal, a reliable KG-RAG framework that integrates plug-and-play verbalized uncertainty quantification, thereby enhancing the traceability and reliability of diverse black-box LLMs. Our work offers a concrete step toward building more reliable and transparent LLM systems, contributing to the advancement of trustworthy AI.

RL-DoublyCal (Full) + Vanilla

SubgraphRAG + Vanilla

RL-DoublyCal + Self-Probing

Acknowledgments

We thank the reviewers for their valuable suggestions. This work has been supported by the National Natural Science Foundation of China (62372483), a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (UGC/FDS16/E23/24), and the Hong Kong Research Grants Council under the Theme-based Research Scheme (T41-517/25-N). This work has also been supported by the Key Special Project of National Natural Science Foundation of China (72442017) and the General Research Funds from the Hong Kong Research Grants Council (PolyU 15207322, 15200023, 15206024, and 15224524).