ScholarSum: Student-Teacher Abstractive Summarization via Knowledge Graph Reasoning and Reflective Refinement

1 Introduction

Abstractive summarization is critical for efficient understanding of scientific literature in domains such as medicine, computer science, and the natural sciences [Cohan et al., 2018; Zhou et al., 2025]. In practice, effective abstractive summarization requires not only producing linguistically fluent text, but also preserving factual faithfulness to the original scientific claims [Gao et al., 2023]. Fundamentally, abstractive summarization can be viewed as learning to reorganize and condense document-level information into a coherent and faithful textual representation [Nan et al., 2021].

† Corresponding author: Mingyue Cheng.

The field of abstractive summarization has evolved through distinct technological developments. Early research predominantly focused on extractive methods, which relied on statistical heuristics to select representative sentences from the source text [Erkan and Radev, 2004]. While such approaches tend to preserve factual content, their reliance on sentence selection often limits linguistic fluency. The advent of deep learning shifted the focus toward abstractive summarization, where encoder-decoder models such as BART and T5 significantly improved the fluency and expressiveness of generated text [Lewis et al., 2020a; Raffel et al., 2020]. Recently, LLMs have further transformed the field by exhibiting strong zeroshot capabilities and generating coherent narratives without task-specific training [Brown et al., 2020]. However, despite these advances in linguistic fluency, ensuring factual faithfulness remains a significant challenge. This observation raises a fundamental question: Can abstractive summarization fully reconcile linguistic fluency with factual faithfulness?

Motivated by this question, as shown in Figure 1 (left), we observe that in real-world writing practice, a student typically first reads and internalizes a paper by organizing its key ideas into a hierarchical mental representation, and then produces an initial summary that is subsequently refined through iterative feedback from a teacher. This process often leads to summaries that maintain linguistic fluency while progressively improving factual accuracy [Xu et al., 2024]. Inspired by this observation, as shown in Figure 1 (right), we explore an underexplored approach that formulates abstractive summarization as a hierarchical content understanding and generation problem guided by LLMs. The underlying intuition is that modeling document-level structure ensures global coherence, while iterative refinement progressively corrects missing or unsupported content. However, this approach introduces several non-trivial challenges. First, constructing a hierarchical representation that faithfully reflects a document's logical structure is non-trivial, especially for long and complex scientific texts [Liu et al., 2024]. Second, even with structural guidance, effectively integrating local evidence to ensure factual precision remains difficult, as global structural coherence does not necessarily guarantee fine-grained factual grounding [Sun et al., 2024].

To address these challenges, we propose ScholarSum, a hierarchical reflective graph-based framework inspired by the student-teacher writing process, designed to generate summaries that are both linguistically fluent and factually faithful. Specifically, ScholarSum first organizes the document into a hierarchical knowledge graph by segmenting it into semantically coherent units, where the multi-layered community structure corresponds to semantically coherent themes and provides a global outline for summary planning. Guided by this hierarchical structure, the student generates an initial draft, ensuring fluent expression and coherent organization. To systematically enforce factual consistency, a teacher-like reviewer then iteratively examines the draft, identifies missing or unsupported content, and prompts targeted re-retrieval and rewriting until the summary meets rigorous quality standards. This hierarchical and reflective design allows ScholarSum to fully leverage the capabilities of LLMs in semantic understanding, contextual reasoning, and coherent text generation, while explicitly addressing the dual requirements of fluency and faithfulness. Extensive experiments on benchmark scientific summarization datasets show that ScholarSum significantly outperforms previous baselines.

2 RelatedWork

In this section, we review prior work relevant to abstractive summarization, focusing on two major lines of research.

2.1 Conventional Abstractive Summarization

Early approaches to summarization relied heavily on extractive methods, where sentences were scored using statistical or neural metrics and top-ranked sentences were concatenated to form summaries. Although effective at capturing salient content, such methods often resulted in fragmented narratives with limited coherence, repetition, and poor stylistic fluency [Erkan and Radev, 2004]. Generative approaches based on pre-trained language models (PLMs) partially alleviated these issues by enabling end-to-end abstraction. Sequence-to-sequence architectures improved surface fluency, and pointer-generator mechanisms with coverage helped balance abstraction and copying [See et al., 2017]. Further tuning-based methods, including task-specific pretraining objectives such as gap sentence prediction and unified text-to-text frameworks, established strong baselines on scientific summarization datasets [Zhang et al., 2020a; Raffel et al., 2020]. Iterative refinement strategies, inspired by human writing processes, introduced cycles of drafting, critique, and revision [Sun et al., 2024; Li et al., 2024]. Related selfrefinement frameworks also demonstrate the value of explicit feedback loops for improving factual consistency [Madaan et al., 2023]. However, these PLM-based approaches still struggle to maintain coherent and globally consistent summaries over long documents, particularly when cross-section integration is required.

2.2 LLM-based Summarization

LLM-based methods leverage large-scale models to handle longer contexts and improve discourse-level fluency. One representative strategy decomposes summary generation into intermediate reasoning steps, enabling more coherent and structured outputs. Stepwise decomposition and prompt chaining outperform single-pass generation, particularly for scientific and technical texts [Xu et al., 2023]. Another line of work adopts retrieval-augmented generation (RAG), where LLMs retrieve key fragments or subgraphs of relevant documents prior to reasoning [Asai et al., 2024; Sarthi et al., 2024]. Graph-aware retrieval further exploits document structure, such as citation and discourse relations, to support multi-hop reasoning and improve global coherence [Edge et al., 2024; Peng et al., 2025]. Related graphbased planning and retrieval methods further highlight the benefits of structured evidence organization [Hu et al., 2025; Chen et al., 2024]. Recent studies on scientific paper mining suggest that explicit search and tool use can enhance literature understanding [Wang et al., 2025; Pan et al., 2026]. Memoryaugmented generation also shows that structured memory can improve downstream reasoning and personalized generation [Tao et al., 2026; Yu et al., 2026].

Tuning-based methods incorporating graph constraints further introduce structural priors into the generation process. By guiding decoding with subgraph-level plans or relation constraints, these approaches combine the fluency of PLMs with evidence-aware generation [Li et al., 2025; Peng et al., 2025]. Despite their substantial improvements in fluency, LLM-based methods remain prone to hallucination and weak evidence grounding, especially when summaries must faithfully reflect complex scientific evidence. This challenge highlights the need for mechanisms that not only enhance generation quality but also enforce explicit verification and logical coherence. Recent verification-oriented methods further reduce hallucinations by prompting models to generate, check, and revise their own outputs through explicit verification steps [Dhuliawala et al., 2024; Gou et al., 2024].

3 Methodology

In this section, we first define the problem, then present an overview of the proposed method, and finally describe the individual components in detail.

3.1 ProblemDefinition

Given a source paper D, the objective of scientific summarization is to generate a summary S that preserves the document's logical topology and factual details. To achieve this objective, ScholarSum formulates summarization as an iterative refinement process. Specifically, ScholarSum proceeds for at most T iterations. At iteration i, the student produces a candidate summary S(i) student by reasoning over a documentderived knowledge graph G, conditioned on the teacher's structured feedback from the previous round, F (i-1) T . The teacher then evaluates S(i) student , returning a quality score σ(i) and updated feedback F (i) T . The process terminates when σ(i) ≥ θ or when the iteration budget T is reached.

3.2 Overview

Figure 2 presents the ScholarSum framework for fluent and faithful scientific summarization. It proceeds through three main steps. First, hierarchical graph construction extracts entities and relations from source sentences to form subgraphs, capturing document-level structure. Second, student abstract generation produces coherent draft summaries guided by these subgraphs. Third, teacher-like refinement iteratively evaluates and revises the draft using retrieved similar articles, calibrating scientific writing style, expected content coverage, and discourse organization.

3.3 Hierarchical Graph Construction

The student drafts candidate summaries by operating over an explicit semantic workspace derived from the source paper. The key idea is to leverage document structure for planning and textual evidence for verification, ensuring that fluency and factuality reinforce each other rather than trade off.

To provide a manipulable representation of the paper's core claims, we construct a knowledge graph G = (V,E) for a given paper D. We first split D into sentence-level or paragraph-level textual units and use an instruction-tuned LLM with a constrained JSON schema to extract scientific entities V . Entities are typed into categories common in scientific writing, including tasks, methods, datasets, and metrics. For relation detection, we enumerate entity pairs within the same or adjacent textual units and classify them into typed relations, such as addresses, uses, evaluates-on, reports, and compares-with. By externalizing the paper into this structured workspace, we enable both global planning and local evidence selection, providing a clear basis for coherent summary generation. Scientific papers often contain multiple coherent threads, including problem motivation, methodological design, and empirical findings. To identify these thematic threads, we partition G using the communities algorithm [Traag et al., 2019]:

where each community Gi corresponds to a focused theme. Tiny communities are removed and highly overlapping ones are merged to produce a stable plan that is neither fragmented nor overly coarse. This thematic planning ensures that the generated summaries respect the paper's internal structure and maintain coherence across different sections.

3.4 Student Abstract Generation

Directly generating a scientific summary in a single pass often leads to either weak discourse organization or factual inaccuracies, as models lack an explicit mechanism to plan content and verify evidence jointly. To address this issue, the student treats the constructed graph not as a static representation but as an explicit semantic workspace that supports structured planning and evidence-aware generation.

Concretely, the student interacts with the graph through two complementary operations. Thematically, it navigates across communities to determine which themes to cover and in what order, using the community structure as a high-level outline where each Gi corresponds to a candidate episode in the summary. Factually, it gathers evidence by querying salient triplets in the neighborhood of key nodes, such as proposed methods or evaluation settings, which serve as factual anchors for generation. Through this iterative interaction, the student repeatedly consults and updates its workspace, using structure to guide discourse flow and evidence to stabilize factual details. Following this workspace-based view, the student generates summaries in two steps, reflecting a drafting-verification workflow. For each community Gi, the model produces a short sub-summary si capturing the central contribution of that theme. These sub-summaries are then composed into a draft:

where ⊕ denotes concatenation in thematic order. This draft provides a coverage-oriented scaffold that emphasizes completeness and logical organization. In the initial round, the draft is constructed without external feedback, while in later rounds, it can be lightly re-planned based on guidance from the teacher, such as reordering themes or expanding insufficiently covered content.

To improve factual fidelity, the student selects a compact set of key triplets from the graph to form a context subgraph G(i) context. This selection is guided by domain-specific cues that commonly signal high-value evidence in scientific writing, including experimental configurations, dataset identifiers, and main quantitative results. Conditioning on G(i) context and the teacher's feedback, the student revises the draft as

with F (0) T = ∅. By grounding revision in explicit evidence, this step corrects issues identified during evaluation and produces a more fluent and factually reliable summary.

3.5 Teacher-like Refinement

Although the student is guided by structure and evidence, draft summaries may still contain omissions, logical inconsistencies, or subtle factual inaccuracies. This is because selfgenerated summaries lack an external reference for domain calibration and systematic quality control. To address this issue, we introduce a teacher-like refinement process that evaluates candidate summaries from a reviewer perspective and provides explicit signals for further improvement.

To ground evaluation in the target scientific domain, the teacher retrieves k similar papers {D′j : j = 1, . . . , k} and their abstracts {A′j : j = 1, . . . , k} from a reference corpus constructed from the training split of the

(a) Hierarchical Graph Construction

(b) Student Abstract Generation

(c) Teacher-like Refinement

corresponding benchmark, excluding the test paper itself. We encode the title and abstract of each paper using text-embedding-3-small and perform cosinesimilarity KNN search over normalized embeddings. These retrieved abstracts serve as in-domain exemplars that help the teacher calibrate expected abstract structure, terminology usage, and the level of detail typically included in scientific summaries. The teacher then evaluates S(i) student against these in-domain references and outputs both a scalar score and structured feedback:

where σ(i) reflects overall quality, and F (i) T records concrete revision targets, such as missing key contributions, incorrect factual statements, misordered logic, or unclear phrasing.

Based on σ(i) , the Teacher selects one of three actions using thresholds θmajor and θminor, where θminor ≥ θmajor. When σ(i) ≥ θminor, the summary is accepted and returned as the final output, denoted as Accept. When θmajor < σ(i) < θminor, the Teacher performs a light edit guided by F (i) T , denoted as Minor revision:

where Fminor rev(·) denotes the minor revision function, which lightly edits the candidate summary S(i) student according to the Teacher's feedback F (i) T . When σ(i) ≤ θmajor, the Teacher returns F (i) T to the Student as a conditioning signal for the next generation round, denoted as Major revision.

By explicitly separating evaluation from generation and grounding feedback in retrieved in-domain references, this refinement process reduces hallucinations, improves factual fidelity, and provides a principled stopping criterion for iterative summarization. The interaction continues until the quality threshold is met or the iteration budget is reached.

4 Experiments

In this section, we evaluate the effectiveness and factual reliability of ScholarSum. Experiments are conducted on multiple standard scientific summarization benchmarks, with comparisons against strong baselines.

4.1 Experimental Settings

Datasets and Metrics. We evaluate ScholarSum on multiple scientific summarization benchmarks: ArXiv and PubMed [Cohan et al., 2018]. For evaluation, we report ROUGE (R-1, R-2, R-L) [Lin, 2004], METEOR [Banerjee and Lavie, 2005], and BERTScore [Zhang et al., 2020b] to assess lexical coverage, semantic matching, and contextual similarity. ROUGE emphasizes lexical overlap, METEOR captures semantic matching with stemming and synonymy, and BERTScore measures contextual alignment via encoders.

Baselines. We compare against three groups of competitive baselines: (1) Traditional encoder-decoder summarization models, including T5 [Raffel et al., 2020], LED [Beltagy et al., 2020], and PEGASUS [Zhang et al., 2020a]; (2) LLMbased prompting methods, namely SumCoT [?] and QAprompting [Sinha, 2025], instantiated with both DeepSeek and Qwen base models; (3) Naive RAG [Lewis et al., 2020b], a retrieval-augmented LLM method. This baseline is particularly important because ScholarSum also uses retrieval; thus, it isolates the benefit of structured graph reasoning and teacher-guided refinement beyond adding retrieval alone. To ensure a fair comparison, all LLM-based baselines share the same backbone models as ScholarSum (i.e., DeepSeek and Qwen). We also enforce consistent decoding constraints, such as temperature and output length budget, across all methods whenever applicable.

4.2 Main Results

Table 1 comprehensively compares summarization performance across the ArXiv and PubMed benchmark datasets.

ScholarSum demonstrates strong performance in most scenarios, often achieving the best or second-best scores across both lexical (ROUGE) and semantic (BERTScore) metrics. Notably, LLM-based baselines like QA-prompting also show competitive results, particularly in generating linguistically fluent text. This further validates the potential of leveraging large language models for scientific abstract writing. However, these models often lack the explicit structural guidance and self-correction mechanisms introduced by our framework, limiting their ability to preserve both content coverage and factual consistency. Retrieval-based baselines such as Naive RAG perform moderately in capturing local evidence, but their performance drops regarding structural coherence (R-L) due to the fragmentation of retrieved chunks.

This contrast underscores the importance of balancing global logical planning with fine-grained evidence grounding. Beyond the overall metrics, the performance gap is particularly pronounced in the PubMed dataset, where scientific terminology and dense experimental data are more prevalent. While standard generative methods often struggle with the domain-specific complexity of medical literature, ScholarSum maintains high precision through its iterative verification loop. This suggests that our approach is especially effective for documents requiring a high degree of technical accuracy. In summary, the consistent gains can be attributed to the core design of ScholarSum: constructing a hierarchical reflective graph to guide a student-teacher iterative process. This unified approach effectively reconciles the inherent conflict between linguistic fluency and factual faithfulness, addressing the critical challenges and structural consistency requirements in real-world scientific summarization.

4.3 Factual Consistency Analysis

Factuality Metric. We evaluate factual consistency using MiniCheck [Tang et al., 2024], which estimates whether the generated summary is supported by the source document. To discourage trivially short but factually safe summaries, we introduce a brevity penalty. Let c and r denote the token lengths of the generated summary and the reference abstract, respectively. The brevity penalty is defined as:

We then compute the final factuality score as:

where SMiniCheck is the MiniCheck score between the generated summary and the source document. A higher Sfact indicates that the summary is better supported by the source document while maintaining sufficient content coverage.

Score

1.0

0.8

0.6

ScholarSum

w/o Factual

Grounding Score

1.0

0.8

0.6

ScholarSum

w/o Minor Rev.

w/o Major Rev.

Figure 3 analyzes how different architectural components contribute to the factual reliability of generated summaries. Incorporating the hierarchical reflective framework leads to substantial improvements, whereas variants using standard prompting or vanilla retrieval consistently underperform. We further examine two key components: hierarchical graphbased structure construction, which supports macro-level thematic organization and global structural guidance, and the refinement mechanism, which enables iterative claim-level verification and correction. While both components improve factual consistency, the refinement mechanism typically yields larger gains in reducing hallucinations. The strong performance of the full model, particularly its clear advantages on ArXiv and PubMed, demonstrates its ability to integrate global structural context with fine-grained evidentiary verification, thereby enhancing summary faithfulness without compromising content richness.

4.4 Ablation Analysis

To assess the contributions of key design choices, we perform systematic ablations on both the Student and Teacher. The results show that factual grounding is necessary for faithful detail retention, while teacher-guided revision is essential for correcting content selection and discourse organization errors that persist in one-shot generation.

Ablation on the StudentModule. Figure 4 (left) examines how factual extractive grounding influences generation performance. The variant without factual grounding consistently underperforms across all metrics, suggesting that generation without explicit evidence support is less reliable. In contrast, incorporating extracted factual evidence provides stable and significant gains, underscoring the importance of evidencebased anchoring for factual correctness. The superior performance of the full model indicates its ability to effectively integrate extracted evidence into the generation process, thereby enhancing factual accuracy.

Ablation on the Teacher Module. Figure 4 (right) analyzes the contribution of the teacher module to overall performance. Removing the teacher leads to the largest performance drop among all ablation settings, indicating that reflective revision plays a central role in improving abstract quality. We further examine different revision pathways within the teacher. The major revision pathway yields more substantial gains than minor edits, suggesting that high-level feedback on content selection and discourse organization is more critical for scientific abstracts. In particular, omissions of core contributions or misplacement of key results tend to degrade overall quality more severely than surface-level wording issues.

4.5 Exploration Analysis

Stability Analysis. Figure 5 shows the performance of ScholarSum across five independent runs with different random seeds. Overall, the results demonstrate a high degree of consistency on all evaluation metrics. The standard deviations of ROUGE scores remain very small (e.g., approximately 0.004 for ROUGE-L), and similarly low variance is observed for METEOR and BERTScore, indicating that the framework is not sensitive to random initialization. Across runs, performance fluctuations are minimal, and no metric exhibits notable instability. This behavior can be attributed to the structured workflow of ScholarSum. The semantic workspace constrains global planning and content organization, while the teacher provides corrective feedback during reflective revision, together reducing the impact of stochastic decoding noise and leading to reliable generation quality.

Hyperparameter Sensitivity Analysis. Table 2 analyzes how two key hyperparameters-decoding temperature and logical keyword selection strategy-affect generation performance. Adjusting temperature reveals a trade-off: lower values make the model overly conservative, limiting diversity, while higher values introduce randomness that reduces coherence and increases factual errors. The results show that a temperature of 0.8 best balances diversity and factual reliability. Different keyword selection strategies also influence content fidelity and relevance, highlighting the importance of hyperparameter tuning. The performance under settings demonstrates the model's ability to generate coherent, factually grounded text while maintaining sufficient variability.

Table 3 presents a comparative analysis of keyword selection strategies for evidence grounding. Overall, explicit query constraints clearly outperform unguided generation, indicating that structured retrieval guidance is important for anchoring high-value facts, such as dataset names and numerical results, and thus helps reduce hallucinations. We also observe a trade-off between precision and coverage across different query variants, as reflected by the variations in ROUGE-2 and ROUGE-L. Notably, the LLM-adaptive strategy achieves performance comparable to manually designed queries. Although it does not consistently achieve the best results across all metrics, it reduces the dependence on hand-crafted query templates, suggesting the potential of more automated evidence selection under realistic deployment scenarios.

4.6 Case Study Analysis

To qualitatively illustrate our iterative refinement process, we present a case study on the influential paper "Attention Is All You Need" [Vaswani et al., 2017]. Figure 6 shows how the generated summary evolves across multiple revision stages. The initial draft captures the core idea of replacing recurrent and convolutional architectures with self-attention, but it lacks quantitative evidence and a clear argumentative structure. Guided by the teacher module, the major revision reorganizes the summary into a problem-solution narrative and incorporates key empirical results, such as BLEU scores and training efficiency. The final revision further improves conciseness, terminology precision, and abstract-style writing.

This progression shows that ScholarSum can address common summarization errors, including missing numerical evidence, terminology drift, and weak discourse organization, through teacher-guided iterative refinement.

5 Conclusion

In this work, we proposed ScholarSum, a hierarchical reflective framework for faithful and fluent scientific summarization. ScholarSum organizes source documents into hierarchical knowledge graphs to capture global logic and guide structured draft generation, and then employs a teacher-like refinement process to detect unsupported content and trigger targeted revision. This student-teacher design combines macrolevel discourse planning with fine-grained evidence grounding, thereby improving both content completeness and factual faithfulness. Experiments on ArXiv and PubMed show that ScholarSum consistently outperforms strong baselines, while ablation and stability analyses further validate the effectiveness and robustness of its key components. Future work will extend ScholarSum to broader scientific domains and incorporate multimodal evidence from figures and tables.

Acknowledgments

This research was supported by grants from the National Natural Science Foundation of China (No. 62502486), the Provincial Natural Science Foundation of Anhui Province (No. 2408085QF193), USTC Research Funds of the Double First-Class Initiative (No. YD2150002501), and the Fundamental Research Funds for the Central Universities of China (No. WK2150110032).

Contribution Statement

Bohou Zhang and Xiaoyu Tao contributed equally to this work. Mingyue Cheng supervised the project, provided guidance throughout the study, and served as the corresponding author. All authors reviewed the final manuscript.