LECDPR:LLM Enhancement and Concept-Document Interactive Modeling for Prerequisite Relation Prediction

1 Introduction

In the fields of education and knowledge engineering [Agrawal et al., 2016; All`egre et al., 2023], how to systematically organize the concepts and plan reasonable learning paths [Xianghan Christine and Stern, 2022; Yu et al., 2024] is one of the core issues for improving learning effectiveness. Concept prerequisite relation learning [Bai et al., 2025], as a key natural language processing task, aims to automatically identify the dependency relations between concepts, and determine whether mastering one concept is a prerequisite for learning another. For example, understanding "addition" is a prerequisite for learning "multiplication", which is of vital importance for building adaptive learning systems [Pech´aˇc et al., 2024], designing intelligent course outlines [Wang and Liu, 2016], and implementing personalized learning path recommendations [Dong et al., 2024].

∗Corresponding author.

Currently, the mainstream methods for concept prerequisite relation learning mainly develop along two technical routes. The first methods uses the relations between concepts and documents to build a knowledge network [Jia et al., 2021], employing GNN methods to learn the features of concept pairs [Sun et al., 2022; Manrique et al., 2018]. This approach leverages the powerful neighborhood information aggregation and propagation capabilities of graph neural networks, for example [Mazumder et al., 2023; Sun et al., 2024; Zhang et al., 2025a]. The second is based on text information, which directly delves into the text to extract clues about concept relations. This method focuses on extracting descriptive evidence related to concepts from text content (such as textbooks, Wikipedia) and using it as the basis for judging the relations between concepts [Chen et al., 2018]. For example, by analyzing the definitions or contextual information of concepts, identifying indicative phrases like "is a prerequisite", "needs to be mastered first", or through semantic analysis to assess the hierarchical and dependency strength between concepts, for example [Zhang et al., 2024; Wang et al., 2024a].

However, existing research still faces two significant challenges. Traditional text extraction method is often corrupted by redundant wording, ambiguous sentences, and other noise, causing downstream models to deviate. Therefore, as shown in Figure 1, we employs LLMs for a "second-stage" refinement: via prompt-based learning, the models map raw text into clean, structured latent representations and then distill key concepts, or confidence scores, markedly suppressing noise and enhancing signal purity. Moreover, due to inherently relations between documents and concepts, these interactions may respect the learning order of both documents and concepts. Modeling such constrained interactions is a complex task. Current graph-based methods may have failed to capture complex interactions, and graph-based methods and text-based methods are often studied in isolation, failing to effectively combine their advantages, which limits further improvements in downstream task prediction performance.

To address these challenges, we propose a LLM Enhancement and Concept-Document Interactive Modeling for Prerequisite Relation Prediction (LECDPR) model. Our idea involves three steps: Firstly, innovatively employing a two-stage LLM prompt to generate high-quality semantic descriptions of concepts from documents, which are then encoded into semantic space embeddings; Secondly, the complex interactive relations between concepts and documents are modeled through three propagation mechanisms to generate realtional embeddings, implementing dual-space alignment and achieving fusion of two concept embedding spaces via adaptive fusion; Finally, we use the fused embeddings to predict prerequisite relations more accurately.

The main contributions of this study are as follows:

• We introduce LLMs into the concept prerequisite prediction task, using a two-stage prompt to generate highquality concept texts, and then employing a word vector generator to obtain semantic embeddings of concepts, significantly improving the quality of concept semantic representations.
• We model complex interaction relations between concepts and documents to obtain relational space embeddings, then achieve dual-space alignment and dynamically fuse semantic and relational information.
• We compare our method with existing prerequisite relation prediction models on three public datasets, and the experimental results verify the effectiveness and superiority of our model.

2 RelatedWork

The core goal of concept prerequisite relation learning is to automatically identify "learning before/learning after" dependencies between concepts from educational resources [Chanaa and Faddouli, 2024], providing a foundation for personalized learning paths and knowledge graph construction [Wang et al., 2021]. Current research on prerequisite relations mainly follows two technical routes: graph-based methods and text-based methods.

2.1 Graph-based Concept Prerequisite Relation Prediction

Graph-based methods treat concepts as nodes and relations as directed edges, using Graph Neural Networks (GNNs) to iteratively propagate neighbor information in the topological space, thereby learning concept embeddings and predicting prerequisite relations between concepts. For example, HGAPNet [Mazumder et al., 2023] constructs a "documentconcept" heterogeneous graph, explicitly characterizing different semantic associations with three types of weighted edges. LCPRE [Sun et al., 2024] uses TF-IDF to obtain sparse RC-graphs, employing multi-heads self-attention to aggregate single-path semantics, then fusing multi-path signals through inter-path attention. DGCPL [Zhang et al., 2025a] combines the "concept-resource hypergraph" and "learning behavior directed graph" into a dual-view, using gated knowledge distillation to fuse node representations from both graphs for prerequisite prediction. GKROM [Zhang et al., 2025b] optimizes three types of edges (conceptconcept, document-concept, and document-document) on heterogeneous graphs, training multi-task GNNs to improve the accuracy of prerequisite prediction.

2.2 Text-based Concept Prerequisite Relation Prediction

Prerequisite relation prediction methods based on text convert relation judgment into text matching or entailment reasoning tasks, extracting concept co-occurrences, contextual statements, or teaching hints from free texts such as course documents and Wikipedia entries, then encoding semantic evidence with pre-trained language models or word vector encoders. For example, EBCPL [Zhang et al., 2024] classifies documents by topic, then uses three heuristic rules to extract evidence sentences that best illustrate the order of concepts, followed by encoding the paths with BiLSTM, and taking the maximum prediction probability across all evidence for each concept pair as the final output. MTN [Wang et al., 2024b] encodes three features ("resource view", "relation view", and "text view") with three parallel transformers, and fuses these features through an attention mechanism to achieve end-toend text-driven precedence relation prediction.

We integrate text and graph-structured approaches: LLMs [Becerra et al., 2024] generate high-quality concept descriptions to neutralize the noise plaguing traditional text-based methods, while structured relational knowledge and deep semantic information are jointly fused to bridge the information gap between the two paradigms, yielding markedly more accurate concept prerequisite predictions.

3 ProblemDefinition

In this section, we introduce the key terminology and research definitions. Let C = {c1, c2, · · · , cn} denote the set of knowledge concepts, where a knowledge concept refers to a core knowledge unit in a specific domain. D = {d1, d2, · · · , dm} represents the set of learning documents, with each document containing one or more concepts.

Our goal is to predict binary labels yij ∈ {0, 1} for any ordered pair of concepts (ci, cj) :

The model needs to learn a mapping from f : C × C → [0, 1] so that f (ci, cj) = yij , providing a reliable basis for personalized learning path planning.

4 Methodology

Our proposed method LECDPR includes the following three modules(Semantic Space Embedding Generation, Concept Embedding Alignment, and Concept Prerequisite Relation Prediction). Firstly, we leverage LLMs to produce concept embeddings that serve as the semantic space representations. Secondly, built upon these semantic embeddings, we model the intricate concept-document interactions and employ graph neural networks to generate relational space concept embeddings. We then align the two space embeddings

via masking and contrastive learning, and adaptively fuse the

two kinds of embeddings into unified representations. Finally, the fused concept embeddings are used to predict prerequisite relations among concepts. Overall architecture of LECDPR is as shown in Figure 2.

4.1 Semantic Space Embedding Generation

If two concepts have a prerequisite relation, they are likely to appear in the same document. Therefore, we first attempt to extract useful concept pair features from the document content.

For a concept, we focus on analyzing related documents of this concept. A document may contain one or more primary knowledge concepts. We consider all primary knowledge concepts in the related documents as related concepts of the current concept. Next, we generate effective text content for the current concept through a two-stage prompt learning with LLMs [Yang et al., 2023], thereby creating the semantic space embedding of the concept.

In the first stage, our goal is to use LLM prompt learning to summarize the raw description content of the current concept and all its related concepts from the corresponding Wikipedia article [Talukdar and Cohen, 2012]. These descriptions may include a definition of the concept and other important content. The LLM prompt consists of two parts: system prompt and input prompt [Wei et al., 2023]. The system prompt provides guiding instructions on what content the LLM needs to generate, and the input prompt is the raw data to be input into the LLM. First-stage prompt for concept description denera-

tion is as shown in Figure 3 below:

Both parts of the prompts should be input into the LLM simultaneously. If a concept does not have a corresponding

Wikipedia article, we will select an article with content clos-

est to the concept as a substitute.

In the second stage, our goal is to obtain the context ev-

idence of the current concept. Similarly, the LLM prompt

includes two parts: system prompt and input prompt. Second-

stage prompt for contextual evidence extraction is as shown

in Figure 4 below:

System prompts:

As a teacher, combine raw description and document content to judge the predetermined relationship between concepts.

(1) Related concepts [yyy] Learn first, the reason is...

(2) You can learn it one after another, because...

(3) Uncertain

.......

Input prompts:

Raw description of [concept 1]: xxx;

<Document 1 >:

Content + Knowledge Concept + Prerequisite / Successive Document;

......

Figure 4: Second-stage prompt for context evidence extraction.

After two stage prompts, the LLM will further generate prerequisite relation text related to the current concept, labeled as textxxx .

Then we encode this content using a word vector encoder. Specifically, we obtain the word vectors for all words in the text and perform average pooling to generate the semantic space embedding of the current concept csem i ∈ R300 :

Using the same method, we can obtain the set of semantic space embeddings for all concepts, denoted as Csem ∈ RN×300 , N is the concept set size.

4.2 Concept Embedding Alignment

Relational Space Embedding Generation

Based on the above semantic space embeddings, we further use graph neural network methods to generate the relational space embeddings for each concept, denoted as Ctrd .

If two concepts appear in the same document, we define these two concepts as related concepts. Based on whether there is a relation between concepts, we construct an adjacency matrix A ∈ {0, 1}N×N ; if other concepts exist in the prerequisite or successor documents of a concept's document, we consider the two concepts having an order, and based on the sequence order between concepts, we construct an order matrix O ∈ {0, 1}N×N . To capture higher-order neighbor information, we compute higher powers of the adjacency matrix and perform weighted aggregation [Tang et al., 2020].

where I (Ak > 0) is an indicator function (value is 1 if the condition is true, 0 otherwise); w is the higher-order adjacency weight, k is the order, Amulti ∈ RN×N .

Then perform normalization:

where D is the degree matrix of the adjacency matrix A.

We initialize the concept features as semantic space embeddings Csem, encode the concept node indices in the adjacency matrix A with order i as positional information, and add positional encoding on top of the semantic space embeddings:

where X(0) ∈ RN×300 , positional encoding PEi,2j = sin(i/100002j/d ), PEi,2j+1 = cos(i/100002j/d ).

Due to the diverse interactive relations that exist between concepts and documents, we then model the complex interactions between concepts and documents. Then we perform multi-layer graph convolution propagation [Kipf, 2016; Wei et al., 2020], where each layer aggregates features through three mechanisms: structural propagation, attention propagation, and sequence-aware propagation. (i). If a document contains multiple concepts, the structural propagation mechanism [Wu et al., 2018] is used to propagate features on the graph through a normalized adjacency matrix, allowing each concept to gather information from its neighbors. (ii). If the same concept appears in different documents, the attention propagation mechanism [Zhang et al., 2022a] dynamically adjusts node importance via self-attention, learning which neighbors are more important for the current concept. (iii). If two conceptually prerequisite concepts occur respectively in two documents that bear a dependency relation, the sequence-aware propagation mechanism uses a sequence matrix to capture directional dependencies. The three aggregated features at the lth layer are:

where αl is a tunable decay coefficient.

Then the features at the l + 1th layer can be updated as:

We define the concept features after L layer propagation as relational space embeddings Ctrd ∈ RN×300 .

Semantic Space Alignment

Now we face a problem: although embeddings in the semantic space and the relational space both represent the same concept, they are generated in completely different ways. The semantic space embeddings come from text encoding, while the relational space embeddings come from graph convolution. Inspired by [He et al., 2022], we align the two space embeddings through random masking and contrastive learning. We train a mapping network to "translate" embeddings from the relational space into a form understandable by the semantic space. Specifically, the mapped embeddings should be able to reconstruct the original semantic embeddings. During this process, a masking operation is introduced to deliberately hide part of the information, forcing the model to learn more fundamental correspondences. This reliable mapping between the two space embeddings is established, laying the foundation for subsequent fusion.

First, we map the relational space embeddings to the semantic space using a MLP:

Then, the mapped embeddings are randomly masked proportionally by ρ, and the resulting embedding vectors are only used within the alignment module to compute contrastive learning loss, not as the final output for downstream tasks.

Then the contrastive learning optimizes spatial alignment, with the loss defined as:

where sim(·) represents the cosine similarity function, and τ is the temperature hyperparameter.

Adaptive Dual-Space Fusion

After alignment, we obtain two types of concept embeddings: one from the semantic space and one from the relational space. Simply averaging these two embeddings to fuse them presents a problem, because not all concepts require both types of information equally.

To fully utilize the complementary information from both space embeddings, we designed an adaptive fusion module. Adaptive fusion allows the model to decide how to reasonably allocate these two types of information for each concept. Using a gating network [Dhingra et al., 2016], the model observes the two embeddings of a concept and then computes a fusion weight. This design ensures that the final concept representation contains both semantic understanding and relational information, combined in the most suitable way for each specific concept.

For the two types of concept embeddings, the fusion weight is computed via a gating network:

where ⊕ indicates concatenation operation.

After nonlinear enhancement of the relational space embedding, the fused embedding is:

Using a fusion regularization loss to ensure the model does not go to extremes or completely ignore one space's information. The regularization term formula for L2 expresses the fusion loss as:

where wtarget represents the target weight.

4.3 Concept Prerequisite Relation Prediction

For two concepts cfused i and cfused j , we construct multidimensional features: difference features xdiff = cfused i -cfused j and product features xhadamard = cfused i ⊙cfused j . These features are concatenated and then compute a logit via a multilayer perceptron.

where [ ; ] indicates concatenation of vectors, and ⊙ indicates the Hadamard product.

The final predicted probability of the prerequisite relation between the two concepts is:

where σ is the Sigmoid function. Finally, we use binary crossentropy loss to compute the loss, with the concept prerequisite relation prediction loss being:

where yij is the true label, which can only be 0 or 1.

The total loss of our model is:

where λrecon is the contrastive loss strength coefficient.

5 Experiments

5.1 Experimental Setup

Datasets: To evaluate the effectiveness of our proposed model, we select three public benchmark datasets. The detailed dataset information is shown in Table 1.

University Course Dataset (UCD) [Liang et al., 2017]: This dataset compiles course information from 377 universities in the United States, covering 376 courses in the field of computer science.

LectureBank [Li et al., 2019]: This dataset originates from an online education platform. It includes a variety of course materials, such as lecture notes and video subtitles.

MOOC [Liang et al., 2017]: This dataset is derived from video playlists in a large-scale academic-video corpus. It aggregates video subtitle texts from 38 carefully curated playlists contributed by 386 computer-science departments worldwide.

Baseline Methods. We compared our proposed model with eight deep learning-based concept prerequisite relation prediction methods: GAE [Kipf and Welling, 2016], VGAE [Kipf and Welling, 2016], PREREQ [Roy et al., 2019], ConLearn [Sun et al., 2022], MHAVGAE [Zhang et al., 2022b], HGAPNet [Mazumder et al., 2023], LCPRE [Sun et al., 2024], and GKROM [Zhang et al., 2025b]. Additionally, two LLM-based [Dong et al., 2025] methods for prerequisite relation prediction are also considered: Qwen2-7B, GPT-4.0 mini.

Evaluation Metrics. We use ACC , F1 score , and AUC to evaluate the performance of the model in predicting concept prerequisite relations. ACC refers to the proportion of correctly predicted samples out of the total number of samples. The F1 score is the harmonic mean of precision and recall. AUC refers to the area under the receiver operating characteristic curve.

Implementation Details. We follow GKROM's setup, dividing the dataset into training, validation, and test sets into 8:1:1 ratio. To ensure fair and accurate evaluation, all baseline models are reproduced and compared under the same data split. We use the Adam optimizer to optimize our model, with 50 training epochs. The random seed and weight de-

cay coefficient are set to 42 and 1E-4, respectively. Additionally, to enrich the text descriptions of concepts, we use the LLM Qwen2-7B-Instruct, employing a two-stage prompt learning process to generate effective text content for current concepts, then use the word vector encoder GloVe to generate 300-dimensional semantic embeddings of concepts. All experiments are implemented using the PyTorch framework, and the experimental environment is a Linux server equipped with an RTX 4090 GPU.

5.2 Overall Performance

In Table 2, we present the performance comparison results of all baseline methods on the three datasets. Our model usually outperformed all other baseline methods on three datasets. In summary, we draw the following conclusions: (1). Our model performs better than methods that only use graph structures (GAE, VGAE, MHAVGAE, HGAPNet, etc.), demonstrating the effectiveness of extracting the highquality concept descriptions and context evidence from raw documents through "two-stage prompting". Rather than asking the LLM to produce a concept definition in one shot, the two-stage prompting progressively distills document context into pedagogically unambiguous and verifiable semantic snippets. (2). LECDPR also outperforms methods that only use text (LCPRE, Qwen2-7B, etc.), this is because we explicitly model the rich concept-document interactions: multiple concepts can appear in one document, one concept can span multiple documents, and concept pairs with prerequisite relations tend to reside in documents that themselves exhibit dependency links, which validates the high-order structural modeling capability of graph neural networks. (3). For the fine-grained educational mining task of concept prerequisite relations, integrating the deep semantic generation ability of large language models, the high-order structural modeling capability of graph neural networks, and the cross-space coordination ability of contrastive alignment mechanisms in a closed-loop manner can truly approach the performance limit.

5.3 Impact of LLMScale on Prerequisite Relation Prediction

To verify the cross-model generalizability of our approach, we replaced the text-generation LLM Qwen2-7B with (i) other models from the same family but with different parameter counts and (ii) models of comparable size yet belonging to entirely different architectures. Figure 5 reports the results on LectureBank for both settings. We observe that once the Qwen2 family exceeds 7B parameters the marginal gain vanishes, indicating that the proposed prompting strategy already exhausts the available semantic evidence at a medium scale. Conversely, experiments with Qwen2-1.5B show that LECDPR retains strong performance even when only a small LLM is available, demonstrating the effectiveness of our twostage prompting across varying model scales. Furthermore, when we substitute the backbone with a same-scale GPTseries model, the outcomes remain stable and competitive, confirming that the two-stage prompt template is robust and transferable across entirely different LLM lineages. Experiments conclusively demonstrate that the two-stage prompting strategy is insensitive to the LLM family, model size, or tokenizer; as long as our two-stage prompting pipeline is followed, high-quality concept descriptions are consistently produced. Consequently, LECDPR maintains robust and stateof-the-art performance across the entire spectrum, achieving genuine cross-model generalization.

5.4 Ablation Experiment

To quantify the individual contributions of each module in LECDPR, we conducted four ablation tests under unified data splits and hyperparameter settings. Table 3 shows the results on UCD. (1) Removing mask alignment (MA). When the contrastive alignment between semantic and relational spaces is disabled, the model degrades to a simple late fusion baseline. This confirms the key role of "forcing the graph encoder to reconstruct semantics generated by the LLM". (2) Removing large language model generation (LLM). Eliminating the two-stage prompt of the LLM causes the semantic space to lose definition and sequence cues, verifying that evidence refined by the LLM is indispensable for fine-grained prerequisite relation discrimination. (3) Removing adaptive gated fusion (AG). Without adaptive gating, the model cannot assign appropriate semantic and structural proportions to different concepts. (4) Removing graph embedding and fusion (GE). If only the LLM semantic embedding is retained, the model reduces to a pure text-matching classifier, with performance dropping, indicating that graph structure and semantic information are complementary.

5.5 Hyperparameter Analysis

As shown in Figure 6, we conducted four sets of key hyperparameters for LECDPR on the MOOC dataset. (1) When wtarget increases from 0.5 to 0.9, performance rises and then drops, peaking at 0.88. A small target weight overemphasizes relational space and suppresses semantic cues, while an overly large weight weakens structural constraints. Thus, wtarget = 0.88 best balances semantic sufficiency and structural consistency. (2) Raising mask ratio from 0.15 to 0.25 improves performance, peaking at 0.22. Too few masks push the mapping network toward an identity function and weaken correspondence learning; too many make reconstruction overly difficult, increase gradient noise, and destabilize convergence. (3) As the contrastive loss weight λrecon increases from 0.5 to 1.0, metrics improve; further increasing to 1.2 causes a decline. Properly increasing contrast strength can more tightly inject LLM knowledge into the graph encoder, but too strong a contrast suppresses the main classification loss, causing gradient conflicts. (4) The learning rate lr varies within 1e-5 to 7e-5, with ACC and F1 forming an inverted U shape curve, peaking at 2.8e-5. When less than 1e5, convergence is slow with high ten-fold average variance; above 5e-5, the loss oscillates.

6 Conclusion

We target prerequisite relation identification and propose LECDPR, an LLM Enhancement and Concept-Document Interactive Modeling for Prerequisite Relation Prediction model. It uses two-stage prompting to extract high-quality concept text from . A complementary representation mechanism of semantic space and relational space is constructed, utilizing random masking and contrastive learning to achieve fine-grained cross-space alignment; simultaneously, an adaptive gating fusion is designed to dynamically allocate semantic and structural weights to different concepts, balancing semantic sufficiency and structural consistency, thereby enabling more accurate prediction of concept prerequisite relations.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (Grant No.62377009, 62407013), the Natural Science Foundation of Hubei Province of China (No.2026AFB746), the Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning (No.2025AISL001), and the Key Laboratory of Intelligent Sensing System and Security (No.KLISSS202504).