Reconciling Cognitive Modeling with Knowledge Forgetting: A Continuous Time-aware Neural Network Approach

1 Introduction

Recent years have witnessed the rapid development of intelligent tutoring systems, which has accumulated a large amount of exercising records of students, and thus, enables a new data-driven paradigm of computer-aided education [Abido˘glu et al., 2017]. Along this line, as an emerging technology, cognitive modeling has attracted increasing attention, which aims to discover students' knowledge proficiency or learning ability. Indeed, the results of cognitive modeling can benefit a wide range of intelligent educational applications, such as performance prediction [Chen et al., 2005] and personalized course recommendation [Lan and Baraniuk, 2016].

∗Corresponding author

In the past decade, while considerable efforts have been made in the field of cognitive modeling, a long-standing research challenge is how to naturally integrate the forgetting mechanism into the learning process of knowledge concepts [Nagatani et al., 2019; Ghosh et al., 2020]. To be specific, according to the theories of educational psychology [Atkinson and Shiffrin, 1968; Murre and Dros, 2015], human memories (e.g., instant, long-term, or short-term memories) usually have different forgetting patterns, which are influenced by various confounding factors, such as the current knowledge state and corresponding learning process of students. In recent literatures, some research efforts have been introduced for exploring the forgetting mechanism in cognitive modeling [Ghosh et al., 2020; Nagatani et al., 2019]. However, the proposed methods either depend on the manually-designed forgetting features or oversimplified process assumptions (e.g., fixed and discrete learning intervals), which significantly limit the flexibility and performance of downstream applications. Therefore, there still lacks a realistic cognitive modeling approach to strike a balance between the learning and forgetting processes of knowledge concepts.

To this end, in this paper, we innovatively propose a novel Continuous Time based Neural Cognitive Modeling (CTNCM) approach to naturally integrate the dynamic and continuous forgetting behaviors into the modeling of students' learning process. To be specific, we first adapt neural Hawkes process with a specially-designed learning event encoding method to model the relationship between knowledge learning and forgetting, which can distinguish the impact of both positive and negative responses of students on their knowledge state. Then, we propose a learning function with extendable settings to jointly model the change of different knowledge states and their interactions with the exercises at each moment. Furthermore, CT-NCM can simultaneously predict the future knowledge state and exercise performance of students. In particular, all the processes in CT-NCM are modeled with continuous time awareness, which can better fit the educational psychology theories and real-world applications scenarios. Finally, we conduct extensive experiments on five real-world datasets with four state-of-art approaches and two more variants of CT-NCM. The experimental results clearly validate the effectiveness of CT-NCM and show its interpretability in terms of knowledge learning visualization.

2 RelatedWork

Considering the dynamics of the knowledge concept learning process, many efforts have been undertaken to track the change of knowledge proficiency or learning ability of each student in cognitive modeling. These existing methods can be classified into the following two categories: (1) Traditional models which represented by Bayesian Knowledge Tracing (BKT) [Corbett and Anderson, 1994] and Factorization models [Vie and Kashima, 2019]; (2) Neural network based sequential models such as Deep Knowledge Tracing (DKT) [Piech et al., 2015]. DKT is the first method of using Recurrent Neural Networks (RNNs) to fit student's knowledge state and infer current exercise performance based on historical learning records.

In recent years, some methods considering the forgetting mechanism have been developed in both categories. These proposed methods mainly depend on the manually-designed forgetting features, which largely limits the performance of learning and prediction. For example, Knowledge Proficiency Tracing [Huang et al., 2020] utilizes a global exponential decay to represent the forgetting of knowledge state over time; DKT+Forgetting [Nagatani et al., 2019] designs forgetting factors via counting past records, which is the same as some factorization models [Vie and Kashima, 2019; Lindsey et al., 2014] do. Even though few works were proposed to design forgetting and learning via purely neural networks learning, the forgetting mechanismmodeling in these works is still based on oversimplified process assumptions (e.g., fixed and discrete learning intervals). For example, Learning Processconsistent Knowledge Tracing (LPKT) [Shen et al., 2021] mainly focuses on learning process modeling and just discontinuously learn the decline of student's knowledge state by designing a forgetting gate, the state is updated discontinuously by a simple weighted combination of learning and forgetting factors; context-aware attentive knowledge tracing [Ghosh et al., 2020] uses a monotonic attention mechanism with a global exponential decay and discrete learning interval assumption to simply simulate forgetting process of different knowledge concepts knowledge.

After that, inspired by the classical Hawkes process [Hawkes, 1971], a recent approach HawkesKT [Wang et al., 2021] has been proposed, which attempts to discover temporal cross-effects between different concepts with the help of collaborative filtering and matrix factorization. However, how the past learning behaviors affect the future learning gain and forgetting rate is modeled by linear accumulation in HawkesKT, which simplifies the cumulative effect of past learning history and contradicts the long-term memory theory in psychology. To this end, in order to design a realistic approach that can strike a balance between learning and forgetting processes in cognitive learning, here we propose a novel cognitive modeling approach CT-NCM, which can better fit the actual process of memory creation and forgetting in psychology.

3 Preliminaries

In this section, we first formally define the cognitive modeling problem. Then, we give our assumptions of the learning process based on related theory in psychology and a brief introduction of neural Hawkes process.

3.1 Problem Statement

Let S = {s1, s2, . . . , sL} be the set of L students, Q = {q1, q2, . . . , qM} be the set of M exercises, K = {k1, k2, . . . , kN} be the set of N knowledge concepts. The relationship between exercises and knowledge concepts is denoted by Qmatrix Q ∈ RM×N , where Qj is an N-dimensional binary vector that indicates which knowledge concepts are required for exercise qj.

Generally, when an exercise is assigned to the student, she will give an answer and get a response, which can be seen as a learning behavior of the corresponding knowledge concepts. We denote the learning sequence of a student with n learning behaviors as Rn = {(q1, k1, t1, r1), (q2, k2, t2, r2), . . . , (qn, kn, tn, rn)}, where the tuple (qi, ki, ti, ri) is the i-th learning behavior; qi ∈ Q is the question; ki ∈ K is the knowledge concept associated with question qi, which is obtained from Q; ti is the timestamp for the answer; and ri ∈ {0, 1} is the response (0 represents fault and 1 represents true). Our problem definition can be expressed as follows:

ProblemDefinition. Given students's historical learning sequence Rn = {(q1, k1, t1, r1), (q2, k2, t2, r2), . . . , (qn, kn, tn, rn)}, our goal is twofold: (1) diagnosing the knowledge state of each student from time t1 to tn through tracking the change of knowledge state over time; and (2) predicting the student's knowledge state and performance score on specific exercise at time tn+1.

3.2 Learning Process Assumption

The multi-store model of memory [Atkinson and Shiffrin, 1968; Murre and Dros, 2015] in the cognitive psychology assumes that different human memories (i.e., instant, short-term and long-term memories) have different forgetting patterns. For example, short-term memory decays relatively quickly after a learning behavior, while long-term memory is more stable but also changes over time. Meanwhile, the study of encoding and retrieval memory [Myers, 2004] proposes that the degree of forgetting is not only related to time, but also to what student already know. Based on these theories, we give the following assumptions about the updating mechanism of knowledge state during the learning process of students.

Assumption. (1) The changing pattern of knowledge state is controlled by the changing mechanism of knowledge memory during the learning process. (2) The knowledge memory changes discontinuously with each successive learning behavior occurrence, and also forgets continuously toward a stable value as time elapses between learning behaviors. (3) Current learning gain and forgetting rate are determined by the complex cumulative effect of past learning history, e.g., the sequential ordering of past learning behaviors.

This paper attempts to naturally model such updating mechanism of knowledge state in the learning process by adaptively applying the neural Hawkes process.

3.3 Neural Hawkes Process

Giving historical sequential events Gn = {(e1, t1), (e2, t2), . . . , (en, tn)}, where each ei ∈ {1, 2, . . . ,E} is an event type and 0 < t1 < t2 < . . . are timestamps of occurrence. Hawkes process [Hawkes, 1971] describes the intensity function of event e over continuous time t as shown in Eq. (1). From Eq. (1), we can see that the intensity of event e in time t is the addition of the following two parts: (1) the basic intensity µe ≥ 0 that the event births with; and (2) the sum of influence from events occurring before time t, where αj,e ≥ 0 and δj,e ≥ 0 indicate the degree of event j initially exciting event e and the decay rate of this excitation respectively.

Neural Hawkes process [Mei and Eisner, 2017] generalized Hawkes process by leveraging an improved LSTM [Hochreiter and Schmidhuber, 1997] approach to capture the realworld complex effect of the past on the future. Specifically, the event intensities are determined from the hidden state g(t) of a continuous-time LSTM according to Eq. (2). The hidden states g(t) can be continually obtained from the memory vector c(t) via Eq. (3), where oi is the output gate after inputting the i-th event. The memory vector c(t) is updated discontinuously with each successive event occurrence and then decay continuously from current value ci toward a stable value ¯ci until the next event occurs in change rate δi according to Eq. (4), where ci and ¯ci are produced by two interrelated LSTM units (named LSTMcell1 and LSTMcell2) and updated immediately once an event occurs.

4 The CT-NCMMethod

In this section, we present how the CT-NCM model diagnoses student's knowledge state dynamically and predicts her future performance. As shown in Fig. 1, the CT-NCM consists of two main parts: (1) the learning and forgetting module, and (2) the performance prediction module. These two modules work periodically over time. Specifically, after a student has finished an exercise qi associated with knowledge concept ki in time ti, the learning and forgetting module will make her knowledge state a change instantly and a continuous decay on the interval (ti, ti+1] until question qi+1 is assigned to her at ti+1. After the knowledge learning gains in time ti and forgetting on the interval (ti, ti+1], the knowledge state is updated to achieve the latest value. Then, in the performance prediction module, student performance on exercise qi+1 is predicted according to her latest knowledge state.

For facilitating description, the discussion below only takes place in a time interval (ti, ti+1], where the left open means that the learning behavior (the student answers exercise qi) has occurred, and right closed means that the next learning behavior has not occurred (i.e., the exercise qi+1 is assigned to the student in time ti+1, but the student has not answered it). Some main notations used in this work have been shown in Table 1.

4.1 Learning and ForgettingModule

We model the student's knowledge learning and forgetting over time by adaptively utilizing neural Hawkes process on the student learning sequence Rn = {(q1, k1, t1, r1), (q2, k2, t2, r2), . . . , (qn, kn, tn, rn)}. More specifically, we use the update mechanism of the hidden state g(t), which is the output of continuous-time LSTM in neural Hawkes process adaptively, to track the change of student knowledge state hs(t) because of discontinuous knowledge learning and continuous forgetting. In the following, how the knowledge state updates instantly because of learning behavior (qi, ki, ti, ri) and decays continuously in the interval (ti, ti+1] due to knowledge forgetting will be introduced in detail respectively.

Learning. We firstly formally define learning event embedding xi based on the learning behavior tuple (qi, ki, ti, ri), which is the i-th input of continuous-time LSTM. We consider that two facts existing: (1) the essence of learning behavior (qi, ki, ti, ri) is the learning of the knowledge concept ki; and (2) the different correctness of student's answers affects her knowledge state in different degrees.

Under such consideration, we generate xi by combining the knowledge concept ki and the response ri as:

where xi ∈ Rm, ai = {aij | j = 1, ..., 2N}, A ∈ R2N×m, m is the embedding size. A is an embedding matrix that needs training.

Entering the event embedding xi into continuous-time LSTM, the student knowledge state is updated instantly from the value hs(ti) to the latest value his = oi · (2σ(2ci) - 1), where ci is the latest memory vector by updating the previous memory vector c(ti) according to Eq. (7). Here, the c(ti) and hs(ti) are the latest memory vector and student knowledge state vector before the learning event xi occurring.

Forgetting. After the update owing to the learning event xi , the knowledge state will face a continuous decay until the next learning event xi+1 occurs, because of the forgetting over time. Specifically, the memory vector c(t) decays from ci towards a steady-state value ¯ci with rate δi, where both ¯ci and δi are determined when the learning event xi occurs. Here, ¯ci is calculated according to Eq. (8); and δi is calculated as a positive vector by Eq. (9), which can guarantee the knowledge state decay in interval (ti, ti+1] because of forgetting. Correspondingly, the knowledge state hs(t) changes over time from his to hs(ti+1) by getting its value from c(t) according to Eq. (3).

where W ∈ Rd×(m+d) , b ∈ Rd×1 , fs(x) = log(1 + ex ) is a softplus function that can guarantee the positive characterization of the decay vector.

4.2 Performance PredictionModule

With the latest knowledge state hs(ti+1), we will predict student's performance on the exercise qi+1 which is assigned to the student at ti+1. Let qi+1 represent the one hot vector of qi+1, the probability ˆ yi+1 that indicates the likelihood her answer qi+1 correctly can be written formally as:

where F(·) is the interaction function between student factors and exercise factors. Inspired by a classical educational psychology theory, the Item Response Theory (IRT) [Lord, 1980], we design F (hs, q) with extendable settings carefully, as shown in Eq. (11). The interpretable parameters hpro s represents student factor (e.g., ability level) that can be obtained from hs; hdiff q and hdisc q represent the difficulty and discrimination levels of exercise q respectively.

Here, we give a special student-exercise interaction function denoted f(hs, q) under such extended formulation in Eq. (11). To be specific, the student factor hpro s is set to hs and the two exercise factors are set as Eq. (12) and (13). F3 is set as a three-layer fully connected neural network to fit the complex interactions, and F2 is a sigmoid function for normalizing the prediction value into range of (0, 1).

where M is the number of exercises, d is the hidden size, σ is the sigmoid function, B and C are two embedding matrixes that need training.

Finally, in order to illustrate the extendibility of Eq. (11) for different variants, we use two representative IRT-based interaction functions: the item response term (IRT) [Lord, 1980] and neural cognitive model (NCD) [Wang et al., 2020] to set F(·), named CT-NCM IRT and CT-NCM NCD, as follows:

IRT. Taking its classical formulation y = σ((hpro s - hdif f q ) × hdisc q ) as an example, where hpro s , hdiff q and hdisc q are unidimensional student proficiency, exercise difficulty and discrimination respectively. Given hs ∈ Rd×1 and q ∈ R1×M , hpro s is obtained through hpro s = σ(W×hs), hdisc q is obtained by Eq. (13), hdiff q is obtained via hdiff q = σ(q×D), where D ∈ RM×1 .

NCD. It models student-exercise interactions more generally as: y = σ(f ′ (Qq ◦ (hpro s - hdiff q ) × hdisc q )), where f ′ denotes multi-layer fully connected layer, hpro s and hdiff q are mapped into the knowledge space with the help of exerciseknowledge relationship vector Qq through the element-wise product ◦. Actually, NCD is an implementation of the extendable formulation in Eq. (11) via the settings: F3 in Eq. (11) is set by f ′ ; F2 is set as a sigmoid function; hpro s is obtained from hpro s = σ(W× hs); hdisc q is set as same as Eq. (13); and hdiff q is set according to hdiff q = σ(q × E). Where W ∈ RN×d , E ∈ RM×N are two matrices need training.

4.3 Optimization

All parameters in CT-NCM are learned by joint training the learning sequences of students. A binary cross entropy loss between the performance prediction ˆ ys t for student s at time t and its corresponding ground truth ys t is used to optimize these parameters, as follows:

5 Experiment

5.1 Experimental Settings

Dataset Description

Five public available datasets are used in our experiments, i.e., ASSISTment12 1 , ASSISTment17 2 , Slepemapy.cz 3 , Junyi 4 , and EdNet-KT1 5 . ASSISTment12 [Feng et al., 2009] is collected from ASSISTments online tutoring service system and contains student exercising data for the school year 2012-2013. ASSISTment17 [Patikorn et al., 2018] is published in ASSISTments Longitudinal Data Mining Competition in 2017. Slepemapy.cz [Papouˇsek et al., 2016] comes from the online adaptive system slepemapy.cz, which is for practicing geography facts. Junyi [Chang et al., 2015] is gathered from Junyi Academy, which is an E-learning platform built in 2012. And EdNet-KT1 [Choi et al., 2020] is collected by Santa, an artificial intelligence tutoring system aiding students in preparing for the TOEIC.

For calculate efficiency, we set the max sequence length to 100 and truncate student learning sequences longer than 100 to several sub-sequences following to [Shen et al., 2021]. Afterward, sequences with lengths less than 5 are removed to ensure that each sequence has enough data for cognitive modeling. The statistics and time span distribution of sequences for five datasets are shown in Table 3 and Fig. 2.

Comparison Approaches and Metrics

As for comparison approaches, in addition to four stateof-art approaches including DKT [Piech et al., 2015], DKT+Forgetting [Nagatani et al., 2019], AKT [Ghosh et al., 2020] and HawkesKT [Wang et al., 2021], we also design a variant of DKT named DKT-V which has the same performance prediction module as the CT-NCM so as to more fairly validate the superiority of the learning and forgetting mechanism in CT-NCM. Moreover, we also provide the results of another two versions of CT-NCM: CT-NCM IRT and CT-NCM NCD, whose performance prediction modules are implemented with IRT [Lord, 1980] and NCD [Wang et al., 2020] respectively. As for metrics, we use accuracy (ACC) and the area under the receiver operating characteristics curve (AUC) to evaluate the performance of all methods on predicting binary-valued future student responses to exercises.

1 https://sites.google.com/site/assistmentsdata/datasets/2012-13school-data-with-affect

2 https://sites.google.com/view/assistmentsdatamining/dataset

3 https://www.fi.muni.cz/adaptivelearning/?a=data

4 https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=1198

5 https://github.com/riiid/ednet

Experimental Settings

We performed 5-fold cross-validation in the experiment 6 . For each fold, 80% of the learning sequences are split as the training set (70%) and the validation set (10%), while the rest 20% are used as the test set. We implement DKT, DKT+Forgetting, AKT and HawkesKT according to their original papers. To be specific, if the parameters are insensitive to different datasets in the original paper, they are set as same as the original paper (e.g., the all parameters of HawkesKT, the parameters apart from learning rate in AKT); otherwise, the parameter tuning was conducted on the validation set according to the value range provided in the original papers (e.g., the parameter of learning rate in AKT, the parameters of DKT and DKT+Forgetting). For CT-NCM and its two versions, we conducted the hyperparameter tuning based on the validation set. To be specific, we used {16, 32, 64, 128}, {16, 32, 64, 128} and {0, 0.1, 0.2, 0.25} as values of the embedding size, hidden size, and dropout rate with Adam optimizer. For DKT-V, the same part with DKT is set according to DKT, whereas the same part with CT-NCM is set according to CT-NCM.

6 https://github.com/BIMK/Intelligent-

Education/tree/main/CTNCM

For each sequence, all interactions except for the first one were used for training, verifying and testing. All experiments were implemented with PyTorch by Python and conducted with Tesla P100 and GeForce RTX3090 GPU.

5.2 Experimental Results

Overall Performance

Table 2 lists the performance of our CT-NCM and all comparison approaches on predicting student future performance across five datasets. Here we report the average values over 5fold cross-validation. The optimal results are shown in bold, and the sub-optimal results are marked in italics and underlined. Additionally, Tables 4 and 5 in Appendix provide the win/tie/loss results for the proposed CT-NCM against the compared algorithms in terms of AUC and ACC with the twosided T-test (p < 0.01), respectively. From the three tables, we can make the following two observations. (1) The CTNCM performs significantly better than all baseline on four datasets and comparably with the best baseline (i.e., AKT) on one dataset (i.e., EdNet-KT1), which validates the effectiveness of the CT-NCM. (2) All the three versions of our CTNCM achieve remarkable performance on all datasets, which can indicate that the relationship modeling between knowledge learning and forgetting has excellent performance on tracing the change of knowledge state.

Robustness Analysis

The length of learning sequences is an important hyperparameter in the modeling of the dynamic learning process. We report the changes in the performance of our model and all baselines when the length of learning sequence changes based on ASSISTment17 dataset, which is shown in Fig. 4. Here, we set the length of learning sequence to 50, 100, 200 and 500 respectively. Note that we chose ASSISTment17 to conduct this experiment since it has the largest average sequence length (about 500). From this figure, we can see that CTNCM exhibits higher stability as the learning sequence length changes in terms of AUC and ACC.

Ablation Study

Furthermore, to validate the effectiveness of forgetting modeling in CT-NCM, we design one variant of CT-NCM, which does not consider the Forgetting process by setting every time interval of sequences with 0, named "CT-NCM no F". Fig. 5 shows the experimental results of the proposed CT-NCM and the variant "CT-NCM no F" on all datasets in terms of AUC and ACC. For ease of view, we use several easily identified abbreviations for original dataset names. The comparison results strongly confirm the effectiveness of forgetting modeling.

Learning Process Visualization

In order to validate the interpretability of our proposed method in modeling student's learning process, we visualized the knowledge mastery degree changes of a student from the test set in ASSISTment12 dataset, as shown in Fig. 3. Besides, the data for visualization was obtained by CT-NCM NCD. Generally, student knowledge state's changing degree over each step is brought by both learning and forgetting, which makes it difficult to analyze the individual influence of these two on the state. Therefore, we roughly made the following assumption: The knowledge state changing is mainly owing to the knowledge learning when the time interval is particularly short (e.g., the light color boxes marked in dotted green boxes on the time row); otherwise, it is mainly due to the knowledge forgetting when the time interval is particularly long (e.g. the deep color boxes marked in the solid green boxes on the time row).

From the figure, we have the following observations. Firstly, the knowledge mastery degree of all the observed concepts improves significantly on steps with particularly short time intervals (from steps 0 to 2 and from steps 10 to 14) owing to the learning effect. Secondly, the knowledge mastery degree of all the observed concepts decay on steps with particularly long time intervals (e.g., from step 4 to 5, step 9 to 10, step 14 to 15 and step 20 to 21) due to forgetting effect. Intuitively, the above analysis clearly validates the effectiveness of our method on modeling the learning and forgetting process of students.

6 Conclusion

In this paper, we proposed a novel cognitive modeling approach with continuous time-aware neural networks, namely CT-NCM. A unique perspective of CT-NCM is to strike a balance between the learning and forgetting processes of knowledge concepts in a realistic manner. Specifically, we adapted the neural Hawkes process with a specially-designed learning event encoding method to model the knowledge learning and forgetting processes with continuous time awareness. Then, an extendable performance prediction function was designed to jointly model the change of different knowledge states and their interactions with the exercises at each moment. In this way, CT-NCM can simultaneously predict the future knowledge state and exercise performance of students. Experimental results on five real-world datasets clearly show the performance gain and good interpretability of our proposed model.

A Appendices

The win/tie/loss results for the CT-NCM against the compared algorithms based on two-sided T-test (p< 0.01) are supplemented in Table 4 and Table 5. Note that the approach "DKT+F" is short for "DKT+Forgetting", "HKT" is short for "HawkesKT".

Acknowledgements

This research was partially supported by grants from the National Key Research and Development Project (NO. 2018AAA0100105), the National Natural Science Foundation of China (NO. 62107001, 61836013, 61976001), the Anhui Provincial Natural Science Foundation (NO. 2108085QF272), and the Key Program of Natural Science Project of Educational Commission of Anhui Province (NO. KJ2020A0036). Haiping Ma gratefully acknowledges the support of the CCF-Tencent Open Fund.