Variational Graph Auto-Encoder Driven Graph Enhancement for Sequential Recommendation

1 Introduction

Recommender systems are extensively utilized in ecommerce, social media, and content streaming to satisfy the users' requirements. On these platforms, understanding the temporal patterns of user interactions can significantly improve the quality of personalized recommendations [Liu et al., 2023a; Ma et al., 2024; Liao et al., 2023]. For the sequential recommendation task, the essential problem is how to utilize sequential patterns of user interactions to provide users with recommendations that satisfy their dynamic preference needs. In the past, a number of attempts [You et al., 2024; Wu et al., 2024] have been made to develop powerful sequential models based on deep learning. Many works [Hidasi et al., 2016; Yue et al., 2024] utilize Recurrent Neural Networks (RNNs) to capture the sequential patterns among user interaction sequences. Additionally, some works [Qi et al., 2022; Ahmed et al., 2023] combine RNNs with attention mechanisms to focus on important information within the interaction sequences. Although above methods have achieved significant results, however, they primarily focus on the sequential patterns of user interactions and fail to capture the more complex item transition patterns in user sequences. Furthermore, user interaction behavior can be influenced by many factors (e.g., benefit incentives, accidental clicks, overrecommendation of popular items, etc.), it can result in user sequences containing some noise. RNN-based methods [Yue et al., 2024; Bach et al., 2020; Ahmed et al., 2023] typically assume user interaction sequences as the primary indicator of user preferences, which may lead to suboptimal recommendation results. Recently, Graph Neural Networks (GNNs) [Scarselli et al., 2008; Nguyen and Tran, 2023; Okamura et al., 2023] have attracted attention for their ability to capture higher-order interactions. In addition, some simplified GNNs (e.g., NGCF [Wang et al., 2019], LightGCN [He et al., 2020], PriGCN [Liu et al., 2023b]) reduce the model complexity by omitting linear transformers and activation functions and are becoming popular in recommender systems. However, these methods still face the challenge of data noise. Directly utilizing all user interaction sequences and performing multi-loop embedding propagation can worsen the noise effect, leading to skewed results.

To address above challenge, recent methods attempt to use self-supervised learning techniques to enhance the noise robustness of recommendation models [Wu et al., 2021; Liao et al., 2023]. These methods create contrastive views for self-supervised learning but rely heavily on the laborious trial-and-error process for selecting augmentation methods. In this paper, we propose a Variational Graph Auto-Encoder driven Graph Enhancement method (VGAE-GE) for robust augmentation in sequential recommendation. We first construct an item transition graph to capture the higher-order interaction information between items. Then we use the inference model of Variational Graph Auto-Encoder (VGAE) to obtain the latent variable distribution of items. The distribution distance between items with interactions should be closer, while those without interactions should be farther apart. Then we use the generative model and latent variables to reconstruct the item transition graph. Finally, we employ a Graph Convolutional Network (GCN) to transform the latent variables back into node embeddings. By combining the reconstructed item embeddings with user embeddings, we utilize a Mamba-based model for sequential recommendation.

• We propose a novel method called VGAE-GE, which can convert item embeddings into potential vectors and capture higher-order information in the potential space. To enhance the noise immunity, we reconstruct the graph structure by using generative model.
• We propose to use GCN to revert the reconstructed graph structure to embedding, and then input the user interaction data into a Mamba-based recommendation model. By combining the basic Mamba blocks with various techniques such as layer normalization and feed-forward networks, the sequence modeling capability is further improved without sacrificing inference efficiency.
• Comprehensive experiments on five public datasets confirm the effectiveness of our VGAE-GE model.

2 RelatedWork

2.1 Graph-based Sequential Recommendation

Prominent works, such as SR-GNN [Wu et al., 2019], GCNGNN [Wang et al., 2020] and ITGCN [Liu et al., 2022], utilize GNNs to model user-item interaction sequences, demonstrating significant improvements in recommendation accuracy. Simplified GNN models, such as NGCF [Wang et al., 2019], LightGCN [He et al., 2020] and UltraGCN [Mao et al., 2021], have also become popular in sequential recommendation by reducing model complexity through the omission of linear transformers and activation functions. In addition, some works have utilized graph-based methods for sequential recommendation based on multiple aspects to overcome multiple challenges [Zhu et al., 2021]. Peintner [Peintner, 2023] attempts to reduce the recommendation bias problem when utilizing graph-based models for sequential recommendation. Focusing on the continuous time modeling problem in sequential recommendation, GDERec [Qin et al., 2024] proposes a new GNN based on ordinary differential equations to simulate implicitly the time evolution. Some works utilize GNNs combined with Graph Masked AutoEncoder to combat data noise and sparsity, such as MAERec [Ye et al., 2023], AutoCF [Xia et al., 2023] and S-CIEE [Wang et al., 2025b]. Despite these advancements, these methods still face challenges related to data noise. Directly using all user interaction sequences and conducting multilayer information propagation can amplify noise effects, leading to biased recommendation results.

2.2 Variational Graph Auto-Encoders

VGAE has proven effective for learning latent representations of graph-structured data, providing a probabilistic approach to encoding graph information. MVGAE [Yi and Chen, 2021] is a model that utilizes VGAE to generate representations of nodes (i.e., mean vectors for semantic information and variance vectors for the noise level of the corresponding modality), which are then applied to recommender systems. However, MVGAE overlooks the temporal relevance of useritem interaction. Similarly, Altaf et al. [Altaf et al., 2019] simply apply VGAE to a dataset recommendation system. SCVG [Ding et al., 2021] is a semi-deterministic and contrastive VGAE for item recommendation. STR-VGAE [Zhu et al., 2023] uses VGAE to learn travel packages that contain users' implicit preferences from their behavioral sequences to make travel packages recommendation. In addition, there are many models such as DR-VAE [Wang et al., 2025a] and MvDGAE [Zheng et al., 2021] that try to utilize VGAE to exploit the latent distribution for enhancing robustness and accuracy, but currently these methods do not consider both the latent distribution and the embedding representation, and ignore the application in sequential recommender systems.

3 Methodology

Let U = {u1, u2, ..., u|U|} and V = {v1, v2, ..., v|V|} denote the set of users and items, respectively. Each user u ∈ U is associated with a sequence of interactions with items over time. We denote historical check-ins for user u as Su = {su1 , su2 , ..., sum}, where m is the length of u's item sequence Su , sut ∈ V represents the item accessed by the user at t-th, and t denotes the corresponding timestamp of interaction with 1 ≤ t ≤ m. Our goal is to predict the item sum+1 that the target user u is most likely to interact with.

3.1 ItemTransition Graph Construction

To obtain item dependencies, we use user behavior sequences to generate an item transition graph G = (V, E) with N = |V| nodes to represent the transition relationships between different items. eij ∈ E indicates that there is an edge between the ith and j th items. A = (aij) ∈ RN×N is the adjacency matrix of G and it indicates the implicit relationships between items. D ∈ RN×N is the degree matrix of A. For the edge set E, we construct it by the following form:

Here, we create an edge between each item and its c-hop neighbor across all user sequences, computing duplicated edges only once. Thus, aij = 1 if (vi, vj) ∈ E and aij = 0 otherwise.

3.2 VGAE-driven Graph Enhancement

To encode interaction patterns among items, we map them into a d-dimensional latent space. Specifically, we create an embedding vector ej ∈ Rd for each item vj . Additionally, we use E(v) ∈ RN×d to represent the embedding of items and map the input graph to low-dimensional representations:

Here, ˜ A is calculated by ˜ A = D-1/2AD-1/2 . Next, multiple embedding propagation layers are used to aggregate the node neighborhood information of the item:

where z(v,l+1) and z(v,l) denote the the embedding representations of item v at the (l +1)-th and l-th layers, respectively. Nv denotes the neighboring item of item v, ∑v′∈Nv z(v′ ,l) denotes the neighbor aggregation information for item v at lth layer. We aggregate the embedding representations of each layer of the item and get the representation.

where ˜ z(v,l) is the final representation of an item v. Subsequently, we utilize ˜ Z to represent the embedding representation of all items.

After obtaining each node representation and the graph structure of the item transformation graph G, we designed a VGAE-based encoder to convert the node representations into a distributed form. The encoder consists of two encoding heads, represented as follows: Z (1) = fReLU ( ˜ Z, A | W (0)); Z (2) µ = fLinear (Z (1) , A | W (1) µ ) ∈ RN×d , Z (2) σ =

fLinear (Z (1) , A | W (1) σ ) ∈ RN×d . Here, W (1) µ is the trainable parameter matrix used to generate each node's mean, W (1) σ is the trainable parameter matrix used to generate each node's variance.

Inference model. The inference model is defined by the distribution q(Z | ˜ Z, A) parameterized by two encoding layers:

where q(Z | ˜ Z,A) denotes joint distribution of latent variables for all nodes, ∏Ni=1 q (zi | ˜ Z,A) denotes the product of the distribution of latent variables for each node zi, q (zi | ˜ Z,A) denotes the distribution of latent variables for node zi, µzi = Z (2) µ [i, :] is the mean vector of a multivariate Gaussian distribution associated within zi, and σ2 zi = Z (2) σ [i, :] is the corresponding variance vector.

Generative model. In the generative model, we use the latent variables Z to generate the distribution of A.

where p(A | Z) denotes the probability of generating a connection between each aij given the latent variable Z, aij are the elements of A.

After completing the design of the inference model and the generative model, consistent with [Kipf and Welling, 2016], we formulate a variational lower bound on the log-likelihood of the input graph as: Lreco = Lpos+Lneg. Here, Lreco denotes the reconstruction loss, which consists of the cross-entropy loss for positive (i.e., Lpos) and negative (i.e., Lneg) samples, respectively. Here, the positive sample reconstruction loss calculates the binary cross-entropy loss between the predicted values of the positive sample edges and the true labels:

where (i, j) denotes the edge between item vi and vj, yij is the true label of edge (i, j), σ(zi · zj) is the probability of the existence of the predicted label, zi and zj are the potential representations of the items vi and vj, respectively. The potential representation zi can be computed using mean and variance: zi = µi + σ ⊙ ϵi, where ϵi ∼ N(0, 1) is the noise matrix generated by the standard normal distribution, ⊙ denotes the Hadamard product. Considering that yij = 1 for positive samples and yij = 0 for negative samples remain constant, the reconstruction loss for positive and negative samples can be simplified as:

In addition, we utilize the Kullback-Leibler Divergence (KLD) to measure the difference between the distribution q(Z | ˜ Z, A) generated in the variational auto-encoder and the prior distribution p(Z), so that the distribution is as similar as possible to the standard Gaussian in the hypothesis. The KLD loss is as follows:

After considering the KLD, the reconstruction loss can be updated as follows:

3.3 Mamba4Rec as Sequence Encoder

After obtaining the reconstructed graph structure, we encode the generated graph using GCN to obtain the item embedding. We still ignore the weight transformations and activation functions in the message passing process. The representation of item is generated as follows:

where Nv denotes the 1-hop neighborhood of v, e(v,l) and e(v,l+1) denote the reconstructed embedding representations of item v at the l-th and (l + 1)-th layers, respectively. ˜ e(v,l) is the final reconstructed representation of a item v. Here, we design a reconstruction rate hyperparameter β. We get the probability of existence of the edge based on the hidden variables, if the probability is greater than or equal to β, then this edge is retained in the reconstructed graph; otherwise, this edge is eliminated from the reconstructed graph.

In the user's interaction sequence, we add positional embedding pi to enhance the temporal information of the sequence:

With the user representation and item representation already obtained, we utilize the user interaction data Eu as input data into the Mamba4Rec model and perform sequence recommendation.

where W (1) and b(1) are the weights and biases of the linear projection, the Conv1d(·) is a 1D convolution operation. Next, the output ˜Eu,m is obtained after State Space Model (SSM) and residual connection:

where ˆEu,m is the is the intermediate value processed by the SSM, rm denotes the residual connection, SiLU(·) is the activation function. The final interaction probabilities are obtained as follows:

Through the above stages, the Mamba4Rec model is able to efficiently process the user's sequential interaction data and generate high quality recommendation results.

3.4 Model Training

We employ the cross-entropy loss function to optimize the recommendation task. All subsequences of Su are used as training data, i.e., {(su1 ), (su1 , su2 ), ..., (su1 , ..., sum-1 )}. The main loss function Lmain for the recommendation task is given as follows:

where ˜Eu,t denotes the embedding of the sequence (su1 , ..., sut ), and vt /∈ Su is the t-th item randomly sampled from the negative samples. In addition, we use regularization loss for preventing model overfitting, obtained by computing the L2 norm of the model parameters:

where θencoder, θdecoder and θrecommender are the parameters of the encoder, decoder and the recommender model, respectively, and λ is the regularization factor. Finally, the total loss L is the weighted sum of the reconstruction loss, the main loss and the regularization loss:

where λ1 and λ2 are hyperparameters that control the strengths of reconstruction loss and main loss, respectively.

4 Experiments

Our experiments focused on validating the performance of VGAT-GE and answering the following key questions:

RQ1: Can our proposed VGAE-GE outperform the baselines for sequential recommendation? RQ2: How does the VGAE affect model performance? RQ3: How to demonstrate the effectiveness of Mamba4Rec Recommender? RQ4: How is the performance of VGAE-GE affected by different parameter settings?

4.1 Experimental Setup

Datasets. We consider five challenging recommendation datasets: Amazon Books (i.e., Books) and Amazon Toys (i.e., Toys) collected from Amazon platform (https://www.amazon.com/ ), Retailrocket (i.e., Retail) collected from an e-commerce website (https://www.kaggle.com/retailrocket/ecommerce-dataset/ ), NYC and TKY [Yang et al., 2013] collected from Foursquare, of which the statistics are shown in Table 1. We consider items with fewer than three interactions in the first three datasets as outliers and remove them, and similarly, check-ins with fewer than five interactions in the last two datasets are considered as outliers and removed.

Evaluation Metrics. We use leave-one-out strategy for evaluation and use two widely adopted ranking-based metrics to evaluate the performance of all methods, namely Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) with top 5/10/20 recommended candidates.

Baselines. We compare VGAE-GE with 9 competitive methods, including RNN-based methods like GRU4Rec [Hidasi et al., 2016] (Abbreviated as G4REC in Table 2), NARM [Li et al., 2017], Transformer-based methods like SASRec [Kang and McAuley, 2018], BERT4Rec [Sun et al., 2019] (Abbreviated as B4Rec in Table 2), CORE [Hou et al., 2022], GNN-based methods such as SRGNN [Wu et al., 2019], attention-based methods like SINE [Tan et al., 2021], and self-supervised learning models like CL4SRec [Xie et al., 2022] (Abbreviated as C4Rec in Table 2), MAERec [Ye et al., 2023].

Implementation Details. Our method is implemented in PyTorch and experiments are run on an NVIDIA 4090 GPU. The Adam optimizer is utilized for parameter inference with a learning rate of 1e-2. For the GNN component, we set the number of layers to 2. The embedding dimension is fixed at 32, with a dropout rate of 0.3 to prevent overfitting. We apply a regularization coefficient of 1e-6 to improve model generalization. The graph is constructed using a distance parameter of 3. For the parameters of the Mamba block, the SSM state expansion factor is 32, the kernel size for 1D convolution is 4, and the block expansion factor for linear projections is 2. For the Books, Retail and Toys datasets, we train the model for 150 epochs; the batch size is set to 2048; the reconstruction rate is 0.3; the maximum user sequence length is restricted to 50. For the NYC and TKY, we train the model for 300 epochs; the batch size is set to 256; the reconstruction rate is 0.6; the maximum user sequence length is restricted to 200.

4.2 Performance Analysis (RQ1)

As can be seen in Table 2, our model consistently achieves the best recommendation results. While other models with better performance (e.g., NARM and MAERec) are also able to achieve better results, their excellent performance cannot be maintained on all datasets, and the robustness of the recommendation is not good as the size of the dataset or the interaction scenario changes. Due to the different sizes and features of the datasets, the MAERec model can present better results on datasets such as shopping platforms, where the average user interaction sequences are shorter. However, its performance significantly declines when applied to check-in scenarios with longer average user interaction sequences. Conversely, NARM performs better in check-in scenarios. This all reflects that the current approach is not stable enough to consistently maintain better performance in multiple scenarios. In contrast, VGAE-GE has better performance and robustness and always maintains the best performance in multiple scenarios. On the other hand, our model mines the user's preference representation more accurately through distributional transformation and combines with the Mamba4Rec recommender to maintain good performance on all datasets, which demonstrates that our proposed VGAE-GE model has good robustness and excellent recommendation performance. Performance Against Data Noise. We replace 5%, 15%, and 25% of user interactions with random negative items to test robustness. Figure 3 shows that VGAE-GE consistently outperforms state-of-the-art model (i.e., CORE and MAERec) under all noise levels on two datasets, with lower performance degradation, demonstrating strong noise resilience.

Ablation Studies (RQ2 and RQ3)

To better understand the contributions of different components in VGAE-GE, we conduct ablation studies by removing specific parts to form the following architectures: w.o. (without) Graph Enhancement (GE), w.o. Mamba4Rec (mam) and w.o. Position Embedding (pos). The results on the five datasets are shown in Table 3. To evaluate the enhancement effect of the graph transformation based on VGAE (RQ2), we conduct ablation studies and propose a corresponding variant, w.o. GE. From Table 3, it can be inferred that the performance of the model significantly decreases across all five datasets when the graph transformation module is removed. This evidence strongly supports the effectiveness of VGAE-GE module. To evaluate the effectiveness of our recommender (RQ3), we proposed a variant, w.o. mam, in which we replace our Mamba4Rec with a widely recognized strong recommender, SASRec. The experimental results show a performance decline across all five datasets. This also demonstrates that our proposed recommender outperforms the widely used SASRec. Furthermore, to measure the role of position encoding in the Mamba4Rec recommender, we remove the position encoding and propose the corresponding variant, w.o. pos. The model with position coding removed still produces a small degradation in performance relative to the VGAE-GE model. This indicates that while the position encoding contributes to improved performance, the core strength of our model does not solely rely on it. Therefore, we retain the position encoding in the Mamba4rec recommender.

4.3 Parameter Sensitivity Analyses (RQ4)

Mamba Layer. Considering that the number of Mamba layers in the recommender can affect the model's performance, we evaluate candidate values from {1, 2, 3, 4, 5} and conduct experiments. The results for HR@10 and NDCG@10 are summarized in subfigures (a) and (d) of Figure 2, respectively. For most datasets, the model performance improved when the number of layers increased from 1 to 2, but subsequently declined with additional layers. Therefore, to standardize the model, we set the number of layers to 2.

Regularization Parameter. For the regularization parameter, we give the candidate values {0, 1e-6,1e-5,1e-4,1e3,1e-2}, and the experimental results of three representative datasets are selected and summarized in subfigure (b) and subfigure (e) of Figure 2. There is an improvement in the model performance when the regularization parameter λ=1e-6, and the subsequent performance gradually decreases. Therefore we set the value of this parameter to 1e-6.

Embedding Dimension. The embedding dimension is a crucial parameter that affects the model's performance. We conducte experiments with candidate values {8, 16, 32, 64, 128} across datasets of different sizes, and the results are summarized in subfigures (c) and (f) of Figure 2. When the dimensionality is too small, it is difficult to accommodate sufficient feature information, and at this time, boosting the dimension size is beneficial to increase the model's information representation capability. However, as the dimension is excessively raised, the computational cost of the model increases and the performance of the model tends to stabilize. For most datasets, the performance began to stabilize once the embedding dimension reached 32. To balance computational cost and performance, we set the embedding dimension to 32 for the datasets used in our experiments.

Reconstruction Rate. The reconstruction rate relates to the amount of information in the reconstructed graph. We specify candidate values {0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8} for the reconstruction rate and then conduct experiments,visualizing the results for two representative datasets: Books and NYC in Figure 4. Since different application scenarios have their unique characteristics, we need to select the best suitable reconstruction rate for different datasets. For the datasets with larger scales (i.e., Books, Retail, and Toys), we chose β = 0.3, and for the datasets with relatively smaller scales (i.e., NYC and TKY), we chose a reconstruction rate of β = 0.6.

c-hop Neighbor. To select the optimal distance for constructing the graph, we consider candidate values for c-hop from {1, 2, 3, 4, 5} and conduct comprehensive experiments on all five datasets. We visualize the results for three datasets and summarize them in Figure 5. When c = 3, the majority of the datasets achieve the best performance. When the value of c is other values, such as c = 1 or c = 5, the performance of the model is not stable enough on different datasets. This indicates that when the graph construction distance is too short, it is difficult to capture higher-order information between items, whereas an excessively long distance incorporates too much noise. In most scenarios, 3-hop neighbors basically provide sufficient neighbor information. And in order to ensure the generality of the model, we make the same settings on all datasets. Therefore, we set c = 3 for all datasets.

5 Conclusion

This paper explores a graph augmentation method based on distribution transformation, utilizing VGAE to enhance graph structure information. The embedding is converted into latent variables using VGAE, and then the hidden information is mined and then reduced to the representation of embedding using GCN. Finally, the embeddings are combined with the Mamba4Rec recommender to enhance the sequential recommendation system. In future work, we plan to design more adaptive graph structure augmentation criteria to further improve the model's adaptability. In addition, we will explore capturing complex hierarchical information between users and items in non-Euclidean space.

Acknowledgments

This work is supported by the Natural Science Foundation of Shandong Province (ZR2023MF007), Open Project of State Key Laboratory for Novel Software Technology (KFKT2024B50, KFKT2024A03), Innovation fund project for graduate students of China University of Petroleum (East China) and the Fundamental Research Funds for the Central Universities (No.25CX04046A).