Sequential Recommendation with Probabilistic Logical Reasoning

1 Introduction

Sequential Recommendation (SR) has been proposed to solve information overload in a variety of real-world applications, including e-commerce, advertising, and other areas. Meanwhile, Deep Neural Network (DNN) is widely used in SR to capture the sequential characteristics of user behaviors and generate accurate recommendations [Hidasi et al., 2016; Kang and McAuley, 2018]. Recent developments in neuralsymbolic methods [Shi et al., 2020; Shi et al., 2022; Zhang et al., 2022] have demonstrated competitive performance against DNN-based SR models, thus boosting significant research interest in combining DNN with symbolic learning.

∗Corresponding authors.

Deep learning and symbolic learning are two different approaches frequently used in the field of artificial intelligence. The former is especially a central concern in current research with the resurgence data-driven learning paradigm. Without explicit common sense knowledge and cognitive reasoning, these data-hungry strategies are typically difficult to generalize [Marcus, 2020]. In contrast, symbolic learning involves the use of logical operators (e.g., AND (∧), OR (∨) and NOT (¬)) and human-interpretable representations of data (e.g., words or numbers) to perform language understanding or logical reasoning tasks. As for SR, DNN-based works (such as SASRec [Kang and McAuley, 2018], etc.) sort to learning expressive representations of items and generating recommendations by calculating the similarity between the representations of historical interactions and target items. However, rather than only calculating the similarity score, symbolic learning-based models focus more on making predictions based on the users' cognitive reasoning procedure [Chen et al., 2021]. For example, after buying a laptop, a user may prefer to purchase a keyboard rather than a similar laptop.

At the same time, DNN and symbolic learning are complementary, and fusing them properly could combine the strengths of both approaches, thus improving the performance of deep learning models [Shi et al., 2022]. For example, symbolic learning can provide a more flexible logical structure to latent features that are learned from DNN. Additionally, the introduction of deep learning enables endto-end training of the symbolic learning and reasoning process. However, neural-symbolic learning for SR remains a challenging problem with open issues in the following aspects. First, the most recent models for logical reasoning are embedding-based. The feature description and logical representation are coupled in the same framework, which makes it hard to distinguish which latent feature contributes to feature representation or logical reasoning [Shi et al., 2020]. And intuitively, the representations that work for feature description and logical reasoning in the model may influence the recommendation differently. Second, most of them assume user preferences are static and embed users and items in a deterministic manner, but ignore that the user's tastes are full of uncertainty and evolving by nature, which incurs inaccurate recommendations [Zhao et al., 2021a].

In this paper, we enhance the classical DNN-based models with logical reasoning and propose a general framework named Sequential Recommendation with Probabilistic Logical Reasoning (short for SR-PLR). In our framework, feature embedding and logic embedding are disentangled in disparate networks, which enables SR-PLR to be benefited from both similarity matching and logical reasoning. Specifically, for the feature part, we take DNN-based SR methods (such as SASRec, GRU4Rec, etc.) as the backbone to learn the powerful latent representations of sequences. For the logical part, two transfer matrices are mentioned to convert the original feature embedding into several independent Beta distributions to represent historical items. Then two closed probabilistic logical operators (i.e., AND, NOT) are conducted on these distributions to infer the target items' distribution with the KL-divergence (Kullback-Leibler). Finally, the feature representation obtained from traditional SR methods is concatenated with the logical representation sampled from the output Beta distribution to make the prediction.

In a nutshell, the contributions of our paper can be concluded as follows:

• We propose a general framework for sequential recommendation that combines the deep learning method with symbolic learning.
• We develop a probabilistic embedding method for recommendation, and conduct probabilistic logical reasoning on users' interaction behaviors for better capturing the uncertainty and evolution of user tastes.
• We successfully implement our framework to traditional and newly released DNN SR models, and our experimental results show that the performance of all these models can be improved with the help of probabilistic logical reasoning.

2 RelatedWork

2.1 Sequential Recommendation

In early works, Markov chains, Recurrent Neural Network (RNN), and Convolutional Neural Network (CNN) are commonly used in SR. For example, FPMC [Rendle et al., 2010] relies on modeling item-item transition relationships and predicts the next item with the last interaction of a user. GRU4Rec [Hidasi et al., 2016] and NARM [Li et al., 2017] capture the sequential patterns with a multi-layer Gate Recurrent Unit (GRU) structure. Simultaneously, Caser [Tang and Wang, 2018] explores the application of CNN for sequential recommendation, which uses two types of convolutional filters to extract the information hidden in the users' sequences. Recently, SASRec [Kang and McAuley, 2018] introduces self-attention into recommendation systems and achieves great success. Beyond that, MLP4Rec [Ma et al., 2019] simply stacks multiple MLP layers to model users' dynamic preferences. S3Rec [Zhou et al., 2020] and CL4SRec [Xie et al., 2022] boost the performance with selfsupervised signals. Different from these models, we aim to integrate symbolic learning into these DNN models and endow SR with cognition ability.

2.2 Neural-symbolic Recommendation

Recent neural-symbolic recommendation models can be divided into two categories. One type of neural-symbolic model (e.g., ENRL [Shi et al., 2022] and NS-ICF [Zhang et al., 2022]) aims to build an explainable recommender system with the aid of symbolic learning. They focus on learning interpretable recommendation rules on user and item attributes, and output how these predictions are generated. Another type (e.g., LINN [Shi et al., 2020], NCR-E [Chen et al., 2021] and GCR [Chen et al., 2022]) represents logical operators with the multilayer perceptron and conducts logical reasoning with the guide of several logical regularizers. In this way, sequential behavior can be presented as a logical expression, so that conducting logical reasoning and prediction in a continuous space. However, different from these methods to conduct reasoning with solely embedding, we disentangle feature and logic embedding in different networks, and then combine feature learning and symbolic learning in a unified framework.

2.3 Probabilistic Embedding for Recommendation

Probability distributions are widely used in representing all the uncertain unobserved quantities in a model (including structural, parametric, and noise-related) and how they relate to the data [Ghahramani, 2015]. There are several works mentioned to model the users and items in probabilistic embeddings. For example, PMLAM [Ma et al., 2020] and DDN [Zheng et al., 2019] represents users and items as learnable Gaussian distributions, and uses Wasserstein distance to estimate whether the user will buy the target item or not. DT4SR [Fan et al., 2021] learns the mean and covariance with different Transformers, which performs effectively for cold-start recommendation, and STOSA [Fan et al., 2022] further proposes novel Wasserstein self-attention based on Gaussian distributions. Furthermore, some works use the deep generative model in SR with probabilistic methods. For example, VSAN [Zhao et al., 2021a] fuse Variational AutoEncoders (VAE) with self-attention networks to capture the long and local dependencies in the behavior sequences. Different from these above probabilistic models, we embed users and items as Beta distributions for not only capturing uncertainty but also facilitating logical reasoning.

3 Formalization

Formally, let U and V be user and item sets. Given u's historical behaviors Vu = {v1, v2, · · · , vm} ⊆ V, the goal of SRs is to predict the next item that u ∈ U will interact with. For DNN SR models, they take Vu as input to predict the most possible top-K items, which can be formulated as vt = argmaxvi∈V P (vm+1 = vi | Vu), where vt represents the target item of u.

In this paper, we complement classical DNN with logical reasoning. For our logic part, the sequential recommendation task can be seen as a logical reasoning task. For example, let V + u , V - u ⊆ Vu denote u's clicked and unclicked item sets, v1, v2 ∈ V + u and v3 ∈ V - u , the logical formula v1∧v2∧(¬v3) can be used to infer vt.

4 Methodology

The SR-PLR framework shown as Figure 1 consists of two main parts: feature representation learning and probabilistic logical reasoning. For the feature part, SR-PLR utilizes traditional DNNs (e.g., GRU4Rec, SASRec, etc.) as the feature encoder to extract the feature representations of users' behavior sequences. For the logic part, SR-PLR conducts logical reasoning with a probabilistic method, which will be fully explained from three points: probabilistic logic embedding, probabilistic logical operators and logical reasoning. More details are illustrated in the following parts.

4.1 Feature Representation Learning

We first briefly introduce how to learn feature representations in SR-PLR. Following the most widely used manner in SR models, the backbone feature encoder used in our framework contains two parts: the embedding layer and the feature encoder.

Embedding Layer

Since side information (e.g., sex, category, etc.) may be not always available in practice, ID information is all we need to build the model in our framework. Hence, we embed the whole items' IDs into the same latent space [Kang and McAuley, 2018] and generate the ID item embedding matrix M ∈ R|V|×d , where d is the embedding dimension. Given the u's interaction sequence, the embedding of Vu is initialized to eu ∈ Rm×d and eu = {m1,m2, ...,mm}, where mk ∈ Rd represents the item's embedding at the position k in the sequence.

Feature Encoder

Given the sequence embedding eu , a deep neural network model (e.g., SASRec) represented as fθ(·) is utilized to learn the representation of the sequence. The output representation of feature encoder Huf ∈ Rd is calculated as Huf = fθ(eu ), and chosen as the representation of u's behavior sequence.

4.2 Probabilistic Logical Reasoning

As illustrated in the Introduction section, symbolic learning could grant DNN models the capability of cognition. We are inspired by the Dual Process Theory [Sloman, 1996] in cognitive science to design two separate systems for intuitive judgment and deeper logical reasoning by using DNN and logical symbols. We: (1) design a probabilistic logical embedding method for the recommender system to represent users/items in the logical space, (2) apply probabilistic logical operators on these probabilistic embeddings, and (3) get the logical representations for the sequence by using logical reasoning.

Probabilistic Logical Embedding

Different from previous neural-symbolic works that treat the same ID embeddings as both feature representation and logical variable, two different types of embedding are used in SR-PLR to describe features and conduct logical reasoning, respectively.

Specifically, we leverage the transfer matrix to covert ID embeddings into the logic space, and use multiple independent Beta distributions to demonstrate items thus modeling the uncertainty of user's tastes. Beta distribution refers to a continuous probability distribution defined on [0, 1], and its Probability Density Function (PDF) is p[(α,β)](x) = 1 B(α,β) xα-1 (1 - x)β-1 , where x ∈ [0, 1], shape parameters α, β ∈ [0,∞], and B(α, β) is the beta function. Given ID embeddings, we use two transfer matrices represented as Wα and Wβ ∈ Rd×d to transfer M into two shape matrices α and β ∈ Rm×d :

For vi, each dimension of vi = [(αi, βi)] = [(αi,1, βi,1), (αi,2, βi,2), · · · , (αi,d, βi,d)] characterizes the uncertainty by an independent Beta distribution instead of a single value, and its PDF pvi (x) is denoted as P(vi) = [p[(αi,1,βi,1)](x), p[(αi,2,βi,2)](x), · · · , p[(αi,d,βi,d)](x)]. Following BetaE [Ren and Leskovec, 2020], we clamp all elements in α and β into (0, 109 ] after multiplying transfer matrices. Note that items' representations in SR-PLR are assumed to follow the Beta distribution rather than the Gaussian distribution because we aim to guarantee the probabilistic logical operators on these Beta embeddings are closed. More details will be illustrated in the next section.

Probabilistic Logical Operators

In this section, we define two probabilistic operators on the Beta embedding space, which are called probabilistic negation operator (N) and conjunction operator (C), respectively. Note that since the disjunction operator can be implemented using negation and conjunction with De Morgan's laws [Ren and Leskovec, 2020], only the aforementioned two operators are discussed in our framework.

Probabilistic negation operator. For probabilistic negation operator, N(vi) is defined as the reciprocal of vi:

It can be seen that, different from [Shi et al., 2020] that relies on logical regularizations, N naturally satisfies N(N(vi)) = vi. In addition, as shown in Figure 1, operator N can inherently reverse high probability density to low density and vice versa [Ren and Leskovec, 2020], which enables N(vi) to represent the dislikes item vi of users.

Probabilistic conjunction operator. Given a user's behaviors Vu = {v1, v2, · · · , vm} and their embeddings v1 = [(α1, β1)], v2 = [(α2, β2)], · · · , vm = [(αm, βm)], we define the output of conjunction v = C({v1, v2, · · · , vm}), where C is probabilistic conjunction operator. Following [Ren and Leskovec, 2020], v's distributions can be represented as

where ∑ and ⊙ are the element-wise summation and product, respectively. wi ∈ Rd is a weight vector. Its j-th dimension wi,j satisfy Σmi=1wi,j = 1. In this way, the PDF of v is calculated as the weighted product of Vu' PDFs:

where ∏ is the element-wise product. In SR-PLR, we adopt the attention mechanism to learn the importance of different items during training:

where ⊕ represents concatenation operation, and MLP : R2d → Rd is a multilayer perceptron taking the concatenation of α and β as its input thus making our logical operators can be learned end-to-end. Obviously, the defined conjunction operator satisfies the equation C({v1, v1, · · · , v1}) = v1 and does not need any logical regularization. It should be pointed out that both operators are closed in the Beta embedding space, which makes these operators could be combined in arbitrary ways and prevents exponential computation [Ren and Leskovec, 2020].

Logical Reasoning

After defining logical embedding and operators, it is convenient for us to conduct reasoning on historical items. For example, given v1, v2 ∈ V + u and v3 ∈ V - u , v1 ∧ v2 ∧ (¬v3) can be represented as C({v1, v2,N(v3)}) to calculate the target items' distribution. Generally, for v1, v2, · · · , vm ∈ V + u and sampled negative item vm+1, vm+2, · · · , vn ∈ V - u , we apply logical operators on them

As logical operators N and C are closed, vu is also a Beta embedding to represent the user's preference. Hence, by measuring the KL-divergence distance between vu and target item's distribution vt, the reasoning result of logic network can be obtained:

where Pk(vu) represents the k-th dimension of P(vu). To this end, we utilize a pair-wise loss (e.g., Bayesian Personalized Ranking (BPR) [Rendle et al., 2009]) to optimize the parameters of this logic network:

where v- is a embedding of v- ∈ V - u , and σ is the sigmoid function.

4.3 Training and Prediction

To fuse DNN and logic network together, we are going to concatenate the feature representation Huf with logical representation Hul together to make the final prediction. The most direct way to get logical representations is to take samples from the distribution vu = [(αu, βu)]. However, the sample operation is not differentiable, which makes the process hard to be trained end-to-end. Hence, in our framework, we choose the mean of distribution vu to represent the sequence:

Then, the corresponding prediction matrix ˆy ∈ R|V| can be generated by:

where E = α α+β . And we use the cross-entropy loss to approximate the ground truth y:

At last, the final objective function is:

where λ is a hyperparameter.

5 Experiments

In this section, we conduct experiments with the aim of answering the following questions: Q1: How do our models SR-PLR perform compared with other baselines? Q2: What is the influence of key components of SR-PLR? Q3: Whether is SR-PLR sensitive to the hyperparameters? Q4: How is the robustness of SR-PLR?

5.1 Experimental Settings

Dataset

Experiments are conducted on three publicly available datasets, Amazon Sports, Toys [He and McAuley, 2016] and Yelp. Following previous works [Kang and McAuley, 2018], we use the '5-core' version for all datasets and adopt the leave-one-out method to split these three datasets. The statistics of each dataset are given in Table 1.

Evaluation Metrics

The widely used Normalized Discounted Cumulative Gain at rank K (NDCG@K) and Hit ratio at rank K (Hit@K) are taken as evaluation metrics in our experiments, and we choose K from 5, 10. As recommended by [Krichene and Rendle, 2020], we adopt all-rank evaluation scores throughout the entire item set to ensure that the evaluation process is unbiased. As [Shi et al., 2020; Chen et al., 2021] are evaluated with sampled metrics in the original paper, we compare SR-PLR with our re-implemented version.

Baseline Methods

Two groups of baselines are used to compare with our models.

DNNbasedmodels. For this group, we compare the Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), self-attention based models, and newly released contrastive learning based model. They not only act roles as baselines, but also as the backbone of SR-PLR to demonstrate the effectiveness of combining logical reasoning.

• GRU4Rec [Hidasi et al., 2016]: a method utilizing GRU to model user sequential behaviors as a strict order,
• Caser [Tang and Wang, 2018]: a method using both horizontal and vertical convolution for SR,
• SASRec [Kang and McAuley, 2018]: a method based on the self-attention mechanism,
• DuoRec [Qiu et al., 2022]: a method developing a positive sampling strategy and using contrastive learning for SR with a model-level augmentation.

Neural-symbolic models. For this group, we compare our framework with some neural-symbolic recommendation methods. As mentioned in [Chen et al., 2021], there are two different versions that base on implicit feedback NCR-I and explicit feedback NCR-E. Since SR-PLR is implicit feedback based, for fair comparisons, we use the implicit feedback versions here for these two neural-symbolic baselines.

• LINN [Shi et al., 2020]: a neural collaborative reasoning based recommendation method,
• NCR-I [Chen et al., 2021]: a personalized method based on LINN.

Implementation Details

We run all methods in PyTorch [Paszke et al., 2017] with Adam [Kingma and Ba, 2015] optimizer on an NVIDIA Geforce 3070Ti GPU, and all these models are implemented based on RecBole [Zhao et al., 2021b]. The batch size and the dimension of embeddings d are set to 2048 and 64 in our experiments. The max sequential length for all baselines is set as 50. We train all models 50 epochs. In the experiment, we keep all the hyperparameters of backbone models in RecBole unchanged and stack our logic network on them. SR-PLR is trained with a learning rate of 0.002. For the logic network, we set the λ in Eq. (12) as a hyperparameter and select it from [0, 1] with step 0.1. For the negative item number in Eq. (6), we choose it from 1 to 10.

5.2 Overall Performance (Q1)

We report the experimental results of different methods in Table 2. Note that 'XX L' means the SR-PLR method that uses 'XX' as the backbone. Where the 'N' is short for NDCG and the numbers in bold indicate the better results that are generated by the same feature encoder. As we can see, the methods combining logical reasoning achieve better performance than other comparative models. Besides, there are more findings in these comparative experiments.

Firstly, SASRec achieves better performance than GRU4Rec and Caser for both NDCG and Hit on all three datasets. It shows that modeling the importance of different interactions is highly effective for describing users' preferences. The main reason is that the items in the sequences contribute differently to final results, e.g., the laptop makes more contributions than clothes in buying the keyboard, which is also why the attention mechanism is used in our probabilistic operator. Secondly, DuoRec performs better than SASRec in most cases, suggesting that pushing the positive views closer and pulling negative pairs away by contrastive learning are meaningful for getting better representations. Thirdly, both neural-symbolic models get similar results with GRU4Rec (consistent with the results reported in [Shi et al., 2020; Chen et al., 2021]). They also leverage deep neural networks to conduct logical reasoning, but only considering the logical equations of items may be not adequate for describing the complex relationship among interactions.

Finally, SR-PLR achieves performance improvement based on four types of feature encoders and performs best on three datasets. We contribute the improvement to the following aspects: (1) combining deep learning and symbolic learning in a dual feature-logic network, and (2) modeling users' dynamic preferences with a probabilistic method. As self-attention based SASRec performs better than RNN and CNN, meanwhile DuoRec is also the development of SASRec, we pay more attention to SASRec and perform more detailed experiments based on SASRec L.

5.3 Ablation Study (Q2)

In this section, we conduct three ablation studies on all three datasets. We are going to examine the effectiveness of main three components, which are the attention mechanism in the probabilistic conjunction operator, the probabilistic negation operator and the feature network. Hence, three variants (which are named 'w/o att', 'w/o neg oper' and 'w/o feat')

are designed to compare against SASRec L, and the results are shown in Figure 2. The details of these variants are as follows:

w/o att. We first evaluate the necessity of item-level attention in our probabilistic conjunction operator. The variant 'w/o att' means to compute sequence distribution by aggregating the hidden states via average pooling operations in Eq. (5). From Figure 2, we can see that our SASRec L far outperforms its variant without the attention operation on three datasets. It indicates that using a fixed weight matrix will be a bottleneck to modeling the dynamic evolving tastes of users. Our attention based logical operation can adaptively measure the importance score for each item, which helps to improve the model performance.

w/o neg oper. The negation operator is also an essential part of SR-PLR. For w/o neg oper, we use positive items and conjunction operator to calculate the distribution of sequences, that is using vu = C({v1, v2, · · · , vm}) in Eq. (6). From Figure 2, the recommendation performance can be booted with the help of operator N. Note that there is nearly no work to consider to conclude negative items during representing the interaction sequences except neural-symbolic based models. Our experiments indicate that the negative operator is vital to make logical reasoning complete, and using the unclicked items with the negative operator in symbolic learning may be a way to leverage unlabeled data, which may be useful for the semi-supervised recommendation.

w/o feat. We also evaluate the effectiveness of the feature network. For w/o feat, we alternatively make the prediction by using ˆy = Hul E⊤ for Eq. (10). As shown in Figure 2, only using the probabilistic logic network will result in a performance drop, which indicates the importance of combing feature learning with symbolic learning, and the effectiveness of probabilistic logic network in boosting feature learning.

5.4 Hyper-parameter Sensitivity (Q3)

In this section, we are going to investigate how the performance varied with the changes of λ and the sampled negative item number. In experiments, we vary one of these two hyperparameters and keep others unchanged to investigate the influence of λ and the negative item number.

Sensitivity of λ. For λ, we conduct the experiment in the range from 0 to 1 for all three datasets. The experimental results are shown in Figure 3. In this figure, we compare SASRec L with SASRec under different λ. It can be seen that adding a logical regularizer, i.e., the Ll in Eq. (8), to optimize the logic network is helpful for the recommendation performance. Essentially, it acts as an auxiliary task to push the sequences' distribution close to the positive items' distributions and far away from the negative ones. In addition, it matters to carefully find a suitable value for each dataset, and values around 0.5 are recommended.

Sensitivity of sampled negative item number. For the negative item used for operator N , we choose it ranges from 1 to 10, and the experimental results are indicated in Figure 4. It can be seen that, in most cases, introducing negative items is effective for logical reasoning, especially in Sports and Yelp. As we use implicit feedback in SR-PLR, the sampled negative items may be not truly disliked by the user, hence the number of negative items is set relatively small during training to maintain the semantic of sequence.

5.5 Robustness Analysis (Q4)

In the logic network, we endow SR-PLR with the capability of modeling uncertainty by using the Beta distribution to represent the items. To verify this point, we randomly mask some items in the sequence with probability r during training and evaluate the performance of models in the original test dataset. Ideally, the item distribution contains more information than the static embedding, making SR-PLR could maintain its superiority with distorted sequences. As the results in Table 3 illustrated, though both models' performances slowly drop, SASRec L still beats SASRec in most cases when r varies from 0.1 to 0.5 in two datasets, which indicates SRPLR is more robust by using the probabilistic method.

6 Conclusion

In this paper, we proposed a general framework named SRPLR to combine deep learning with symbolic learning. Our main idea is two-fold: One is to disentangle items' embeddings and endow the DNN based SR with cognitive capability; Another is to model the uncertainty and dynamic tastes of users. Therefore, we designed a dual feature-logic network and applied probabilistic logical operators on the Beta embeddings. Experiments on three real-world datasets showed that SR-PLR exhibits significant improvement against traditional baselines and neural-symbolic models. Our future work will consider modeling content information and knowledge graph with more interpretable and transferable logic neural networks in SR, and that may lead this framework to be more transparent and achieve significant performance.

Acknowledgments

This research is partially supported by the NSFC (61876117, 62176175), the major project of natural science research in Universities of Jiangsu Province (21KJA520004), Suzhou Science and Technology Development Program (SYC2022139), the Priority Academic Program Development of Jiangsu Higher Education Institutions.