SeeDRec: Sememe-based Diffusion for Sequential Recommendation

1 Introduction

Recommender system (RS) has become essential for many real-world applications owing to its ability to accurately mine users' personalized interests [Wang et al., 2015; Fan et al., 2023; Meng et al., 2020; Ma et al., 2021; Ma et al., 2023a]. Recently, numerous advanced techniques in NLP have been assimilated into RS, continuing to demonstrate their impact [Kang and McAuley, 2018; Sun et al., 2019; Ma et al., 2023b]. The sequential modeling process of user behaviors in sequential recommendation (SR) bears semblance to the language modeling task prevalent in NLP, that is, SR seeks to recommend the next-item that user may be interested in by modeling the sequential dependencies of user's temporal behaviors [de Souza Pereira Moreira et al., 2021].

As motivated by the outstanding distribution generative performance of the Diffusion Model (DM) in various domains (image synthesis [Ho et al., 2020], audio processing [Kong et al., 2021] and semantic segmentation [Brempong et al., 2022]), some pioneering studies attempt to explore the effectiveness of DM in recommendation. DiffRec [Wang et al., 2023e] conducts DM on item indices to infer users' interaction probabilities in a denoising manner, CDDRec [Wang et al., 2023f], DiffuRec [Li et al., 2023], DreamRec [Yang et al., 2023] and DDRM [Zhao et al., 2024] consider injecting uncertainty into item embeddings via DM under the generative paradigm. PDRec [Ma et al., 2024a] enhances DiffRec by leveraging the pre-trained DM on item indices as plugins to improve SR, which exhibits universality and analogous inference costs to the sequence encoders. However, the ID-based methods typically exhibit: (1) a linear increased complexity with the growing number of corpus, leading to the worse scalability in large-scale RS, (2) an excessive dependency on item indices, lacking the cross-domain transferability, (3) a limitation in fully utilizing the semantic correlations within items, thereby resulting in sub-optimal performance.

To tackle these challenges, an intuitive idea is to conduct DM on other objectives that are atomic, multi-domain transferable, and encapsulate the semantic correlations requisite for SR, rather than item indices. We notice the definition of sememe, which is regarded as the minimum semantic unit in NLP [Niu et al., 2017]. NLP researchers believe that each word can be decomposed into a limited set of manuallydefined and language-independent sememes [Qi et al., 2022]. Inspired by this, we creatively propose the concept of recommendation sememe (abbreviated as sememe) to represent users' minimal interest unit in recommendation. As shown in Fig. 1, all items can be represented as the combination of several sememes, thereby reducing the quantity of DM objectives, accomplishing the transferability across diverse domains and incorporating semantic correlations among items in SR. To do this, we present a model-agnostic Sememebased Diffusion framework for Sequential Recommendation (SeeDRec). It regards each user's original interest on sememes observed by the system as a "Seed", nourishes it with both local and global user preferences, and cultivates the "Blooming flower" of generalized interest distribution via our sememe-based diffusion. Specifically, we first propose the Sememe-to-Interest Diffusion Model (S2IDM) to upgrade the existing DM-based recommenders from item index to sememe, mining each user's generalized interest distribution by synergistically considering temporal, frequency, and cooccurrence information interlinking sememes. The user generalized interest distribution learned by S2IDM contains both the user's local personalized behavioral preference and the global sememe correlations implied by all user behaviors. Furthermore, to incorporate the generalized interest distribution into the sequential modeling, we design an Interest-aware Prompt-enhanced (IPE) strategy to guide the sequential modeling towards the direction for better personalized behavior understanding. With the sememes powered by S2IDM and IPE, SeeDRec could achieve improvement on various base models as a plugin with better scalability and interest generalization. Moreover, the knowledge of user interest diffusion encoded in S2IDM can be transferred to other domains via our proposed recommendation sememe anchors.

We conduct extensive experiments on nine SR datasets and four cross-domain SR (CDSR) datasets to demonstrate the superiority of SeeDRec. For simplification and universality, we also use existing taxonomies and words as sememes to simulate practical scenarios. We further conduct various ablation studies, universality analyses, few-shot analyses, and model analyses to validate the effectiveness of the proposed S2IDM and IPE. The contributions of this work are as follows:

• We have verified the feasibility of conducting diffusion on the recommendation sememe and incorporating it in SR. To the best of our knowledge, we are the first to conduct DM on sememes to enhance SR and CDSR tasks.
• We propose S2IDM to mine diffused user generalized interest distribution by simultaneously considering global sememe correlations and users' local sequential behaviors.
• We creatively present the IPE strategy to enable the precise interest transfer from the discrete sememe interest distribution to the continuous representation to improve the personalized sequential modeling via a prompt learning paradigm.
• We conduct extensive evaluations on 13 real-world datasets to verify that the proposed SeeDRec is effective, universal, and easy-to-deploy. We also design comprehensive analyses to demonstrate the effective mechanism of SeeDRec and validate it in the more challenging few-shot scenarios.

2 RelatedWork

Sequential Recommendation Benefiting from the advancement of sequential modeling techniques, SR is proposed to format and encode user temporal behaviors to infer their dynamic interests [Sun et al., 2022; Wang et al., 2023g;

Wang et al., 2023d; Sun et al., 2024]. Within its evolutionary trajectory, early SR methods reason users' short-term preference through the Markov Chains [Rendle et al., 2010]. GRU4Rec [Hidasi et al., 2016] leverages the Gate Recurrent Unit as the sequential encoder to capture users' long-term dependencies. Caser [Tang and Wang, 2018] imports the Convolutional Neural Network (CNN) to extract sequential patterns from user behaviors at different time intervals. Inspired by the success of self-attention mechanism, SASRec [Kang and McAuley, 2018] incorporates it to jointly model users' short- and long-term preferences. CL4SRec [Xie et al., 2022] and MStein [Fan et al., 2023] additionally utilize the mutual information on self-supervised signals to improve the sequential representations. Unlike the complex network structures and stochastic augmentations in previous SR works, SeeDRec smartly leverages the prevailing diffusion models to incorporate the generalized user interests in sequential pattern modeling. Furthermore, SeeDRec can be effortlessly deployed on diverse SR models with advanced techniques to bring significant and consistent performance improvements (see Sec. 4.5).

DiffusionModel for Recommendation Inspired by the remarkable performance in image generation [Ho et al., 2020; Nichol and Dhariwal, 2021; Wang et al., 2023b],scene text editing [Wang et al., 2023a], and machine translation [Chen et al., 2023], some researchers attempt to incorporate DM in recommendation. CODIGEM [Walker et al., 2022] is the pioneer DM-based recommender which generates improved collaborative signals via the Denoising Diffusion Probabilistic Model (DDPM). DiffRec [Wang et al., 2023e] reduces the scheduled noises into the reverse process to infer user interaction probabilities in a denoising manner. PDRec [Ma et al., 2024a] is the state-of-the-art DM-based recommender, which takes the DM trained on all corpus as the plugin in SR to fully leverage the diffusion-based user preferences. Moreover, a series of related DM-based works (e.g., DiffuRec [Li et al., 2023], DiffRec* [Du et al., 2023] and DreamRec [Yang et al., 2023]) employ DM on the continuous item embedding space with additional transitions and noising strategies, which shares the same modeling pipeline with classical SR methods. Disparate objectives make them inherently non-comparable with this work, and they can serve as the base SR model within our SeeDRec. In particular, we have employed PDRec as the backbone to substantiate the efficacy of SeeDRec over DM-based recommenders.

In conclusion, existing DM-based methods typically conduct DM on the discrete item indices or the continuous embedding of each item to achieve the uncertainty injection in recommendation. Therefore, the augmentation of the number of items exacerbates the multiplication of their time and space complexities, rendering them arduous to deploy in real-world million-level recommendation systems. Moreover, these DM-based recommenders solely rely on item indices, demonstrating the lack of scalability and the inability to fully leverage the semantic correlations within items. Every instance of encountering a new dataset compels them to be trained from scratch without leveraging the transferability across multiple domains, leading to the recurrent process that demands substantial computility and time consumption.

3 Methods

3.1 Problem Statement

In this article, we focus on exploring the DM-enhanced recommendation task in multiple recommendation scenarios, with an illustration of the example of SR. We define the behavior sequence SI u = {i1, i2, · · · , ip} of user u ∈ U, where ij ∈I is the j-th interacted item of user u and p denote the historical behavior length. Given SI u , we adopt SASRec [Kang and McAuley, 2018] as the sequential encoder to predict the target item ip+1 that will be preferred by user u.

3.2 Overall Structure

In this section, we elaborate on the proposed model-agnostic Sememe-based Diffusion for sequential recommendation, which utilizes the DM on sememes to explore each user's atomized multi-interest units for the multiple recommendation scenarios. As illustrated in Fig. 2, the proposed SeeDRec consists of two main components, including the Sememeto-Interest Diffusion Model (S2IDM) and the Interest-aware Prompt-enhanced (IPE) strategy. Specifically, SeeDRec first proposes S2IDM, which creatively conducts diffusion on the sememe level rather than the item level used in conventional works, with the aim to model users' local minimal interest units from the global interest diffusion aspect. To adeptly guide the attention of sequential modeling towards the user's personalized interests, SeeDRec designs an IPE strategy to convert the diffused generalized interest distribution into informative prompts via multiple personalized prompt generators. It enables the accurate extraction of the interest-centred information from each user's historical behavior sequences, thereby achieving precise user dynamic interest modeling. Furthermore, SeeDRec continues to manifest its superiority in SR and CDSR tasks where the explicit item-sememe hierarchical taxonomy information is even absent (as a more challenging setting with increased noise). Detailed experimental results and analyses can be found in Sec. 4.2 and Sec. 4.4.

3.3 Sememe-to-Interest DiffusionModel

In this section, we describe our Sememe-to-Interest Diffusion Model based on DDPM [Ho et al., 2020], named S2IDM. The overall structure of S2IDM is illustrated in the left part of Fig. 2. In contrast to the direct diffusion employed by DiffRec [Wang et al., 2023e] and PDRec [Ma et al., 2024a] at the ID indices, S2IDM creatively deploys DM at the sememe level, concomitantly refining it in accordance with the intercorrelation of sememes, the sememe overlap of item and the multiple interest drift of behavioral sequences.

Forward Process

In the forward process, S2IDM corrupts the original sememe distribution of each user by injecting the Gaussian noises step by step in a discrete manner. To be specific, given the behavioral sequence SI u in the index level, S2IDM first extracts the prior sememe behaviors, re-weights it via the frequency and time interval of each sememe to obtain the original interest distributions′0 and finally corrupts it in a discrete manner.

Diffusion on sememe With the behavioral sequence SI u = {i1, i2, · · · , ip} of user u, we first LOOKUP the prior sememe behaviors su = {si1 1 , si1 2 , · · · , sip k } of user u as the initial input ˆ s0 from the item-sememe dependency matrix DS ∈ R|I|×q , where sip k ∈ S denotes the k-th sememe of item ip and q = |S| denotes the total length of the sememe set S. This enables the transfer of DM from the item domain to that of sememe, yielding a threefold advantage: (1) It entails a reduction in the space and time complexity of DM as stipulated by |S| ≪ |I|, enabling the practical scalability of our sememe-based diffusion model. (2) By virtue of its foundational definition as the minimal unit of interest, the inherent multi-domain sharing characteristic of sememe endows it with transferability in CDSR tasks. (3) It brings in the semantic interrelationship inherent in sememes to furnish comprehensive support to DM.

Frequency-time aware reweighting In light of the pervasive one-to-many association between items and sememes, highly correlated sememes tend to iteratively manifest within user behavior sequences. The unadorned utilization of the reweight strategies in TI-DiffRec is untenable, obscuring users' authentic interests. To this end, we first count the frequency of each sememe in user behaviors to obtain the s0, then we conduct the time-interval reweighting strategy in TIDiffRec to generate the time-interval weights wik 1 = wik 2 = · · · = wik j = wmin+ tik -t1 tp-t1 (wmax-wmin) for all sememes sik 1 ,sik 2 ,· · ·,sik j of item ik with the item-level timestamp sequence T I u = {t1,t2,· · ·,tp}. Here sik j is the j-th sememe of item ik in s0. Ultimately, we assign these weights to each sememe, amalgamating the weight of identical sememes to derive the original interest distributions′0 .

Discrete noising Discrete noising is one of the core phases in DM that injects uncertainty into the original interest distribution. Different from the existing DM-based recommenders, we gradually corrupt the original interest distribution s′0 on sememe via a forward transition q(s′t |s′t-1) = N (s′t ; √1 - βts′t-1 , βtI), whereβt ∈(0, 1)denotes the scale of the added Gaussian noise at the step t, which is generated by the linear noise schedule [Wang et al., 2023e; Li et al., 2023]. With the reparameterization trick [Kingma and Welling, 2013] and the inherent additivity of the independent Gaussian distribution, we can formulate that s′t = √ ¯αts′0+√1 - ¯αtϵ, where ϵ ∼ N(0, I) is the added Gaussian noise, αt =1-βt and ¯αt = ∏tt′=1 αt′ . We also observe that as t→+∞, s′t undergoes a gradual convergence towards the standard Gaussian distribution.

Reverse Process

The task of the reverse process in DM is to remove the added Gaussian noise step by step to recover the interest distribution s′0 from the perturbed sememe distribution s′t . The related reverse transition pθ (s′t-1 |s′t ) is defined as: pθ (s′t-1 |s′t ) = N (s′t-1 ; µθ (s′t , t);Σθ (s′t , t)), where the mean µθ (s′t , t) and variance Σθ (s′t , t) can be modeled by a deep neural network due to the fact that the relatively small Gaussian noise ensures the transition kernel pθ (s′t-1 |s′t ) follows a Gaussian distribution. The iterative process can be formulated as follows:

Optimization Strategy

The training objective of S2IDM is to force the reverse transitionpθ (s′t-1 | s′t ) to closely approximate the posterior distribution q(s′t-1 |s′t , s′0 ), which is achieved by minimizing the Kullback-Leibler (KL) divergence. With the simplification [Ho et al., 2020] and importance sampling technique [Nichol and Dhariwal, 2021], the above DKL can be rewritten as the weighted Mean Square Error (MSE) loss LD to focus on more difficult denoising tasks and alleviate the unnecessary noise:

where g(t) denotes the discrepancy between the Signal-toNoise Ratio at step t and t - 1 with the sampling probability, and xθ(.) is a deep neural network [Wang et al., 2023e].

Diffusion Inference

Intuitively speaking, the inherent occasional noise pervades the collected user behaviors, and the user generalized interest mining at the sememe level will also inevitably introduce a certain degree of bias. Consequently, we refrain from introducing additional noise and instead treat the original sememe distribution s′0 as the inherently noise-enriched ¯ s′t , proceeding directly with the reverse process on it as ¯ s′t → ¯ s′t-1 → · · · → ¯ s′0 . This not only retains more personalized information to improve the precision of DM but also ensures that the inference process is not initiated from a fully disordered state, leading to a more robust inference process that precisely aligns with the intrinsic nature of recommendation tasks.

3.4 Interest-aware Prompt-enhanced Strategy

It is intuitive that there exists an insurmountable variation between the diffusion-based user interest and the temporal preference, which motivates us to explore how to incorporate the generalized interest distribution into the sequential modeling. In parallel, the interest distribution obtained from S2IDM (a) exhibits correlation with the generalized user interests, so as to provide some attention clues that are somewhat relevant to SR tasks, and (b) enables the multi-domain inter-connectivity at the sememe level, thereby leveraging its semantic relationships to uncover nuanced user preferences and yield enhanced information gain. To this end, we follow the prompt learning paradigm [Lester et al., 2021; Wu et al., 2024] to propose the Interest-aware Prompt-enhanced (IPE) strategy to convert the generalized interest distribution on discrete sememes into multiple continuous prompts. This can smooth the bias between the semantic and behavioral intentions and guide the sequential modeling towards understanding personalized behavioral patterns to enrich the user representation.

Specifically, given the generalized interest distribution ¯ s′0 , we first project it into the same feature space via multiple prompt generators fθ(.), which is built with a two-layer fullyconnected network with the Tanh activation. After obtaining the multi-interest prompts M = {m1,m2,· · ·,mk} where mi = fθi(¯ s′0 ), we place these prompt-enhanced knowledge before the original input matrix DI to inject the diffused user generalized interests into the self-attention functions in pretrained sequential models. Here we designate the length k of the multi-interest prompts as 3 for streamlining. Beyond the rectification of diffusion objective within S2IDM, the lower computational complexity of SeeDRec is also evident in various facets. First is that the generalized interest distribution ¯ s′0 used in IPE only needs to be generated once via the inference process of S2IDM before model training, yielding the asymptotic similar time complexity to its sequence encoder in online serving. Secondly, the implementation of IPE only requires a limited number of additional prompt generators to be trained, which also mitigates the space complexity of the existing DM-based recommenders. Note that the proposed IPE is an universal strategy, devoid of the tailored design in sequence modeling in SR. Thus, IPE can harness the possible advancement in future sequence modeling, thereby extending the lifecycle of SeeDRec (see Sec. 4.2 and Sec. 4.5).

Optimization Objective Following [Kang and McAuley, 2018], we opt the Binary Cross-Entropy Loss L of each useritem pair (u, i) in training set R as the optimization objective:

where ˆ yu,i = u⊤ei is the predicted probility between user representation u and item embedding ei, yu,i = 1 and yu,i = 0 denote the positive and negative samples respectively.

3.5 In-depthModel Discussion

The most related work of SeeDRec in existing DM-based recommenders is PDRec [Ma et al., 2024a], which fully leverages the diffused preference on item indices to improve SR. Since the DM within PDRec is trained on item indices, its time complexity will be limited by the number of item corpus, thereby encountering the scalability challenge. Secondly, due to the ID-based diffusion, PDRec relies on overlapped users to re-train the DM from scratch in CDSR tasks. Finally, the ID-based DM proves challenging in fully harnessing the associative information within the item semantic hierarchy, potentially resulting in sub-optimal performance.

In contrast, SeeDRec proposes S2IDM on the discrete sememes to mine each user's generalized interest distribution through minimal interest units modeling. Hence, the time complexity on the DM side can be simplified from |I|·O(D) to |S| ·O(D), where O(D) denotes the time complexity of all other facets within DM. According to the consensus of |S| ≪ |I|, S2IDM successfully addresses the long-standing scalability challenge that has persistently plagued PDRec. Moreover, SeeDRec exhibits the capability to comprehensively leverage the semantic associative relationships and cross-domain transferability under the sememe hierarchy (see the complexity comparison in Sec. 4.3). This effectively addresses the cross-domain generalization and semantic correlation challenges inherent in PDRec. The experimental results in Sec. 4.2 concurrently affirm that SeeDRec can yield significant and consistent performance improvements upon PDRec. Besides, we also employ the masked discrete DM [Austin et al., 2021] in S2IDM, which yields comparable SR performance. So we just implement S2IDM in the regular mannner to simplify the computational complexity.

4 Experiments

We conduct extensive experiments on nine SR datasets and four CDSR datasets to answer the following questions:

• RQ1: Does the proposed SeeDRec outperform the base SR models and the SOTA DM-based SR methods?
• RQ2: How does SeeDRec perform in datasets where explicit item taxonomies (e.g., categories) are absent?
• RQ3: How does each component proposed in SeeDRec impact the recommendation performance?
• RQ4: Is our SeeDRec effective and universal enough with different base SR models and cross-domain SR tasks?
• RQ5: How does SeeDRec function in the interest distribution transfer and the few-shot scenarios?

4.1 Experimental Settings

Datasets As shown in Table. 1, we construct five SR datasets from two platforms (i.e., Amazon [Lin et al., 2022] and PixelRec [Cheng et al., 2023]) with existing categories viewed as sememes. To verify that SeeDRec could work well without existing taxonomies, we also build four SR datasets and four CDSR datasets (with similar sizes) respectively, intentionally overlooking the inherent categories and using words as natural sememes via certain tokenization, lemmatization, and filtering on item titles with the NLTK library.

Baselines We implement SeeDRec on two SR models (SASRec [Kang and McAuley, 2018] and CL4SRec [Xie et al., 2022]) and a SOTA DM-based model (PDRec [Ma et al., 2024a]) to verify its effectiveness and universality. Besides, we also compare with two DM-based CF models (T-DiffRec [Wang et al., 2023e] and TI-DiffRec [Ma et al., 2024a]).

Parameter settings We conduct a comprehensive grid search to select the optimal hyper-parameters. That is, the learning rate is tuned from 0.001 to 0.05. The batch size and the maximum sequence length are defined as 512 and 200 for fair comparisons. It is imperative to underscore that SeeDRec essentially possesses very few parameters (e.g., k in IPE). We just assign k = 3 via our empirical knowledge. For S2IDM, we define ωmin = 0.1, ωmax = 0.5 and the step T as 10 for all datasets. We use the early-stop strategy to avoid overfitting. Following [Wang et al., 2023c], we randomly sample 999 negative items for each positive instance to speed up the evaluation. All reported results are the average values of five runs with different seeds on the same NVIDIA Tesla V100.

4.2 Performance on SR (RQ1 &RQ2)

In this section, we leverage NDCG@10 (N@10), Hit rate@10 (H@10) and AUC as the evaluation metrics. From Table 2, we can observe that: (1) SeeDRec achieves significant improvements on all metrics and datasets compared to its base sequential models, with the significance level p < 0.05. It indicates that SeeDRec can precisely mine the user's interest distribution and successfully incorporate it into SR via the tailored multi-intention prompts on different types of base SR models. (2) Upon a lateral examination across various datasets, SeeDRec is more beneficial on relatively denser datasets. This aligns with its foundational assumption, that is, S2IDM can intricately better capture users' generalized interest distributions from the diffusion on minimal interest units on denser datasets where user behaviors are abundant. (3) SeeDRec attains the peak results across most of the datasets on the basic of PDRec. It confirms that SeeDRec exhibits the different underlying mechanisms of DM utilization with PDRec, positioning it as a stalwart support for the evolution of more advanced DM-based recommenders in the future.

To verify the practical usage of SeeDRec when datasets do not have appropriate categories, we build a more challenging setting that directly uses words of item titles (broadly existed in authentic datasets) as sememes. From Fig. 3, we observe that SeeDRec functions well without existing taxonomies (even comparable with using categories as sememes). It verifies the effectiveness and robustness of SeeDRec.

4.3 Ablation Study (RQ3)

To verify the effectiveness of each component in SeeDRec, we implement two versions, SASRec w/ S2IDM (leveraging a MLP-based feature fusion to replace our prompt-based fusion with the generalized user interest distribution) and SASRec w/ IPE (replacing the generalized user interest distribution given by S2IDM with the original interest distribution) for comparisons. We conduct ablation studies on four SR datasets (word as sememe) in Fig. 3, five SR datasets (category as sememe) in Fig. 4, and four CDSR datasets in Table 4. Here, SeeDRec equals SASRec+IPE+S2IDM.

We notice that: (1) Each component in SeeDRec can bring incremental improvement over its backbone on all datasets, where SeeDRec outperforms all ablation versions on different tasks, error range<0.003. It validates the effectiveness of both IPE and S2IDM in SeeDRec. (2) The comparison between SeeDRec and SeeDRec w/ IPE demonstrates the indispensability of S2IDM in SeeDRec, which simultaneously model the local personalized interests and the global sememe correlations via the tailored DM. Table 3 also verifies that the modification of diffusion objective enables the training and evaluation time of S2IDM in a low magnitude, making it hopeful in large-scale recommendation applications.

4.4 Performance on Cross-domain SR (RQ4)

Based on the transferability of SeeDRec when using words as sememes, SeeDRec could also handle CDSR tasks even without overlapped users (which is indispensable for most CDR models [Ma et al., 2024b]). Specifically, we directly adopt the pre-trained and fixed S2IDM learned from the source domain (word as sememe) to infer the interest distribution (merely from the original sememe distribution in the target domain), and adopt it via IPE to enhance the target-domain SR.

The results in Table 4 demonstrate that: (1) SeeDRec outperforms all its ablation versions in CDSR, indicating the effectiveness and transferability of SeeDRec across domains. Both IPE and S2IDM are essential in CDSR. (2) It is crucial to emphasize that we directly use the source-domain S2IDM for the target domain SR, which is more challenging. Moreover, SeeDRec does not impose a strict mandate for user overlap, and thus is more practical and easy-to-deploy. These implicate the potential application of SeeDRec as a universal diffusion-based model for CDSR: training a general multidomain S2IDM based on words, and transferring it to various target domains through domain-specific IPE.

4.5 Universality Analyses (RQ4)

We conduct SeeDRec on different classical SR models in five SR datasets to verify its universality. From Table 2 and Fig. 4, we have: (1) SeeDRec is a model-agnostic method that simultaneously exhibits its superiority on SASRec, CL4SRec, and even diffusion-based PDRec. Meanwhile, each component in SeeDRec achieves its incremental improvement on most datasets with different base models. (2) CL4SRec and PDRec outperform SASRec by advanced techniques such as contrastive learning and diffusion model, which is also reflected when armed with SeeDRec. It implies the compatibility between the sememe-based diffusion of SeeDRec and advanced sequential modeling techniques. Consequently, SeeDRec is likely to retain its universality and effectiveness in cooperating with future sophisticated sequential models.

4.6 In-depthModel Analyses (RQ5)

SeeDRec in Few-shot Setting The diffusion-based generalization exhibited by S2IDM serves to facilitate the exploration of associated interests for users with fewer behaviors or discovered interests. It motivates us to investigate SeeDRec's potential in few-shot scenarios. Hence, we set a sememe threshold for each dataset to simulate users that only have very few interests discovered by the real RS application and report the few-shot performance in Table 5.

We observe that: (1) Comparing with the full evaluation, SeeDRec achieves more significant improvements over SASRec on the few-shot setting. It verifies the feasibility of SeeDRec as a few-shot recommender thanks to the diffused interests. (2) Comparing the relative improvements across diverse datasets, it becomes evident that SeeDRec exhibits superior few-shot performance on Pixel, where the average number of sememes per item is comparatively lower. This may stem from the fact that SeeDRec elegantly addresses the twofold challenges of behavioral sparsity and preference unitary in this particular setting, thereby implying some ingenious usages of SeeDRec for new users (e.g., asking new users to pick several "seed" interests and generalize them via SeeDRec).

Case Study of User Interest Diffusion We visualize the original sememe distribution and the generalized interest distribution of S2IDM to illustrate how S2IDM affects the sememe-based interest diffusion. Fig. 5 shows several representative cases on Pixel and Toy. Take Fig. 5 (a) as an example, the user's historical preference singularly comprises one sememe as "Single-player Game". Following the successive inference process of S2IDM, it not only generalizes "Esport" and "Mobile Game", which are strongly linked to the core notion of "Single-player Game", but also captures some weaklyassociated yet widely-popular sememes like "Funny". This aligns with the conceptual rationale of S2IDM, that is, it can accurately mine each user's diffused interests by simultaneously considering the intricate interplay between the local interest evolution and the global interest generalization.

5 Conclusion and FutureWork

This paper proposes a Sememe-based Diffusion framework (SeeDRec), which captures each user's sememe-based diffusion-generalized interest distribution to enhance SR. With the proposed recommendation sememe powered by S2IDM and IPE, SeeDRec is verified to be effective, transferable, and scalable on thirteen SR and CDSR datasets. The proposed SeeDRec could also be easily adopted with different types of sequential models without much additional inference computation costs, which will be welcomed by the industry.

Future investigations should encompass the design of more cogent minimal interest units and the exploration of the semantic correlations among sememes during the diffusion process. Furthermore, we believe that SeeDRec indicates the future direction for the subsequent DM-based recommenders beyond the diffusion on indices. The proposed recommendation sememe and S2IDM can facilitate seamless integration into diverse scenarios to bring consistent improvements.

Acknowledgments

This work is supported in part by the National Key R&D Program of China (Grant no. 2021YFC3300203), the TaiShan Scholars Program (Grant no. tsqn202211289), the Shandong Province Excellent Young Scientists Fund Program (Overseas) (Grant no. 2022HWYQ-048), the Oversea Innovation Team Project of the "20 Regulations for New Universities" funding program of Jinan (Grant no. 2021GXRC073) and the Young Elite Scientists Sponsorship Program by CAST (2023QNRC001). ChatGPT and Grammarly are utilized to improve grammar and correct spelling. Corresponding Authors: Lei Meng and Ruobing Xie.