Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation

1 Introduction

In the era of information overload, Recommender Systems (RS) are essential for online services and are broadly utilized in a variety of areas such as online shopping, movies, music and news reading services. Two paradigms of recommender systems are most popular nowadays: general recommender and sequential recommender. General recommender aims to learn users' long-term preference which are presumed to be statistic or change slowly over time. Factorization-based collaborative filtering [Koren et al., 2009; Koren, 2008] methods are the best-known techniques in this area. However, the intent of user behaviors is inherently variable, which can be influenced by various factors such as interest evolution, instant demands and global mainstream fashion during a certain time period. Recently, the sequential recommender has attracted considerable attention due to its superiority in capturing itemto-item sequential relations, and this superiority meets the needs of exploring users' short-term preference. Motivated by the successful applications of recurrent neural networks (RNN) in other domains, especially in Natural Language Processing (NLP), RNN approaches have now become the mainstreammodels for sequential recommender [Wu et al., 2017; Okura et al., 2017; Beutel et al., 2018; Zhou et al., 2019].

∗Corresponding authors.

The vanilla RNN presumes an evenly distribution pattern between successive items in a sequence. For example, in NLP applications intervals between any two successive words in a sentence can be regarded as equal, and the whole sentence is organized for expressing the same (semantic) topic. However, a user's behavior sequence in the real-world is much more complex, and in this paper, we focus on two key problems: dynamic time intervals and dynamic latent intent. The first problem indicates the time interval between a user's successive actions can be varying. e.g., his/her next action may happen on the next day or in the next week. Intuitively, two actions within a short time interval tend to share a closer relationship than two actions that within a long time interval. Thus this kind of temporal distance deserve special handling. On the other side, the dynamic latent intent problem indicates a type of semantic distance. Customer intent, also known as the user's main purpose behind his/her behavior, is often changing from session to session. Irrelevant actions are useless for predicting a user's certain future action. For example, suppose a user's purchase history is [iPhone xs, airpods, yogurt X, cat food X], when we want to recommend him with laptops, only the first two actions, i.e. the iPhone xs and airpods, are strongly signals to infer that he/she may be interested in MacBook series. Some recent works partially discuss these two challenges [Zhu et al., 2017; Zhou et al., 2019], but none of them address both dynamic time interval and latent intent problems in a unified model.

Motivated by the aforementioned observations, we first propose a novel sequential recommender based on LSTM [Hochreiter and Schmidhuber, 1997], named TC-SeqRec, for better short-term user modeling. There are two key mechanisms in our TC-SeqRec model, corresponding to time-aware controller and content-aware controller. Specifically, to cope with the time-aware distance, we propose to utilize the gated mechanism in classical LSTM, and it is capable of controlling to what extent the information should be translated to future stages according to time context. To cope with the content-aware distance, we adapt attention mechanisms to dynamically filter out irrelevant actions in a user's behavior sequence, since attention mechanisms have achieved remarkable results in multiple domains such as recommender systems, question answering, machine translation, and image captioning. Via jointly training with both the time-aware and content-aware mechanisms, TC-SeqRec is more effective and robust for short-term user modeling.

Usually we do not merely rely on RNNs for sequential recommender systems. Although users' interest may be time-varying, there is still some static components influencing users' behaviors, which reflect their long-term preferences. That is why existing approaches tend to combine both short-term and long-term interaction [Rendle et al., 2010; Wu et al., 2017]. We find that a naive and static combination of these two aspects are insufficient. The aforementioned time-aware and content-aware observations can also be applied in this combination stage. We propose an attentionbased fusion method to adaptively incorporate the short- and long-term preferences under specific circumstances, such as when (if the next action occurs shortly after the last action, short-term information may play a major role in prediction; otherwise, we should shift more weight to the long-term component) and what (if recent actions share a distinct intent, then the next action may have a higher probability to share the same intent). We name the integrated model SLi-Rec, which is short for Short-term and Long-term preference Integrated RECommender system. We conduct extensive experiments on both a public dataset and our industrial dataset, and results demonstrate that our proposed model outperforms several state-of-art models significantly.

2 RelatedWork

2.1 General Recommender

A general recommender models users' long-term preference which reflects users' inherent characteristics and are static or changed slowly with time. Items recommended by this type of recommender are supposed to be interacted with by the user eventually without any consideration of the order. Factorization-based collaborative filtering methods [Koren et al., 2009; Koren, 2008] are one of the most effective techniques for this goal. Some popular topics under this direction include unifying collaborative and content-based filtering [Basilico and Hofmann, 2004], implicit feedback [Rendle et al., 2009], general user profiling [Zhong et al., 2015] and so on. In recent years, deep learning techniques have been widely and successfully applied in RS, for example, [He et al., 2017] proposes NCF to learn an arbitrary function from data by replacing the inner product with a neural architecture, [Cheng et al., 2016] jointly trains wide linear models and deep neural networks for combining memorization and generalization, [Lian et al., 2018] aims to learn effective higherorder feature interactions based on factorization machine.

2.2 Sequential Recommender

In contrast, a sequential recommender pays attention to the order of user behaviors. Some early work considers Markov chains to model sequential behavior by learning a transition graph over times that is used to predict the next action based on the recent actions of a user [Rendle et al., 2010]. Recently, recurrent neural networks (RNN) have demonstrated great power in sequential recommender, including in learning user/item evolution [Wu et al., 2017], building session-based RS [Hidasi et al., 2015; Wang et al., 2018] and next item/basket prediction [Li et al., 2017; Zhu et al., 2017]. [Liu et al., 2016] employs adaptive contextspecific input matrices and transition matrices to improve conventional RNN models. [Zhu et al., 2017] proposes several time gates to model time intervals with the goal of capturing both of users' long-term and short-term interests. The sequential recommender is not only popular in academia, but also popular in industrial systems [Okura et al., 2017; Beutel et al., 2018; Zhou et al., 2019]. In this paper, we leverage RNN structures for model users' short-term preference, and further propose an attention-based adaptive fusion schema to dynamically combine users' both short-term and long-term preference.

3 Our Model

Let U = {u1, u2, ..., un} denote a set of users and I = {i1, i2, ..., im} denote a set of items. A user's behavior history is represented by an ordered list: B(u) = {(iu1 , tu1 ), (iu2 , tu2 ), ..., (iu|B(u)| , tu|B(u)| )}, where (iu1 , tu1 ) means a user u interacted with an item iu1 at time tu1 , ti < tj for i < j, and |B(u)| denotes the number of actions in the user's behavior sequence. Our task belongs to the category of embedding-based user modeling, i.e., given B(u) we will generate a dense user vector pu = 〈p1, p2, ..., pd〉 which encodes the user's preference and can be used (together with other rich features) to predict which item he/she will interact at a future time tp, where p = |B(u)| + 1 . To make user vectors be more expressive, we will introduce a method to incorporate both long-term and short-term preference. In real industrial recommender systems, users' long-term preference can be learned based on a relatively longer time range and computed offline. On the other side, users' short-term preference can be inferred at online serving based on a relatively shorter range of recent behaviors to reduce computational cost. Next, we will introduce our proposed model in detail in three parts: short-term modeling, long-term modeling, and adaptive fusion stage.

3.1 Short-TermModeling

Due to their remarkable ability in sequential user modeling, RNNs have been attracting great attention recently in both academia [Zhu et al., 2017; Wu et al., 2017; Donkers et al., 2017] and industry [Beutel et al., 2018; Okura et al., 2017; Zhou et al., 2019]. The sequential updating process is very simple and can be formulated by:

Where g is the activation function, xk is the latest user action and hk-1 is the last hidden state. Among all RNN-based models, LSTM (long short-term memory) [Hochreiter and Schmidhuber, 1997] and GRU (gated recurrent unit) [Cho et al., 2014] are most commonly used for RS. On our own industrial dataset we observed that LSTM is slightly better than GRU, without loss of generality, in this paper we formulate the model with LSTM, whose equations are as follows:

where W∗, U∗ ∈ RD×D are trainable parameters, D indicates the dimension of input embedding and hidden layer in RNN (for notation simplicity, we presume the size of dimensions are equal). fk, ik, ok represent the forget, input, and output gates, respectively. ck represents the cell status, xk denotes the k-th item's input embedding, and ⊙ denotes the elementwise product. Usually, σ is the sigmoid function, and φ is the tanh function.

Different from words of a sentence in NLP domain, where items can be regarded as evenly spaced and semantically consistent, the sequence of user behaviors are much more complex. Here we mainly focus on these two challenges: (1) Time irregularity. Time intervals between two successive actions can be various. E.g., user a's purchase history is Ba = {(ia1 , Feb 1st ), (ia2 , Feb 2nd ), (ia3 ,Apr 2nd )}, it's more reasonable to transmit more information from ia1 to ia2 than from ia2 to ia3 , because item ia2 is purchased just one day after purchasing item ia1 , while item ia3 is purchased two months after last purchase. (2) Semantic irregularity. Items within a user's behavior sequence may not always share the same semantic topic (which we usually call customer intent). E.g., user a's purchase sequence may contains items {iphonexs, airpods, lawnmower,Bikini}, these items represent his/her different demands and only the first two items are strong signals for predicting his/her next activity for electronic products.

To tackle the problem of time irregularity, we modify the gating logic in LSTM to make it sensitive to time changes. We introduce two time-aware features, i.e., time interval feature δtk and time span feature stk , as follows:

where Wδ, Ws ∈ RD and Wtδ, Wts ∈ RD×D . The time span feature stk encodes the absolute temporal distance between current state tk and prediction state tp, while the time interval feature δtk encodes the relative temporal distance between two consecutive states. Similar to [Beutel et al., 2018], we add a fully connected layer to convert the time-aware features into dense vectors (δtk Wtδ and stk Wts), then compute time gates (Tδ and Ts) accordingly. Eq.(4) is now changed to:

and Eq.(5) is updated as:

To tackle the problem of semantic irregularity, we adopt attentive mechanisms to suppress the information that deviates from the target direction. Actually, attention mechanisms have been widely used to filter out irrelevant items or distinguish different levels of influence scores for relevant items [Wang et al., 2018; Ying et al., 2018; Zhou et al., 2019; Li et al., 2017]. An item's attention score is computed by:

where ep represents another embedding vector of item we want to make prediction, we call it prediction embedding in contrast to the input embedding vector xp. The attention score ak determines which item should be emphasized or neglected according to target items. We utilize these scores to adjust the cell and hidden states:

Instead of using the last hidden state as user representation, i.e., pushort = ˜ hk, we formulate user's short term representation as the weighted average of all the hidden states:

We call our proposed short-term model TC-SeqRec, which indicates that the model is a time- and content-aware sequential recommender.

3.2 Long-TermModeling

In this component we aim to model users' general preference that are inherent and supposed to be static or changed slowly. Matrix factorization techniques are the most successful methods for learning this long-term preference. Instead of providing an explicit parameterization for users, we adopt the attentive "Asymmetric-SVD" [Koren, 2008] paradigm, which represents users through the items that they interacted with:

similar to the Attentive Collaborative Filtering [Chen et al., 2017], alj is the weighting score for behavior j. We presume that not all behaviors contribute equally, thus it is meaningful to assign higher (lower) weights for the corresponding behaviors that are more informative (less informative). The weighting score is computed as:

where W lv ∈ RD×D and τl ∈ RD . Usually we can add a general user vector to the Asymmetric-SVD, i.e., plong u = p′u + ∑j∈B(u) alj xj. But to reduce parameters and prevent overfitting, we merely use Eq.(18) as user representation.

3.3 Adaptive Fusion Approach

Both short-term and long-term components have strengths and weaknesses. It is necessary to accommodate these two components. Instead of using a naive way to combine them, e.g., pfinal u = pshort u + plong u , we design an adaptive way for information fusion. The motivation is which component should play a more important role is determined by the specific context, such as when (if next action is taken shortly after the previous behaviors, then short-term preference may be more informative) and what (some categories of items such as mobile phone are better inferred from long-term preference, while some categories such as mobile accessories are better inferred from short-term information). Thus information fusion in a dynamic fashion is beneficial, and we propose the following attention-based adaptive fusion method:

where [pshort u , plong u , xcontext] represents a concatenation of short-term information, long-term information, and the contextual information. Note that for contextual information, we can include various kinds of important features according to what we have at hand, such as time interval, time stamp, location, and target item category. In our experiments we only include the item interval and the prediction embedding of the target item. We name the model SLi-Rec, which is short for Short-term and Long-term preference Integrated RECommender system.

A typical function for measuring user and item interaction is the doc product in the form of ˆ yui = 〈pfinal u , ei〉. A more flexible approach is to feed them into a multilayer perception (MLP). In this way, various additional features, such as user profiles and contextual features, can be easily incorporated for industrial recommender systems. To emphasize the impact of different user representations, in this paper we only concatenate the user vector and item vector as the input for a two-layer MLP, and all compared models in the experiment section will share this design, i.e., ˆ yui = MLP ([pf inal u , ei]).

Because our real-world industrial task is related to CTR (click-through rate) prediction, we formulate the recommendation task as a binary classification problem, where the negative log-likelihood function (log-loss) is usually used as the loss function:

where N is the total number of training instances. yi = 1 indicates a positive instance (the user has interacted with the item) and yi = 0 indicates a negative instance. The optimization process is to minimize the loss function together with a regularization term:

where Θ denotes the set of trainable parameters.

4 Experiments

4.1 Datasets

We use both a public dataset and our own industrial dataset for experimental analysis. The basic statistics of datasets (after filtering) are shown in Table 1.

Amazon dataset. [He andMcAuley, 2016]. This is a public dataset containing product reviews and metadata from Amazon. Nowadays it's widely used as a benchmark dataset in the RS domain. Reviews can reveal customers' shopping behaviors. We choose three subcategories, i.e., Electronics, Movies and TV, CDs and Vinyl, as well as a subset of the entire merged Amazon dataset for experiments. Item is represented by item id and category id. Given a user's previous T behaviors, we want to predict his T+1 behavior. For each user, his/her last behavior is hold out as the test set. For each positive instance we randomly sample one negative instance. 50% of the test set is used for hyperparameter tuning and the rest is used for reporting evaluation metrics.

Industrial dataset. Our real application scenario is native advertising 1 . We display personalized advertisements on MSN 2 homepage in a fashion that advertisements look like part of the editorial flow of the page. A user's behavior sequence is comprised of an ordered list of general browsing records. Thus the input embedding (i.e., xk) is derived from titles of browsing records while prediction embedding (i.e., ek) is from advertisement items. We extract the impression logs from 10 consecutive days in November 2018, and downsample the non-click logs to make the ratio of positive instances to negative instances be 1:5. The first 7 days are used as the training set, and the latter 3 days are used for the test set. Similar to the Amazon settings, 50% of the test set is used for hyperparameter tuning and the rest is used for reporting evaluation metrics.

1 https://en.wikipedia.org/wiki/Native advertising

2 https://www.msn.com/

4.2 ComparedMethods

We compare SLi-Rec with the following competitive models: ASVD [Koren, 2008] represents users by the items that they have interacted with. Items contribute equally. We further use Eq.(18) to get an attentive ASVD, or A2SVD for short.

DIN [Zhou et al., 2018] uses an attentive mechanism to dynamically activate related items in a user's behavior history according to the target item.

LSTMThe classical LSTMmodel for sequential prediction. NARM [Li et al., 2017] is a neural attentive recommendation machine, which captures a user's main purpose in the current session by incorporating an attention mechanism into RNN.

RRN [Wu et al., 2017] also leverages LSTM to capture dynamics in addition to a traditional low-rank factorization. The biggest difference between this method and the LSTM baseline is that, in RRN the user-item interactions are aggregated by a time step granularity. We have tried the granularity of daily and weekly, denoted by RRN-day and RRN-week.

LSTM++ combines the A2SVD and the LSTM baselines to capture both long-term and short-term preferences. We denote it as LSTM++ for convenience.

CA-RNN [Liu et al., 2016] is the context-aware recurrent neural networks which employs adaptive context-specific input matrices and transition matrices in the RNN framework.

T-LSTM [Zhu et al., 2017] equips LSTM with time gates to model time intervals. Different from our model, it doesn't handle time span and relies on the time gates to capture both long-term and short-term interests.

DIEN [Zhou et al., 2019] is the most recent related work which models the user's sequential behaviors with two layers of GRU. Its key components consist of interest extracting layer and interest evolving layer. The long-term component in this baseline is ASVD.

We implement the models with Tensorflow, the source code is available at https://github.com/zepingyu0512/sli rec. Optimizer is Adam. Dimension for item/category embedding and RNN hidden layers is 18, while the dimension for MLP is 36. We adopt grid search to find the best hyperparameters for each model with validation set. Optimal settings for our model are: learning rate is 0.001; L2 regularizataion is 0.0001; no dropouts; batch normalization is used only after the concatenation of the user's (final) embedding and item embedding; activation function of MLP is Dice [Zhou et al., 2018] for Amazon and ReLU for Ads dataset; maximum length for user behaviors is set to 100.

Evaluation metrics. In Section 3.3 we formulate the recommendation task as a binary classification problem, thus we use AUC and F1-score as the evaluation metrics. AUC measures the probability that a positive instance will be ranked higher than a randomly chosen negative one. F1-score is the harmonic mean of precision and recall. These metrics summarize a model's performance from different aspects.

4.3 Results

Comparison to baselines. Tables 2 and 3 show the overall performance of different models in terms of AUC and F1-score respectively. ASVD, A2SVD and DIN are models without sequential mechanism, while the other models have a sequential component. A2SVD and DIN outperform ASVD, which demonstrates that assigning importance score to items are beneficial. Containing both short-term and long-term interest is meaningful for recommender systems, and this can easily be verified via comparing LSTM++ with LSTM. NARM is original designed for capturing the user's main purpose in the current session, which is different from our application scenario, thus its performance is not very outstanding in our cases. CARNN, T-LSTM, and DIEN can outperform LSTM and RRN in most cases, which directly verify that considering contextual information in sequence is helpful. At last, our proposed SLi-Rec significantly outperform all the baselines on five datasets across the three evaluation metrics. The superiority of SLi-Rec depends on three components, i.e., the short-term part, the long-term part, and the adaptive fusing part. Next we will investigate the effectiveness of these components separately.

Model variants for short-term component. To verify whether our design of time-aware and content-aware controllers are necessary and effective for short-term preference modeling, we study several variants. Table 4 shows the results. All the other models are better than LSTM, which demonstrates that consider complex user behavior patterns is indeed meaningful. T-SeqRec is our short-term model which only enable the time-aware controller. Compared with TLSTM, T-SeqRec not only uses time interval, but also uses time span feature. TC-SeqRec∗ are different approaches to enable the content-aware controller, where i means the item attention score (Eq.(13)) is applied to input features instead of to cell and hidden states, g means we use the last state as short-term preference instead of averaging all the hidden states (Eq.(17)). Via comparing with these various variants, we can observe that both the time-aware and content-aware controllers are beneficial to short-term user modeling.

Effectiveness of adaptive fusion. Table 5 demonstrates various choices of α in Eq.(22). α = 0 and α = 1 indicate only using a long-term or short-term component, respectively. α = fixed means we empirically search a fixed optimal value for each dataset (similar to an end-2-end ensemble of two components). We observe that the short-term component is always significantly better than the long-term component, thus sometimes a naive combination of these two components turns out to be even worse than a mere short-term component. An adaptive fusion mechanism is necessary and effective. To get an intuitive sense of how α would change with context, we split test instances according to the time interval of a user's last behavior. For instance, 1 hour means than the behavior which we want make prediction occurs within one hour after the user's last behavior. Figure 1 shows the results. There is a clear trend that importance for long-term component increases with the time interval, which matches our assumption.

Attention in long-term component. To verify whether Eq.(18) can learn discriminative importance scores for different items, we output item's weighting values in Eq.(20)

on the entire dataset. For better illustration, for each category we at most sample 1000 items, and ignore categories that contain fewer than 100 items. We sort items by their normalized importance score and plot Figure 2a. For each category, we count the mean and variance of all items' scores under this category and draw Figure 2b (the scores are scaled by 1e-5). We can observe that items are assigned with discriminative scores, and few items (less than 2%) are assign a normalized score greater than 0.5.

5 Conclusion

In this paper, we have proposed the novel SLi-Rec model for integrating both the short-term and long-term preference for better user modeling. We observe that users' behavior sequences are much more complex and challenging than sequences in other application domains (such as sentences in NLP), thus we propose the time-aware and content-aware controllers to make the classical LSTMmore suitable for user behavior modeling. We further propose an attention-based fusion method to adaptively combine the long-term and shortterm preference according to the specific context. We have conduct extensive experiments on both public dataset and industrial dataset, experiments demonstrate that our proposed model outperforms state-of-the-art methods consistently.

Acknowledgments

We gratefully thank the Microsoft Audience Network (MSAN) team, especially Yajun Wang and Mehul Parsana for their contributions and support for experiments and applications in native advertisement.