Collaborative Multi-Level Embedding Learning from Reviews for Rating Prediction

1 Introduction

Personalized rating prediction is a fundamental problem for recommender system and has attracted a lot of attention since Netflix Prize Challenge [Bell and Koren, 2007] was successfully held. This problem aims at leveraging historical ratings to predict users' ratings (e.g. integers from 1 up to 5) for the items they have not rated before. Among various methods for this problem, latent factor models such as matrix factorization perform better [Koren et al., 2009].

As an extension of the above classical problem, personalized review-based rating prediction is newly formulated [McAuley and Leskovec, 2013] which gains extra knowledge from review text for predicting rating scores. The problem is significant as reviews can reveal the preference of users and indicate the characteristics of items, which turn out to be important references for making buying decisions. Besides, abundant of reviews can be easily acquired since they are widely available in electronic commerce companies (e.g. Amazon), video sharing websites (e.g. Youtube), and community websites (e.g. Yelp). It may enable us to achieve better prediction performance and obtain interpretable learned user and item factors.

Different from traditional sentiment analysis tasks, personalized review-based rating prediction is to predict ratings for user and item pairs when review text information only exists in historical data and is not available in prediction stage. Consequently, many sentiment analysis techniques over text cannot be exploited [Liu, 2012; Pang and Lee, 2005]. To enable effective utilization of textual content in historical data, the main challenges are how to learn useful knowledge from review text and how to adapt the learned knowledge to user and item factors to generate more accurate ratings.

Limitations of prior studies. Existing techniques [McAuley and Leskovec, 2013; Bao et al., 2014; Diao et al., 2014; Hu et al., 2015] for this problem are often based on latent Dirichlet allocation (LDA) [Blei et al., 2003] or non-negative matrix factorization (NMF) [Lee and Seung, 2000] to obtain topic factors of review text and then integrate them with matrix factorization model. However, almost all these methods suffer from at least one of the following two limitations: ignorance of word local context information and direct equivalence between user, item, and review topic factors. First, for only using document-level co-occurrence information to model words in review text, LDA and NMF will lose word order and local context information, which is a prominent issue for sentiment analysis [Wang and Manning, 2012; Johnson and Zhang, 2015]. Second, review topic factor is directly equaled to the normalized exponential transformation of user or item factor, which is not very suitable. It is intuitive because users and items cannot be fully represented by reviews due to their limited length. And not all words and topics in each review are relevant to its rating score. Direct equivalence may bring irrelevant information for rating prediction to some dimensions of embeddings.

Our solution. To address the above two limitations, we propose a novel Collaborative Multi-Level Embedding (CMLE)

model for the personalized review-based rating problem. From a whole perspective, it naturally integrates word embedding model [Mikolov et al., 2013b; Le and Mikolov, 2014] with bias matrix factorization [Koren et al., 2009] in a unified framework and consists of three levels (see Figure 2). In the top level, CMLE captures word local context information by word and review embedding learning, which addresses the first limitation. In the middle level, an additional projection level is proposed to project review embedding to user and item embeddings, which tightly couples them without the restricted requirement of equivalence and thus can cope with the second limitation. In the bottom level, MF is leveraged to generate ratings based on the projected user and item embeddings. In particular, we formulate a joint optimization problem to simultaneously learn word, review, user and item embeddings. An efficient stochastic gradient ascent algorithm is adopted to ensure efficiency. Without specification, embedding and factor denote the same concept in this paper.

Contributions. To sum up, the major contributions of this paper are presented as follows:

• ÷ We emphasize the two limitations existed in the previous methods for personalized review-based rating prediction problem.
• ÷ A novel model called CMLE is proposed to cope with these limitations and a joint optimization problem is formulated to learn all embeddings. To the best of our knowledge, CMLE is the first model towards integration of word embedding learning with standard latent factor model for personalized rating prediction problem.
• ÷ Experimental results on real datasets demonstrate that CMLE outperforms several previous methods for this problem and verify its benefits from overcoming the two limitations.

2 Preliminaries

2.1 ProblemFormulation

Let U and V denote user and item sets, respectively. In accordance with existing studies on personalized rating prediction, we adopt R to represent the rating scores users assigned to items. Specifically, for 8u 2 U and 8v 2 V, if u has rated v, then ruv denotes the corresponding rating score. Otherwise, it is missing and needed to be predicted.

For the new problem we studied, review textual content is considered besides rating scores. Assume D is a collection of review text, then each review duv (duv 2 D), written by user u for item v, consists of a series of words from a vocabulary W and is associated with a rating score ruv. Based on the above formulation, we can define the problem as follows,

Problem (Personalized Review-based Rating Prediction)

Given a user set U, an item set V, and a historical review text collection D accompanied with known entries (r 2 ˜R) in a rating matrix R, the target is to predict the scores of the missing entries (r 2 R ∩ ˜R) in the rating matrix.

2.2 Motivations

As we mentioned, although integrated topic and latent factor models are capable of utilizing reviews for rating prediction, they suffer from two limitations. Inspired by the idea, we emphasize two intuitions which motivate this work.

Intuition 1 Word local context information plays an important role for rating score determination.

Word local context information refers to the surrounding text for each word. This is a very natural intuition about how words in reviews determine rating scores. Consider the example shown in Figure 1. Generally speaking, the word "good" is a positive sentiment word, no matter which aspects of items it describes. However, the negative word "not" will change its sentiment polarity. As we know, traditional topic models only consider word co-occurrence in document level and regard all words in each document equally. Thus, local relation such as "no" for "good" cannot be captured. Another example is the word "high" shown in the review. The sentiment polarity of "high" depends on the aspect it describes. In this scenario, "high price" obviously means negative sentiment. However, if "high" is employed to modify "quality", then it conveys positive sentiment. Overall, if word local context information is not considered, then sentiment polarity may not be analyzed accurately. Inspired by the first intuition, we adopt the recently proposed embedding model (see first level of Figure 2) which naturally exploits word local context information by constructing embedding for each word based on its context words and review. Hence the proposed model can consider word local context information.

Intuition 2 Direct equivalence between user, item, and review embeddings is a little too restricted.

For user u and item v, we define wu 2 RK and wv 2 RK as their embeddings, where K means the dimension of embedding. duv represents the review written by user u to item v and its embedding is denoted as wduv . Direct equivalence means associating review embedding with user and item embeddings with normalized exponential transformation (e.g. wduv ⇠ exp(wu + wv)). The above intuition can be simply explained from two aspects. First, some words or segments in a review may be irrelevant to the final sentiment polarity. For example, the italic segment in Figure 1 just plays a transitional role in the review and has nothing to do with the sentiment. In other words, not each topic shown in the review text contributes to the rating. Second, the average length of reviews is limited [Zhang and Wang, 2015]. Hence sparse review textual content may not fully reveal the characteristics of users and items. In summary, direct equivalence [McAuley and Leskovec, 2013; Bao et al., 2014] between these embeddings still have potential to improve. Inspired by this intuition, we propose a projection layer (see second level of Figure 2) to connect review representation to user and item representations, which relaxes the restriction of direct equivalence.

2.3 RelatedModels

We briefly introduce two most related models here, i.e., word embedding model and matrix factorization, which serve as two major components of CMLE.

Word Embedding Model.

Word embedding models are successful in many research fields, especially natural language processing. Two representative models, continuous bag-of-words (CBOW) and skip-gram model [Mikolov et al., 2013a], are widely applied. The major difference between these two models is that CBOW leverages local context words (preceding and succeeding words) to predict its current word, while in skip-gram model, local context words are predicted by its current word.

More precisely, suppose Nduv refers to the length of the review duv. For CBOW, its learning objective is to maximize the following log-probability,

where wk+c k-c represents a sequence of words surrounding wk with the specified context length c which is commonly set to be 5. CBOW further defines the probability p(wk♣wk+c k-c ) as,

where ewk (ewk 2 RK) and ˆewk represent the input and output embedding of word wk, respectively. It is worth noting that in this paper, we use subscripts of variables in probability expressions, like p(wk♣wk+c k-c ), and specify probability computational formulas with corresponding variables, such as those shown in Equation 2.

Skip-gram model maximizes a different log-probability objective function, formulated as below,

It is clear that the difference lies in the probability p(wj♣wk) which is defined as follows,

Matrix Factorization.

Matrix factorization [Koren et al., 2009] is a state-of-the-art collaborative filtering model for recommender system. Typically, rating prediction ˆ ruv of user u to item v can be computed as below,

where bu and bv correspond to their rating biases. And g is a global rating bias.

3 Computational Model

3.1 Model Description

To consider the two intuitions in an unified view, we propose CMLE which constructs review embedding with word embedding and further learns user and item embeddings based on review embedding and ratings. Figure 2 (rating bias terms omitted) illustrates the main idea of our model. CMLE basically consists of three levels. In the top level, it leverages the CBOW method to model each word of review text while incorporating review-level embedding. Further, review, user, and item embeddings are associated through a projection similar to skip-grammethod in the middle level. Finally, the bottom level is devoted to calculating ratings through matrix factorization. In what follows, we will formalize the model level by level detailedly.

In the top level, inspired by the idea of PV-DM [Le and Mikolov, 2014], we extend the CBOW method to obtain review embedding which will later be associated with user and item embeddings. Therefore, embedding of each word in reviews depends on both its surrounding context word and review embeddings. Assume wk belongs to review duv, then the probability of generating the word can be defined as the following softmax function,

where ¯vwk can be calculated as below,

where wduv is the embedding of review duv which summarizes the review and can be learned automatically. Because the model learning process involves simultaneously updating all the embeddings, each review embedding will be influenced by word local context information revealed by word embedding through Equation 6. As a result, review embedding can partially capture local context information, which satisfies the first intuition.

The proposed middle level with projection operation is similar to the one adopted in the skip-gram method, but their roles are fundamentally different. Through this level, CMLE can seamlessly associate review embedding with user and item embeddings. It is obvious that review embedding bridges between word embedding and user, item embedding. This is reasonable since user and item both influence construction of review text [Zhang and Wang, 2015]. To the best of our knowledge, this is the first work towards integrating word embedding learning with latent factor model for personalized rating prediction. More specifically, we define the probability of user and item embeddings based on review embedding as below,

Compared with the formulas used in [McAuley and Leskovec, 2013; Bao et al., 2014], which make user or item factor equivalent to review topic distribution through a direct normalized exponential transformation, CMLE adopts inner product of these embeddings and thus can relax the equivalence restriction.

The bottom level concentrates on the target of rating prediction. It adopts standard matrix factorization shown in Equation 5. In combination with the middle and bottom levels, we can observe that user and item embeddings capture knowledge not only from rating behaviors, but also review text information. After learning, given a new user-item pair, we can predict the rating by its user and item embeddings.

Optimization Target.

Now based on previous formulations, we derive the optimization target for later model learning. Actually, each level of CMLE has its own optimization target. For the top level, we define the target as,

where p(wk♣wk+c k-c ↪ duv) can be calculated by Equation 6. Similarly, we obtain the following log-probability for the middle level,

To keep consistency with the above two targets, the objective of rating prediction in the bottom level is formulated as logprobability of Gaussian distribution as well,

where σ2 is set to be one for simplicity and ˆ ruv can be computed through Equation 5. To avoid overfitting issue, Gaussian distributions N (wu↪ λU ) and N (wv ↪ λV ) are usually incorporated into the above formula to play a regularization role. The precisions of Gaussian priors, λU and λV , can be regarded as hyper-parameters of regularization terms.

In summary, we can construct the final optimization target by linearly combine the above three objectives together. As learning user and item embeddings from review text is auxiliary to the main goal of predicting ratings, we utilize a weighting factor ↵ (↵ 2 [0↪ 1]) for the middle objective [Toutanova et al., 2015]. It controls the relative influence of reviews and ratings on user and item embeddings. We can determine ↵ based on the performance on validation datasets. Finally, the whole objective function can be written as,

3.2 Model Learning

We employ stochastic gradient ascent algorithm (SGA) to optimize the function shown in Equation 12, which is efficient and suitable for large-scale learning problem. For CMLE, the model parameters can be expressed as ⇥ = ¶e1:♣W♣↪ ˆe1:♣W♣↪w1:♣D♣↪w1:♣U♣↪w1:♣V♣↪ b1:♣U♣↪ b1:♣V♣↪ g♦. Then all model parameters can be updated based on their gradients, i.e., ⇥t+1 = ⇥t + ⌘ @L @⇥ , where ⌘ is the learning rate and t is the current iteration number. Now the key step is to derive all gradients of the parameters. Given a training instance (u↪ v↪ duv↪ ruv), the key gradients are calculated as below,

where ¯vwk and ˆ ruv can be calculated through Equation 7 and Equation 5, respectively. For space limitation, we omit the gradients of the bias terms which are easily derived. Due to the normalization terms in Equations 6 and 8, computing these related gradients suffers a heavy cost. To speed up the learning process, we employ noise contrastive estimation (NCE) [Mnih and Teh, 2012] to approximate the softmax functions by sampling several negative words, users, and items. In this paper, we set the number of negative samples to be five and the sampling strategy is used the same as [Mikolov et al., 2013b]. The above learning algorithm guarantees the cost time of each iteration grows linearly with the number of reviews and makes CMLE scalable to large datasets.

4 Experiments

4.1 Evaluation Setting

Dataset.

We conduct experiments on several real datasets which are publicly available [McAuley and Leskovec, 2013]. Based on their data sources, we call them Food, Video, and Beer. We first convert all words of the reviews into lowercase and then remove the reviews which are too short. The basic statistics of the three datasets are: Food has 1635 users, 510 items, and 34431 reviews; Video has 2674 users, 2791 items, and 135765 reviews; Beer has 8848 users, 11190 items, and 1343985 reviews. For later comparisons, we randomly split the three datasets into train, validation, and test sets with the ratio of 7 to 1 to 2, respectively. Meanwhile, we remove users and items with less than ten occurrences in training data to ensure that users and items are associated with enough reviews. We repeat the above process five times and report the average results. We test the prediction performance in terms of two standard metrics, i.e., mean square error (MSE) and mean absolute error (MAE).

Comparison Methods.

To demonstrate the superiority of CMLE, we consider the following comparisons:

GloAve: This method simply computes the mean of historical rating scores and then uses the mean as predictions.

ItemKNN: ItemKNN [Sarwar et al., 2001] is a basic item based collaborative filtering method for rating prediction.

PMF: Probabilistic matrix factorization [Salakhutdinov and Mnih, 2007] formulates matrix factorization from a probabilistic perspective without considering rating biases.

HFT: This method [McAuley and Leskovec, 2013] first proposes to utilize reviews to learn interpretable user or item representation for personalized review-based rating prediction problem, which inspires many studies later.

TopicMF: TopicMF [Bao et al., 2014] is an extension of HFT which associates users and items with their corresponding reviews simultaneously under a non-negative matrix factorization framework.

JMARS: This complex model [Diao et al., 2014] distinguishes aspect, sentiment, and background words in an unified framework. Its input depends on Part-of-Speech tagging results and thus we employ Stanford log-linear POS tagger [Toutanova et al., 2003] here.

BMF: BMF [Koren et al., 2009] is a state-of-the-art method for personalized rating prediction. Its results equal to CMLE when ↵ = 0 as it makes the gradients of users and items in CMLE are only related to ratings according to Equation 3.2.

To verify the rationality the first intuition, we design the following strategies which are used in the first level of CMLE: RandomD: This strategy initializes review embedding randomly and keeps it fixed during the learning process of CMLE. Hence user and item embeddings cannot gain useful knowledge from review embedding.

RandomW: It only initializes word embedding randomly and keeps it fixed during the learning process, which means the model learns review embedding purely from rating behaviors.

LDA: This strategy initializes review embedding with its corresponding review topic distribution learned from latent Dirichlet allocation [Blei et al., 2003]. As LDA ignores word local context information, it can be used as a direct comparison with our adopted strategy (Doc2Vec).

The following equivalence strategy is used in the middle level to indicate the rationality of the second intuition:

Equivalence: We adopt the same strategy used in [McAuley and Leskovec, 2013; Bao et al., 2014] to connect review embedding to user and item embeddings. This is regarded as a comparison with the strategy (Projection) we proposed.

Hyper-parameter Setting.

We tune the hyper-parameters of all methods based on their performance on validation datasets to ensure fair comparisons. For CMLE, we initialize the learning rate ⌘ = 0▷2, regularization hyper-parameters to be 0.1 (same for other factor based methods such as BMF), and the relative weight ↵ = 0▷1. All experiments are conducted with embedding dimension K = 40. We also try other settings and find that 40 already ensures to reach stable results.

4.2 Results Analysis

Performance Comparisons.

Table 1 shows the results of CMLE and its comparisons. Overall, all the methods perform better on Food and Beer than Video. This may be explained by the fact that users' rating behaviors in video and beer domain are more diverse than that in food domain. Due to the lack of considering both personalization and review text information, GloAve performs worst among all the methods. BMF outperforms PMF consistently, which indicates the necessity of incorporating user and item rating bias for prediction. As jointly associating user and item factors with review topic, TopicMF obtains better results than HFT and its performance is roughly the same as the more complex JMARS method. Although HFT and TopicMF can provide interpretability for the learned factors, their prediction results are not as effective as BMF, which reveals their performance may be influenced by direct equivalence and is still far from satisfactory. Finally, CMLE achieves the best results among the adopted methods and the improvements are significant under t-test, which demonstrates the superiority of our model design. Besides, as the embeddings of word, user, and item lie in the same semantic space, the learned user and item embedding can be easily interpreted by the top nearest words to them (e.g calculation through inner product). Due to space limitation, the subsequent experiments do not show the results on the Beer dataset from which we can get the same conclusions as those on the other two datasets.

Effects of Considering Local Context Information.

Figure 3 depicts the variation of results with the different settings of embeddings. It is sensible that RandomD performs worst due to its random review embedding bringing no effective information. RandomW gains a minor improvement over RandomD because review embedding can be learned in the optimization process. Using LDA, review embedding can obtain valid knowledge from reviews and transfer them to user and item embeddings with the continual learning process. Therefore this strategy behaves significantly better than the previous two strategies. CMLE further considers word local context information to obtain more reasonable review embedding for rating prediction and thus it achieves the best results. This indicates the rationality of the first intuition.

Results of Projection versus Direct Equivalence.

We further compare the ways of associating review, user, and item embeddings. It is obvious that using the projection layer in CMLE significantly outperforms the direct equivalence strategy adopted in previous methods. This reveals direct equivalence between these embedding through a normalized exponential transformation is a little restricted. Overall, we can conclude the second intuition is reasonable.

Influence of Relative Weight ↵.

Finally, we study how ↵ influences the performance of rating prediction. From Figure 5, we observe the results are optimal when ↵ ranges from 0▷1 to 0▷2 and significantly better than those when ↵ = 0, which verifies the rationality of considering review text.

5 RelatedWork

Recent years have witnessed growing interest of combining personalization factor with review text information for personalized rating prediction. Personalized review-based rating prediction and personalized review-aware rating prediction are two representative problems. They differ in whether knowing review text information when prediction.

As aforementioned, many recent studies [McAuley and Leskovec, 2013; Bao et al., 2014; Diao et al., 2014; Zhang et al., 2014; Hu et al., 2015] integrate variants of topic modeling with matrix factorization for the personalized reviewbased rating prediction problem. [McAuley and Leskovec, 2013] first connected review topic distribution to user or item latent factor and inspired many following studies. [Bao et al., 2014] further associated user and item with review text simultaneously. Furthermore, [Hu et al., 2015] concluded incorporating social relations can improve prediction performance, which is analogous to our work. As we discussed, these studies suffer from the two limitations which hinders them from getting better performance. [Diao et al., 2014] tried to infer aspect concentrations of users and items. Their model is a little complex and suffers from the ignorance of word local context information. [Zhang et al., 2014] could also capture word local context information by first extracting aspect and sentiment words and then constructing explicit factor model. Nevertheless, their results may be influenced by the noise originated from the preprocessing step. Besides, their model is hard to incorporate user and item rating biases which are important for rating prediction.

There are fewer studies for personalized review-aware rating prediction problem. [Li et al., 2014] first proposed a tensor topic model to predict the ratings for each triple of user, item, and review. [Tang et al., 2015] recently formulated a deep neural network model to incorporate user and item information. As these methods require review text information known when prediction, they cannot be easily adapted to the problem we studied.

6 Conclusion and FutureWork

In this paper, we have studied the problem of personalized review-based rating prediction. We emphasize the two limitations of related methods, i.e., ignorance of word local context information and direct equivalence between user, item, and review embeddings. To cope with these limitations, we propose a novel model called CMLE. To the best of our knowledge, it is the first study to integrate the word embedding model with matrix factorization for personalized rating prediction. The experimental results on real datasets demonstrate CMLE is effective and can solve the two limitations well.

For future work, several interesting directions can be explored. First, utilizing phrase embedding [Yu and Dredze, 2015] based on mined phrases [Liu et al., 2015] instead of word embedding may be beneficial. Second, recent deep learning approaches [Socher et al., 2013] with more ability of modeling text may be applied to the studied problem.

Acknowledgments

We thank the anonymous reviewers for their valuable and constructive comments. This work was supported in part by National Basic Research Program of China (973 Program) under Grant No. 2014CB340505, National Natural Science Foundation of China under Grant No. 61532010 and 61272088, Tsinghua University Initiative Scientific Research Program under Grant No.20131089256, the U.S. Army Research Lab. under Cooperative Agreement No. W911NF-092-0053 (NSCTA), National Science Foundation IIS-1017362, IIS-1320617, and IIS-1354329, HDTRA1-10-1-0120, and grant 1U54GM114838 awarded by NIGMS through funds provided by the trans-NIH Big Data to Knowledge (BD2K) initiative (www.bd2k.nih.gov). Besides, Wei Zhang is supported by China Scholarship Council.