DARec: Deep Domain Adaptation for Cross-Domain Recommendation via Transferring Rating Patterns

1 Introduction

Recommender systems have long been beset by a host of well-known issues, e.g., the cold-start and sparsity problems. As one solution, cross-domain recommendation (CDR) leverages the information from several different source domains to build a better recommendation model for the target domain. In real-world datasets, although the number of users and items are normally very large with limited available feedbacks, the users/items can be normally categorized into domains according to certain criteria, making CDR a promising technique. CDR has been studied from many perspectives in different research areas [Khan et al., 2017]. Recommendation scenarios can be divided into four classes: 1. No User-No Item overlap (NU-NI); 2. User-No Item overlap (UNI); 3. No User-Item overlap (NU-I); 4. User-Item overlap (U-I). In this work, we focus on single-domain recommendation task under the U-NI scenario. In other words, we model rating patterns in both source and target domains to enhance the recommendations in the target domain.

Under our research settings, multiple approaches have been proposed. Some are based on clustering [Li et al., 2009], and others use variations of matrix factorization (MF) [Singh and Gordon, 2008]. Recent works apply deep learning (DL) techniques to perform the knowledge transfer [Elkahky et al.,

2015; Lian et al., 2017]. Most clustering and MF-based methods fail to model the nonlinear patterns in the ratings and normally require a dense rating matrix for the source domain. DL methods show superior performance to the above ones, but most of them learn and transfer knowledge from auxiliary contents. Directly extracting and transferring patterns from the sparse rating matrix are rarely studied. As DL has been reported to have state-of-the-art performance in rating prediction and top-N recommendation tasks by only using rating matrices [He et al., 2017; He et al., 2018], we propose to leverage deep neural networks (DNN) to learn the transferable rating patterns.

Domain adaptation (DA) is a technique that allows knowledge from a source domain to be transferred to a different but related target domain. DA is widely employed in semisupervised learning, where the target domain are not labelled or have only a few labels. DA aims to learn a classifier or predictor in presence of a distribution shift between the source and target domain, and has been shown to have state-of-theart performance in various computer vision and natural language processing tasks [Bousmalis et al., 2016].

Typical recommendation problems do not exactly match the scenarios of DA. However, as a fundamental question in cross-domain recommendations, how to model the datadependent effect of the ratings when transferring the dataindependent knowledge needs to be addressed. As the source and target data may be with different distributions, inspired by the domain adversarial neural networks (DANN) [Ganin and Lempitsky, 2015], we propose a deep Domain Adaptation Recommendation (DARec) model. It contains a rating pattern extractor, a domain classifier and a rating predictor. It can learn abstract representations of shared patterns and transfer them between two domains via DNN. Furthermore, we apply a weighted loss function to balance the significance of the source and target domains to control the transfer direction. Due to the sparsity in the rating matrices, we first apply AutoRec [Sedhain et al., 2015] to learn a dense embedding for each user to represent the user's preferences, after which the embeddings are fed into the following DNN. With DARec, the rating estimation accuracy in the target domain can be improved. We further compare DARec with several state-of-the-art CDR methods on public datasets, showing that DARec achieves best recommendation results. Our major contributions are as follows:

• We propose a deep domain adaptation cross-domain recommendation model (DARec) to extract and transfer abstract rating patterns, using only the information from rating matrices. Utilizing a domian classifier, shared rating patterns of the same user in different domains are learned and transferred via adversarial training, leading to U-DARec model, while distinctive rating patterns of unrelated items in different domains are separated, resulting in the I-DARec model;
• Using serveral public datasets, we empirically investigate the capability of our approach to improve on the recommendation performance in single-domain, which confirms the effectiveness of DARec to transfer knowledge between domains. We compare our model with some state-of-the-art single-domain rating prediction methods and demonstrate that benefiting from the transferred information from the source domain, DARec is superior to all the selected baselines;
• We select several alternative cross-domain recommendation methods that can fit in our research settings, including MF-based models and recently-proposed DLbased ones. On the public datasets, we demonstrate that due to DL, DARec is superior to all the selected MF models, and I-DARec exceeds the performance of stateof-the-art DL-based methods.

2 RelatedWork

Generally, CDR can be categorized as collaborative filtering-based CDR (CFCDR) methods and content-based CDR (CBCDR) methods.Early CFCDR approaches use neighborhood-based solutions [Berkovsky et al., 2007], but these methods are soon surpassed by MF-based solutions. Collective Matrix Factorization (CMF) [Singh and Gordon, 2008] couples target rating matrix and all auxiliary matrices on User dimension to share the user factor matrix across all domains. Cross-domain Triadic Factorization (CDTF) [Hu et al., 2013] learns the triadic factors for user, item and domain through tensor factorization, which can better capture the interactions between domain-specific user factors and item factors. Cluster-level Matrix Factorization [Mirbakhsh and Ling, 2015] employs SVD and K-means to build a cross-domain cluster-level coarse matrix that captures shared patterns between the cluster of users and the cluster of items among domains. Most of these models have three major shortcomes: 1. most of them assume the auxiliary data is relatively dense for all users or items; 2. user/item overlaps are normally required; 3. the linear MF methods are not powerful enough to extract nonlinear patterns .

To overcome such limitations, CBCDR techniques establish domain links by leveraging the auxiliary information about users/items without requiring user/item overlaps or relatively dense rating matrices. Moreover, DL is ready to be applied on auxiliary data to better represent users/items. [Elkahky et al., 2015] is a multi-view framework that extends the deep structured semantic model which maps highdimensional features into low-dimensional dense features in a joint semantic space. [Lian et al., 2017] formulates a multiview neural network learning framework that combines MFextracted features with user/item features. [Kanagawa et al., 2018] proposes a DA-based model, where stacked denoising autoencoders are used to extract item features, and a domain separation network is responsible for recommendation. The above methods all rely on auxiliary contents to learn shared patterns while emphasize less on the patterns in the rating matrices. Recently, a work named collaborative cross networks (CoNet) [Hu, 2018] employs deep cross connection neural networks to learn and transfer shared rating patterns among domains without using any auxiliary contents.

Unlike all the previous works, our approach is different in four perspectives:1. we leverage DNN to learn linear and nonlinear shared rating patterns from rating matrices, which is more powerful than MF- or clustering-based methods; 2. no dense data from the source domain is required for extracting transferable knowledge, as we rely on DNN to learn such knowledge from the raw sparse rating matrices; 3. we do not need any auxiliary user/item features; 4. we combine DA in our approach so that the possible distribution mismatch between two domains can be alleviated, which is more effective than simply using fully-connected neural networks to model the difference between two domains as CoNet .

3 ProposedMethod

3.1 ProblemDefinition

In this paper, we assume that the input data takes the form of explicit feedbacks such as user ratings of items. We also assume that the source and target domains have the same set of users (i.e. the U-NI scenario), denoted by U = {1, 2, ...,U}. The item sets from the source and target domains are I S = {1, 2, ..., I S } and I T = {1, 2, ..., I T } respectively, with the corresponding rating matrices given by Y S = {yS ui |u ∈ U, i ∈ I S } and Y T = {yT ui |u ∈ U, i ∈ I T }. We denote the set of observed item ratings given by u as I S u (I T u ), and the unobserved ones as ¯ I S u (¯ I T u ) for source(target) domain. In each domain, the goal of recommendation can be specified as selecting a subset of items from ¯ I S u (¯ I T u ) for user u according to certain criteria that maximizes the user's satisfaction. In other words, the recommendation algorithms aim to give predictions on the unknown ratings of each user, i.e., ˆ yS ui (u ∈ U, i ∈ ¯ I S u ) or ˆ yT ui (u ∈ U, i ∈ ¯ I T u ). The values of yS ui (yT ui ) are numerical ratings in a certain range, e.g. [1, 5]. The recommendation task we consider here is the single-domain problem [Khan et al., 2017], where we predict ˆ yT ui (u ∈ U, i ∈ ¯ I T u ), via leveraging the information from the source rating matrix Y S . We use the widely-adopted metrics to measure the performance of all approaches used in this paper, such as RMSE defined by √1/(MN) ∑Mu=1 ∑Ni=1 (ˆ yT ui - yT ui )2 , where M,N denote the number of users and items in the testing set respectively.

3.2 Deep Domain Adaptation for CDR

Figure 1 shows the proposed model to integrate DA for CDR. When the source and target domains share the same set of users, but different items, we use AutoRec to learn a set of embeddings to represent each user's preferences. AutoRec

[Sedhain et al., 2015] leverages an autoencoder (AE) to predict the missing values in the rating matrix. U-AutoRec (IAutoRec) first takes the partially observed rating vectors for each user (item), i.e. yu (yi) as input, and maps each vector into a low-dimensional latent space, followed by a reconstruction layer as output to recover the rating vectors so that the original missing ratings can be generated for recommendation purpose. Formally, the reconstructed vectors are written as:

where h(·) and g(·) are activation functions, W1,W2, b1, b2 are corresponding weights and biases of the AE. The loss function is regularized square loss (U-AutoRec):

where ‖·‖F and ‖·‖2 denote matrix Frobenius norm and vector l2 -norm respectively, O denotes the observed ratings in each vector yu, α controls the regularization strength.

After training the AutoRec, we calculate fu = g(W1 yu+ b1) with the trained parameters W1 and b1, as the lowdimensional latent factors for each user in both domains. fu are then fed into a modified DANN network. To better extract the shared rating patterns from the latent space, the sequence of the latent factors for target domain are interleaved by those from the source domain. To modify the original DANN [Ganin and Lempitsky, 2015] for rating prediction recommendation task, we use one deep feed-forward neural network with parameter set Θr to reconstruct the rating vectors for source and target domain separately (rating predictor) and another neural network with parameter set Θc to predict the corresponding domain label c ∈ {0, 1}, where 0 denotes the input vector belongs to the source domain and 1 the target domain (domain classifier). A fully-connected neural network is applied for feature extraction with parameter set Θf (rating pattern extractor). The training procedure to find the domain-invariant features involves optimizing on Θf to maximize the loss of the domain classifier, while optimizing on

Θc and Θr to minimize the loss of the domain classifier and the label predictor. The total loss function is the combination of the predictor loss and the classifier loss:

where we employ binary cross-entropy for domain label classification loss (ˆ cu is the predicted domain and cu is the ground-truth). R = ‖Θf‖2 + ‖Θr‖2 + ‖Θc‖2 is the regularizer term to alleviate overfitting, in which λ controls the regularization strength. β is used to balance the significance of the source and target parts in the total loss function. µ controls the portion of loss contributed by the domain classifier. Then the optimization objective can be expressed as:

To solve the above problem by stochastic grandient desent (SGD)-like algorithms, the original DANN introduces the gradient reversal layer (GRL). GRL is a simple structure that has different behaviours for forward and back propagation with no intrinsic parameters. In forward propagation, GRL acts as an identity function to let data pass through, while in backpropagation, it calculates the gradient from the subsequent level, multiplies it by -µ and then passes it to the preceding layer. Formally, GRL can be treated as a "function":

where I is the identity matrix. GRL is placed immediately after the rating pattern extractor and before the domain classifier. Therefore, the total loss function adapted to SGD-like algorithm is converted as:

Equation 4 is designed for extracting the deep shared rating patterns for each user in the target and source domain, which is called U-DARec. Our approach can also be applied on obtaining patterns from each item rating vector. We first use I-AutoRec to train a set of latent factors for all items in both domains, and feed them into the following DANN. As there is no overlap for items in the two domains, here, we extract the domain-exclusive rating patterns from the raw rating vectors and separate them using the rating predictor. Thus, the loss of the domain classifier is minimized, leading to a different optimization objective from Equation 4:

Consequently, the GRL layer is no longer needed and SGDlike algorithms can directly optimize Equation 7. We call this model I-DARec.

To train DARec, we apply mini-batch Adam algorithm for both user embedding training and rating prediction procedures. In AutoRec, we initialize W1,W2 with normal distribution (zero mean, 0.01 standard deviation), b1, b2 with zeros and randomly draw a mini-batch U ′ with size B from U to obtain the embeddings. Then, the calculated fu (fi for IDARec) in both domains are interleaved with each other, and are subsequently fed into the modified DANN. Similarly, we initialize the weights in DANN with normal distribution (zero mean, 0.01 standard deviation), and biases with zeros. Then, we train the parameters ˆΘf , ˆΘr, ˆΘc in Equation 4 or Equation 7 for U-DARec and I-DARec respectively.

4 Experiments

In this section, we systematically evaluate the DARec model on multiple subsets extracted from the Amazon dataset with shared users in different categories. After giving the detailed dataset settings, we investigate the effectiveness of knowledge transfer in our approach and compare it with several state-of-the-art methods for cross-domain recommendation.

4.1 Experimental Settings

Datasets

We use the public dataset collected by J. McAuley [He and McAuley, 2016] and define different item categories as domains, where we select users with at least 5 ratings. We deliberately select categories that are as irrelavent as possible and the statistics are summarized in table 1. Both the source and target domains are extremely sparse with at least 99.8% of the ratings are unobserved, which poses a major challenge on most clustering-based and MF-based CDR methods that normally require a relatively dense source domain.

Parameter Settings

In the embedding training stage, we leave out 10% of the data as validation set to tune the hyperparameters where we adjust the number of hidden neurons from 100 to 1500 and regularizer coefficient from 0.1 to 0.00001 to get the lowest RMSE that can be achieved by AutoRec. Then, we feed all the data into the pre-trained AutoRec to get the embeddings calculated as fu (fi). After the sequence of embeddings from source domain is interleaved with those from the target domain, we send it into the DANN. For the rating pattern extractor, we use only one layer with hidden neurons varying from 50 to 500. In the rating predcitor, we apply 3 layers for each domain and 2 layers for the domain classifier, where the number of neurons in each layer follows a pyramid shape and varies with the dimension of the previous pattern extractor. The parameter β is changed from 0.0001 to 1 so that the source domain is emphasized, and µ, λ are varied from 0.0001 to 10, 000. To process the dataset, we randomly leave out 10%(20%) of the data for testing and 90%(80%) for training. 10% of the training set is used as validation set for hyperparamater tuning.

4.2 Effectiveness of Transfer Learning

We first investigate whether DARec can enhance the recommendation accuracy in the target domain. Apart from the standard AutoRec, we compare our approach with several state-of-the-art single-domain rating prediction models:

• PMF [Mnih and Salakhutdinov, 2008] Probabilistic Matrix Factorization is one of the most populare MF methods. It obtains the predicted ratings from the inner product of corresponding user and target item latent factors.
• RBM [Salakhutdinov et al., 2007] Restricted Boltzmann Machine uses neural networks to reconstruct the unknown ratings from the observed ones.
• AutoRec [Sedhain et al., 2015] AutoRec is an RBM-like model which replaces the RBM with an AE to perform the unknown rating reconstruction procedure.
• CF-NADE [Zheng et al., 2016] Neural Autoregressive Distribution Estimator is applied for CF tasks where it replaces the role of RBM for rating reconstruction.

For PMF, we use the package Suprise 1 . For RBM, we rewrote a tensorflow version according to the opensource implementation 2 and for CF-NADE, we choose the version of I-CFNADE-S and reworte a tensorflow version with reference to the opensource implementation 3 . We tune the latent factor dimensions, learning rate and regularizer coefficients to get their best performance.

Figure 3 shows the average RMSE values of the compared single-domain recommendation methods, from which we can observe that both U-DARec and I-DARec obtain obvious improvment over U-AutoRec and I-AutoRec respectively. In U-DARec, the shared rating patterns are transferred via the DNN, while in I-DARec, the distinct rating patterns in each domain are extracted. For instance, in the Office Products and Movies & TV dataset (No.1), U-DARec improves 3.66% in RMSE compared with U-AutoRec and in the Sports & Outdoors and CDs & Vinyl dataset (No.2), the improvement is

1 https://github.com/NicolasHug/Surprise

2 https://github.com/felipecruz/CFRBM

3 https://github.com/Ian09/CF-NADE

4.30%. Although U-DARec still can not compete with IAutoRec and CF-NADE, yet we achieve best performance by further improving on I-AutoRec via I-DARec. The superior performance from I-DARec is probably due to two reasons: 1. the larger number of different input embeddings for I-DARec than that for U-DARec, making the training more effective for I-DARec; 2. the adversarial objectives in UDARec pose a trade-off between extracting shared patterns and enhancing rating prediction accuracy.

We then investigate the impact of the embedding size trained by AutoRec on the single-domain recommendation performance. We first fix the structure of the DANN(i.e., the number of hidden neurons of rating pattern extractor is set to 100) and then change the dimension of the embeddings from 200 to 1200 for all datasets. From figure 2, it can be observed that for both U-DARec and I-DARec, there exist optimal embedding sizes. For instance, in the Office Products and Movies & TV dataset (No.1), U-DARec reaches minimum RMSE at around 800 and I-DARec at around 400. When the input embedding dimension is not large enough, the rating patterns can not be effectively extracted, while at a large embedding size, the learned embeddings are too noisy with undesired latent feature details, bringing detrimental effects on the overall performance. Furthermore, we also notice that although I-DARec achieves the lowest RMSE in general, yet the enhancement from U-DARec is greater than that from IDARec, which can be well understood from the aligned-user settings of the datasets as the model is more effective in learning the shared rating patterns.

4.3 Comparison with Baselines

• CMF [Singh and Gordon, 2008] Collective Matrix Factorization (CMF) is a latent factor model that establishes relationships among multiple domains via MF. Our experiment settings correspond to the three-entitytype schema with two domains. The first entity type is the items in the source domain and the third entity type is the items in the target domain, with the shared users as the bridging entity type. A set of latent factors is learned for each entity type and rating predictions are made via inner products of corresponding latent factors. It is a MF-based transfer learning approach that jointly factorizes two related domains.
• CDTF [Hu et al., 2013] Cross Domain Triadic Factor-

ization (CDTF) is another latent factor model that leverages the triadic relation user-item-domain. It learns the triadic factors for user, item and domain via factorizing a three-order tensor with weighted square loss, where each slice of the tensor corresponds to one domain. CDTF matches the CDR scenario we consider, where each domain has equivalent number of users but vary in the number of items.

• FM-CDCF [Loni et al., 2014] Factorization Machine Cross-Domain Collaborative Filtering (FM-CDCF) is based on FM, where the feature vector in single-domain FM is extended to incorporate collaborative information from other domains. It is well suited for the aligneduser CDR scenario where each input feature vector is composed of the one-hot representation of the user u, one-hot representation of the item i in the target domain, and all the normalized ratings given by u in the source domain. It is still a shallowmodel without the ability of capturing nonlinear interactions between domains.
• CoNet [Hu, 2018] Collaborative Cross Networks (CoNet) is a state-of-the-art deep CDRmodel that transfers knowledge between domains with aligned users, which matches our research settings. It uses DNN to learn latent features in each domain and applies a modified cross-stitch neural network to enable knowledge transfer. It is a highly competitive method and achieves best performance among other non-DL approaches.

For CMF, we rewrote a python version based on the original Matlab code 4 . For FM-CDCF, we adopt the libfm implementation 5 for FM and feed in the extended input feature vectors. For CDTF and CoNet, we implement our own python version according to the original papers. For all the baselines, we tune the latent factor dimensions, learning rate and regularizer coefficients to get their best performance.

Table 2 reports the comparison results among the selected baselines for CDR. Note that in each domain pair, the chosen categories are as irrelevant as possible, so that we can better examine the performance of our model on learning and transfering knowledge from the rating matrix only. It is obvious that DL-based approaches (i.e., CoNet and DARec) are superior to non-deep models. This can be explained by the ability of nonlinearity modelling from the deep neural networks. Also, MF-based mehtods perform better when the source domain dataset is relatively dense, but the datasets we consider in this work are of high sparsity (over 99.8%). For example, in the Office Products and Movies & TV dataset with 90% training data, CoNet gains 2.38% RMSE improvement over FM-CDCF and I-DARec achieves 3.45% over FMCDCF. This result shows that DL-based methods are more suitable for extreme sparse datasets, which is in consistency with the findings in [Hu, 2018]. Furthermore, we notice that CoNet performs better than U-DARec as U-DARec adopts adversarial training to balance the extraction of shared user rating patterns and the pursuit of high rating prediction accuracy. This weakness is overcome by I-DARec which minimizes the losses of domain classifier and rating predictor at the same time. Thus, as we can see, I-DARec achieves the best performance among the baselines.

4 http://www.cs.cmu.edu/ ajit/cmf/

5 http://www.libfm.org/

5 Conclusions

In this paper, we proposed the DARec model for crossdomain recommendation, where we consider the singledomain recommendation task with users aligned and only rely on the rating matrices. To extract the shared user rating patterns from the two domains, we first apply AutoRec to generate embeddings to represent each user. Then, the embeddings from the source and target domains are interleaved and fed into DANN for knownledge transfer. Using a domain classifier, the shared deep rating patterns from the two domains are extracted, followed by a rating predictor that separates the estimated ratings for each domain. We further extend our model by taking the ratings for each item as input to extract the distinct features of the two items. We demonstrate that our model achieves best performance in rating prediction task among several widely-adopted and state-of-theart cross-domain recommendation approaches. In the future, we will integrate content-based domain adaptation models to gain further performance improvement.