Causal Embeddings for Recommendation: An Extended Abstract

1 Introduction

In recent years, online commerce has outpaced the growth of traditional commerce. As such, research work on recommender systems has also grown significantly, with recent Deep Learning (DL) approaches achieving state-of-the-art results. Broadly, these DL approaches frame the recommendation task as either:

• A distance learning problem between pairs of products or pairs of users and products, measured with Mean Squared Error (MSE) and Area Under the Curve (AUC), like in the work by [Grbovic et al., 2015; Vasile et al., 2016; Pennington et al., 2014].
• A next item prediction problem that models user behavior and predicts the next action, measured with ranking metrics such as Precision@K and Normalized Discounted Cumulative Gain (NDCG), as presented in [Hidasi et al., 2015; Hidasi et al., 2016].

∗Work first published as: Bonner, S., Vasile, F. (2018). Causal embeddings for recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems (pp. 104-112). ACM.

However, we argue that both approaches fail to model the inherent interventionist nature of recommendation, which should not only attempt to model the organic user behavior, but to actually attempt to optimally influence it according to a preset objective.

Using a causal vocabulary, we are interested in finding the treatment recommendation policy that maximizes the reward obtained from each user with respect to the control recommendation policy. This objective is traditionally denoted as the Individual Treatment Effect (ITE) [Rubin, 1974].

In our work, we introduce a modification to the classical matrix factorization approach which leverages both a large biased sample of biased recommendation outcomes and a small sample of randomized recommendation outcomes in order to create user and products representations. We show that using our method, we improve the ITE prediction performance over traditional matrix factorization and causal inference approaches.

1.1 Causal Vocabulary

Below we briefly introduce the causal vocabulary and notation that we will be using throughout the paper.

The Causal Inference Objective

In the classical setup, we want to determine the causal effect of one single action which constitutes the treatment versus the control case where no action or a placebo action is undertaken (do vs. not do). In the stochastic setup, we want to determine the causal effect of a stochastic treatment policy versus the baseline control policy. In this case, both treatment and control are distributions over all possible actions. We retrieve the classical setup as a special case.

Recommendation Policy

We assume a stochastic policy πx that associates to each user ui and product pj a probability for the user ui to be exposed to the recommendation of product pj:

For simplicity we assume showing no products is also a valid intervention in P.

Policy Rewards

Reward rij is distributed according to an unknown conditional distribution r depending on ui and pj :

The reward Rπx associated with a policy πx is equal to the sum of the rewards collected across all incoming users by using the associated personalized product exposure probability:

Individual Treatment Effect

The Individual Treatment Effect (ITE) value of a policy πx for a given user i and a product j is defined as the difference between its reward and the control policy reward:

We are interested in finding the policy π∗ with the highest sum of ITEs:

Optimal ITE Policy

It is easy to show that, starting from any control policy πc, the best incremental policy π∗ is the policy that shows deterministically to each user ui the product p∗i with the highest personalized reward r∗ i :

IPS Solution For π∗

In order to find the optimal policy π∗ we need to find for each user ui the product with the highest personalized reward r∗ i .

In practice we do not observe directly rij, but yij ∼ rij πc(pj |ui).

The current approach to estimate rij constitutes in using Inverse Propensity Scoring (IPS)-based methods to predict the unobserved reward rij :

This assumes we have incorporated randomization in the current policy πc. Even with the existence of randomization, the main shortcoming of IPS-based estimators is that they do not handle well big shifts in exposure probability between treatment and control policies (products with low probability under the logging policy πc will tend to have higher predicted rewards).

Addressing the Variance Issues of IPS

Ideally, in order to maintain minimum variance, we should collect data using fully randomized recommendations, e.g. when: πc = πrand. However, this means zero recommendation performance and therefore cannot be a solution in practice.

Our question: Could we learn from πc a predictor for performance under πrand and use it to compute the optimal product recommendations p∗i ?

2 Our Approach: Causal Embeddings (CausE)

We are interested in building a good predictor for recommendation outcomes under random exposure for all the userproduct pairs, which we denote as ˆ yrand ij . We make the assumption that we have access to a large sample Sc from the logging policy πc and a small sample St from the randomized treatment policy πt=rand (e.g. the logging policy πc uses egreedy randomization).

To this end, we propose a multi-task objective that jointly factorizes the matrix of observations yc ij ∈ Sc and the matrix of observations yt ij ∈ St. Our approach is inspired by the work in [Rosenfeld et al., 2016] and shares similarities with other domain-adaptation based models for counterfactual inference such as the work in [Johansson et al., 2016; Shalit et al., 2017].

2.1 Predicting Rewards ViaMatrix Factorization

By using a matrix factorization model, we assume that both the expected factual control and treatment rewards can be approximated as linear predictors over the shared user representations ui, as shown in Fig. 1.

As a result, we can approximate the ITE of a user-product pair i, j as the difference between the two, see eq.1 below:

Proposed Joint Optimization Solution

The joint optimization objective contains two terms, one measuring the performance of the solution on the treatment sample and one on the control sample. The novel part of the objective comes from the additional constraint on the distance between the treatment and control vectors for the same action/item, that can be directly linked to the ITE effect of the item.

Sub-objective #1: treatment loss term Lt. We define the first part of our joint prediction objective as the supervised predictor for yt ij , trained on the limited sample St, as shown in the eq. 2 below:

where:

• Pt is the parameter matrix of treatment product representations.
• U is the fixed matrix of the user representations.
• Yt is the observed rewards matrix.
• L is an arbitrary loss function.
• Ω(.) is a regularization term over the model parameters.

Linking the control and treatment effects. Additionally, we can use the translation factor in order to be able to use the model built from the treatment data St to predict outcomes from the control distribution Sc:

Sub-objective #2: control loss term Lc. Now we want to leverage our ample control data Sc and we can use our treatment product representations through a translation:

which can be written equivalently as:

where we regularize the control Pc against the treatment embeddings Pt (W ∆ = Pt - Pc). As shown in the eq. 4 below, we can see that IPS is a function of W ∆. Therefore, by regularizing W ∆ we are effectively putting a constraint on the magnitude of the IPS term.

Overall Joint Objective

By putting the two tasks together (Lt and Lc) and regrouping the loss functions and the regularizer terms, we have that:

where L(.) is the reconstruction loss function for the concatenation matrix of Pt and Pc, Ωdisc(.) is a regularization function that weights the discrepancy between the treatment and control product representations and Ωembed(.) is a regularization function that weights the representation vectors.

Generalization to User Shift

Our objective function can be altered to allow for the user representations to change, we obtain the equation below:

Putting the loss functions associated with the user and product dimension together (Lprod CausE , Luser CausE ), we reach the final loss function for our method:

3 Experimental Results

3.1 Experimental Setup

The task is predicting the outcomes yrand ij under treatment policy πrand, where all of the methods have available at training time a large sample of observed recommendations outcomes from πc and a small sample from πrand. Essentially this is a classical conversion-rate prediction problem so we measure Mean-Squared Error (MSE) and Negative LogLikelihood (NLL). We report lift over average conversation rate from the test dataset:

3.2 Baselines

We compare our method with the following Matrix Factorization baselines:

Algorithm 1: CausE - Causal embeddings for recommendations.

Input : Mini-batches of Sc = {(ui, pcj , δc ij )}Mc i=1 and St = {(ui, ptj , δt ij )}Mt i=1 , regularization parameters λembed and λdist, learning rate η Output: Pt, Pc, Ut, Uc - Product and User Control and Treatment Matrices 1 Random initialization of Pt, Pc,Ut,Uc ; 2 while not converged do 3 read batch of training samples; 4 for each product pj in Pc, Pt do 5 Update product vector: pj ← pj - η∇Lprod CausE (p, λembed, λdist)) 6 end 7 for each user ui in Uc, Ut do 8 Update user vector: ui ← ui - η∇Luser CausE (u, λembed, λdist)) 9 end 10 end 11 return Pt, Pc,Ut,Uc

• Bayesian Personalized Ranking (BPR) To compare our approach against a ranking based method, we use Bayesian Personalized Ranking (BPR) for matrix factorization on implicit feedback data [Rendle et al., 2009].
• Supervised-Prod2Vec (SP2V): As a second factorization baseline we will use a Factorization Machine-like method [Rendle, 2010] that approximates yij as a sigmoid over a linear transform of the inner-product between the user and product representations.

And these Causal Inference baselines:

• Weighted-SupervisedP2V (WSP2V): We employ the SP2V algorithm on propensity-weighted data, this method is similar to the Propensity-Scored Matrix Factorization (PMF) from [Schnabel et al., 2016] but with cross-entropy reconstruction loss instead of MSE/MAE.
• BanditNet (BN): To utilize BanditNet [Joachims et al., 2018] as a baseline, we use SP2V as our target policy πw. For the existing policy πc, we model the behavior of the recommendation system as a popularity-based solution, described by the marginal probability of each product in the training data.

3.3 Experimental Datasets

We use the Netflix and MovieLens10M explicit rating datasets (1-5). In order to validate our method, we preprocess them as follows: We binarize the ratings yij by setting 5-star ratings to 1 (click) and everything else to zero (view only) and generate a skewed dataset (SKEW) with 70/10/20 train/validation/test event split that simulates rewards collected from uniform exposure πrand t , following a similar protocol with the one presented in previous counterfactual estimation work such as in [Liang et al., 2016; Swaminathan and Joachims, 2015] and described in detail in the long version of our paper [Bonner and Vasile, 2018].

3.4 Experimental Setup: Exploration Sample St

We define 5 setups for incorporating the exploration data:

• No adaptation (no) - trained only on Sc.
• Blended adaptation (blend) - trained on the blend of the Sc and St samples.
• Test adaptation (test) - trained only on the St samples.
• Product adaptation (prod) - separate treatment embedding for each product based on the St sample.
• Average adaptation (avg) - average treatment product by pooling all the St sample into a single vector.

3.5 Results

Table 1 displays the results for running all the approaches on the datasets. Our proposed CausE method significantly outperforms all baselines across both datasets, demonstrating that it has a better capacity to leverage the small test distribution sample St. We observe that, out of the three CausE variants, CausE-prod-C, the variant that is using the regularized control matrix, clearly out-performs the others. Further, figure 3 highlights how CausE is able to make better use of increasing quantities of test distribution present in the training data compared with the baselines.