Does Every Data Instance Matter? Enhancing Sequential Recommendation by Eliminating Unreliable Data

1 Introduction

Sequential recommender systems (SRSs) generally predict each user's next-item given the preceding (and succeeding) item sequence [Pei et al., 2017; Yuan et al., 2020]. Thereby, a training instance of SRSs is typically constructed by an input item sequence and its next-item as target. Such training procedure implicitly assumes that each input is related to its target, which might not always hold. For example, we may accidentally click on songs or videos that we are not interested in, or receive recommendations from friends who are inconsistent with our preference. Such behaviors could cause unreliable instances (i.e. targets mismatch inputs), which has been rarely investigated by previous studies.

To fill this gap, we first verify the existence and severity of unreliable instances by evaluating seven state-of-the-art SRSs. The results unveils that SRSs trained without unreliable data detected by our proposed data reliability measurement reach higher accuracy with a maximum 5.71% lift on NDCG, which firmly emphasizes the necessity of eliminating unreliable data for performance-enhanced SRSs. Based on these, we further conduct analysis and gain two insightful observations: (1) unreliable instances are prone to have higher training loss; and (2) high-loss instances are not necessarily unreliable. They can also be uncertain instances caused by blurry patterns of input sequences that are correlated with uncertain users whose preferences shift frequently, or uncertain items that co-occur with various subsequent items.

∗Corresponding Author

Next, we exploit both loss and uncertainty as guidance to better distinguish unreliable instances, as depicted in Fig. 1. First, instances with low loss and low uncertainty can be kept as reliable ones. Second, instances with high loss but low uncertainty can be cautiously removed as unreliable ones. Third, instances with high loss and high uncertainty are either uncertain & reliable or uncertain & unreliable instances. Hence, an SRS is expected to be capable of lowering the uncertainty, so as to identify and handle unreliable instances as indicated by the arrows in Fig. 1. Lastly, low loss and high uncertainty are conceptually contradictory, as high uncertainty is generally caused by blurry patterns of input sequences which are hard to fit with high training loss.

However, there are two challenges for achieving such effects: (1) how to accurately model loss and uncertainty of per-instance; and (2) how to mine unreliable instances from high uncertainty ones. To this end, we propose a novel framework BERD that enhances SRS By Eliminating unReliable Data. Specifically, BERD models the input of each instance as a Gaussian distribution, with its mean representing the extracted dynamic user preference for loss calculation, and its covariance representing the uncertainty of the extracted preference. An extended objective function is designed to properly endow uncertainty to every training instance. Meanwhile, we equip BERD with a novel uncertainty-aware graph convolution network (UGCN) so as to reduce the uncertainty of instances with blurry sequential patterns. As such, uncertain instances (either reliable or unreliable) can be clearly identified and properly handled as depicted in Fig. 1.

In sum, our contributions are threefold: (1) we are the first to define unreliable instances (i.e. targets mismatch inputs) in SRSs; by verifying their presence and severity, we highlight the necessity of eliminating them for performance-enhanced SRSs; (2) we conduct analysis gaining two insightful observations, which inspire us to design BERD to detect unreliable instances by modelling per-instance loss and uncertainty via a Gaussian distribution. Moreover, a novel UGCN facilitates to mine unreliable instances from high uncertainty ones by lowering instance uncertainty; and (3) extensive experiments on four real-world datasets demonstrate the superiority of BERD. Additionally, detailed ablation study further confirms the effectiveness of each module of BERD.

2 RelatedWorks

Early SRSs adopt Markov Chains to model users' dynamic preference, such as FPMC [Rendle et al., 2010]. Recently, deep learning advances are more prevalent. For example, Caser [Tang and Wang, 2018] and GRU4Rec [Cho et al., 2014] exploit CNN and RNN to model users' sequential behaviours, respectively. Later, researchers find it is essential to highlight relevant input items given the target. Typical works include NARM [Li et al., 2017], RUM [Chen et al., 2018], SASRec [Kang and McAuley, 2018] and GARG [Wu et al., 2020], which adopt attention mechanisms; MCPRN [Wang et al., 2019] proposes a multi-channel purpose routing network; HRL [Zhang et al., 2019] leverages the hierarchical reinforcement learning. BERT4Rec [Sun et al., 2019] and GRec [Yuan et al., 2020] verify the benefits of bidirectional context for dynamic preference modelling. Besides, HGN [Ma et al., 2019], MA-GNN [Ma et al., 2020] and M3R [Tang et al., 2019] reveal the usefulness of modelling long-term user interests.

Previous works, however, posit each input sequence as a whole should be related to its target, thus fail to handle unreliable instances (i.e. targets mismatch inputs). Although several studies have investigated noisy ratings [Burke et al., 2006; Li et al., 2019], they are mainly designed for the rating prediction task with static preference and beyond our scope.

3 Reliability Analysis on Data Instances

In SRSs, each user u chronologically interacts with a list of items Su = [iu1 , iu2 , . . . , iu|Su | ], where ium is the m-th item u interacts with (1 ≤ m ≤ |Su |). During training, an SRS aims to predict u's next item as target, given her id and L preceding items as input. Formally, the training instance for u to predict her t-th interaction is denoted by an input-target pair < {u, Su t,L}, iut >, where Su t,L = [iumax(1,t-L) , . . . , iut-2 , iut-1 ]. The instance with target iut mismatching its input Su t,L is called an unreliable instance.

Measurement of Data Reliability

To quantify the reliability of each instance, i.e., the matching degree between inputs and targets, we propose a data reliability measurement with the aid of item attributes (e.g., category and tag) and item co-occurrence relation. Assume there are A attributes and N items. Each item i is characterized by two tf-idf vectors qai ∈ RA and qci ∈ RN , treating attributes and co-occurred items as terms, respectively. Taking instance < {u, Su t,L}, iut > as an example, we define the similarity between Su t,L and iut as the averaged cosine similarity between iut and each itemwithin Su t,L, given by,

where · indicates vector dot product; || · || denotes the L2 norm; sima (·) indicates the similarity measured by item attributes; simc (·) can be analogically calculated with qci . We then sample a set of negative items I - u (I - u ∩ Su = ∅) and rank the target iut among the negative ones to get the relative matching degree between the targets and inputs:

where [x]+ is an indicator function whose value is 1 with x > 0, and 0 otherwise; ra indicates the relative matching degree between the target and its input w.r.t. attributes, for example, ra = 0.7 means the target matches the input better than 70% of negative items. rc holds similar calculation and meaning w.r.t. item co-occurrence. The measurement classifies the instances with both ra and rc lower than 0.9 as unreliable.

Existence and Severity of Unreliable Data

To verify the existence and severity of unreliable instances, we evaluate the performance of seven state-of-the-art SRSs on four real-world datasets (see Table 2) under two views w.r.t. the training process: Origin - directly uses the original training instances; Filter - removes the unreliable data from training sets based on our proposed reliability measurement, where the column 'U-Ratio' in Table 2 shows their respective percentage w.r.t. the total training instances. The final recommendation results are reported in Fig. 4, where both views are also considered during test (i.e. Test-Origin and Test-Filter). We can observe that SRSs trained with filtered datasets (i.e. Train-Filter) consistently reach higher accuracy with a maximum lift of 3.88%, 5.71% on NDCG under both test views, respectively. This validates the existence and severity of unreliable instances, as well as shows the necessity of eliminating them for performance-enhanced SRSs.

Characteristics of Unreliable Data

Although the eliminated instances are generally unreliable based on our proposed reliability measurement, the reliance on high-quality auxiliary information restricts its generality. Therefore, we further conduct an in-depth analysis on the detected unreliable instances, whereby two insightful observations are noted. These help us discover intrinsic and distinctive characteristics of unreliable instances.

Obs. 1. Unreliable instances generally have high training loss; but high-loss instances are not necessarily unreliable.

Figs. 2(a-b) show the loss distribution of reliable (blue bars) and unreliable (red bars) instances at different epochs when training Caser on ML-1M and CD. Similar trends can be found with other SRSs and datasets. We note that (1) the loss of unreliable instances is generally higher than that of reliable ones; and (2) the high-loss area also contains blue bars, suggesting that high-loss instances are not necessarily unreliable. Obs. 1 implies that modelling per-instance loss is helpful but not sufficient for distinguishing unreliable instances.

Obs. 2. High-loss also can be caused by reliable but uncertain instances with blurry sequential patterns. Such patterns are correlated with uncertain users or items.

In SRSs, there are always uncertain users whose preferences shift frequently, or uncertain items that are complex and cooccur with various succeeding items. The instances whose inputs are associated with such uncertain users/items could have blurry sequential patterns, thus being hard to fit and leading to high training loss. Formally, the uncertainty of a user (or an item) can be measured by the ratio of unreliable instances over the total number of instances the user (or item) holds. A higher ratio suggests a higher uncertainty. Their existence can be reflected in Fig. 2(c), where the uncertainty of around 6% users and 4% items is higher than 0.3. We then plot the training loss of reliable instances on ML-1M, where yellow bars indicate instances that are related to these uncertain users (Fig. 2(d)) or items (Fig. 2(e)), while blue bars hold the opposite case. We note that yellow bars are generally distributed to the right of blue bars, indicating that uncertain users and items do increase the loss of reliable instances, confirming that uncertain users/items bring uncertainty into sequential patterns, thus leading to high training loss.

To summarize, the two observations suggest that high loss can be caused by either unreliable or uncertain instances, which inspire us to leverage both training loss and uncertainty to help distinguish unreliable instances as depicted by Fig. 1 , instead of relying on side information.

4 The Proposed BERD

In this section, we propose a novel neural framework - BERD - that boosts the recommendation performance of SRSs By Eliminating unReliable Data.

4.1 Framework Overview

Fig. 3 depicts the overall framework of our proposed BERD, mainly composed of four modules, including (1) Sequence Modelling with UGCN - it extracts input sequential patterns with reduced uncertainty on the base of uncertainty-aware graph convoluntion network; (2) Instance Uncertainty Modelling - it models input sequential patterns as a Gaussian Distribution, with its mean representing the extracted dynamic user preference, and its covariance representing the uncertainty of the extracted preference (i.e. instance uncertainty); (3) Extended Objective Function - it helps endow proper uncertainty to every training instance by extending the conventional recommendation objective with a sampled loss and an uncertainty regularizer; and (4) Unreliable Instance Filter - it eliminates unreliable instances with high loss but low uncertainty. Among these modules, Instance Uncertainty Modelling and Extended Objective Function accurately model the training loss and uncertainty of each instance; Sequence Modelling with UGCN endeavors to mine unreliable instances from high loss & high uncertainty ones by reducing the instance uncertainty. As a result, the instances with high loss but low uncertainty can be cautiously removed as unreliable instances by the Unreliable Instance Filter.

4.2 Sequence Modelling with UGCN

To extract more accurate sequential patterns from {u, Su t,L} (input), it is fundamental to get accurate user and item embeddings, especially for uncertain users and items which blurs input sequential patterns. To this end, we propose UGCN ensuring that users and items with high uncertainty will contribute less to their neighbors but be compensated more. Formally, we build a graph that includes users and items as nodes, while the edges are constituted by user-item interaction and itemitem adjacency relationships. Each user u (or item i) is initially encoded as an embedding vector e0u ∈ Rd (or e0i ∈ Rd )

and an uncertainty factor fu (or fi), where d denotes the embedding size. Take user (node) u as an example, UGCN propagates u's embedding by restricting that her neighboring nodes with higher uncertainty contribute less:

where eku (k ≥ 1) is the refined embedding of u after k layers' propagation; Mu is the set of items u has interacted with; Mj is the set of users who have interacted with item j and the adjacent items of j; σ(·) is the sigmoid function to rescale the uncertainty factors into (0, 1); thus 1-σ(·) ensures that high-uncertainty neighbors affect u less. After K layers' propagation, we aggregate the embeddings obtained at each layer to form the final representation of u:

where e0u and eku (k ≥ 1) contain the information from u and her neighbors, respectively. Thus their weights 1-σ(fu) and σ(fu) ensure that u's final representation eu will be compensated more by u's neighbors if u has high uncertainty. Similarly, item i ∈ Su t,L is also embedded into ei according to Eqs. 3 and 4. As such, UGCN restricts the propagation of uncertain information while encourages the spreading of certain signals between users and items. This facilitates to produce accurate representations especially for uncertain users and items, allowing Sequence Modelling to extract sequential patterns with lower uncertainty. In particular, we stack the item embedding ei, i ∈ Su t,L into matrix Eut ∈ RL×d with sinusoidal positional encoding [Vaswani et al., 2017], which is then fed into a multi-head self-attention network [Vaswani et al., 2017] to extract u's short-term interest:

where h is the number of heads; W Qj ,W Kj ,W Vj ∈ Rd×d/h and W O ∈ Rd×d are learnable projection matrices. The Lth row of MH(Eut ) ∈ RL×d encodes u's short-term interest, which is further concatenated with her general preference to denote the sequential pattern of user u at time step t:

4.3 Instance UncertaintyModelling

The obtained sequential pattern su,t merely represents user u's preference at time step t. In our study, we aim to additionally model the uncertainty of the extracted preference, which is used to represent instance uncertainty and better distinguish unreliable instances. To this end, Instance Uncertainty Modelling aims to model u's preference at time step t as the mean µu,t of a Gaussian Distribution, while its covariance Σu,t represents the uncertainty of the extracted preference (i.e. instance uncertainty). As modeling a full covariance matrix is expensive, we constrain it to be diagonal,

where W1,W2 ∈ R2d×d and b1, b2 ∈ Rd are trainable parameters; ReLU(·) is the activation function [Nair and Hinton, 2010] to restrict the values of Σu,t to be nonnegative; avg(·) is the average function encoding user/item uncertainty into instance uncertainty. Next, we quantify the instance uncertainty hu,t (i.e. the uncertainty of u's preference at time step t) as the entropy of a multivariate Gaussian distribution,

where j indices each dimension of Σu,t.

4.4 Extended Objective Function

To accurately model recommendation loss and uncertainty of each instance, we expand the conventional recommendation objective with a sampled loss and an uncertainty regularizer. Specifically, the conventional objective encourages the model to rank target item iut higher than the sampled negative item i′ by maximizing the gap between their predicted scores:

To seamlessly accommodate the learning of instance uncertainty, we introduce an additional sampled loss, which aims to push Z samples from N(µu,t,Σu,tI) to be close to the target item iut and far away from the negative item i′ :

where λ balances the importance of the sampled loss; pzu,t denotes the embedding of the z-th sample, generated with a reparameterization trick [Kingma and Welling, 2014]:

where ⊙ is the element-wise multiplication. However, simply training with Lrec and Lsam is not sufficient. Specifically, large covariance will enforce the generated samples far away from each other based on Equ. 11, thus leading to high Lsam according to Equ. 10. Hence, during optimization, the model will instinctively incentivize all covariance to be zero for a reduced Lsam. To avoid such trivial solution, we constrain the model to maintain certain degree of instance uncertainty with an uncertainty regularizer:

where γ is used to control the degree of uncertainty to maintain, and we thus do not introduce additional regularization coefficients. Accordingly, the final objective function is,

where Θ denotes the parameter set of BERD.

Minimizing L encourages BERD to properly endow high uncertainty to uncertain instances, and vice versa. The rationale behind is that - for certain & reliable instances, their mean vectors (e.g. µu,t) are generally close to target vectors (e.g. eiut ). Therefore, reducing their uncertainty can effectively help decrease Lsam as the generated sample (e.g.

pzu,t ) will be concentrated around the target. For certain & unreliable instances, their input is certain with salient sequential patterns as certain & reliable ones, thus their uncertainty can be collaboratively reduced. For uncertain instances, their mean vectors are normally far away from target vectors, thus lowering their uncertainty cannot ensure a reduced Lsam. Recall that Lreg is introduced to maintain certain degree of uncertainty. Hence, the model has to allocate high uncertainty to uncertain instances, as it is a smarter way to minimize L.

4.5 Unreliable Instance Filter

Till now, we have obtained the recommendation loss Lrec (Equ. 9) and uncertainty hu,t (Equ. 8) for each instance from the Instance Uncertainty Modelling and Extended Objective Function; meanwhile, the total number of uncertain instances is comprised by the Sequence Modelling module via UGCN. Next, we remove these unreliable instances with the Unreliable Instance Filter. Specifically, in each epoch, we rank all the instances according to their Lrec and hu,t in a descending order, respectively. Then, the instances ranked at top α of Lrec and bottom half of hu,t are considered as unreliable instances, and their gradient will not be backpropagated to the model parameters in this epoch.

5 Experiments and Analysis

Datasets. We adopt four datasets varying w.r.t. domain, size, sparsity level, and ratio of unreliable data, as shown in Table 2. ML-1M [Harper and Konstan, 2015] is a popular dataset for movie recommendation. Steam [Kang and McAuley, 2018] is a game recommendation benchmark collected from Steam. CD and Elect are product review datasets crawled from Amazon [McAuley and Leskovec, 2013] for cd and electronics, respectively. Following [Huang et al., 2018], we discard inactive users with fewer than 7 interactions.

Baselines. BPRMF [Rendle et al., 2009] is a traditional matrix factorization model with a pairwise ranking loss; FPMC [Rendle et al., 2010] is a classic SRS that combines matrix factorization and Markov Chains; GRU4Rec [Hidasi et al., 2016] and Caser [Tang and Wang, 2018] adopt RNN and CNN to extract users' dynamic preferences, respectively; SASRec [Kang and McAuley, 2018] and GC-SAN [Xu et al., 2019] use attention to highlight salient input items, while GCSAN additionally adopts GNN to obtain accurate item representations; BERT4Rec [Sun et al., 2019] considers bidirectional context to model users' dynamic preferences; and HGN [Ma et al., 2019] applies a hierarchical gating network to fuse long-term interest for dynamic preference modelling. Evaluation Protocol. We follow the same evaluation protocol as in [Sun et al., 2019]. To be more specific, for each user, we split the last two interactions (instances) into validation and test sets, respectively, while the rest are used for training. To improve the test efficiency, we pair each target

item in the test set with 100 negative items that the user has not interacted with, and rank the target item among the 101 items. The negative items are sampled according to the distribution of item popularity. HR, NDCG and MRR are adopted to evaluate the ranking quality. Generally, higher metric values indicate better ranking performance.

Parameter Settings. We adopt Xavier [Glorot and Bengio, 2010] initializer and Adam [Kingma and Ba, 2015] optimizer with d = 50; the learning rate η = 0.01 with batch size of 8192; the weight of sampled loss λ = 0.01 and uncertainty margin γ = 1; the number of propagation layers K = 2; the input length L = 5; the sample size Z = 4; the filter ratio α = 0.05 for Steam and CD, and α = 0.1 for ML-1M and Elect; the head number of self-attention is set to 2.

Overall Comparison. Table 1 shows the performance of all methods, where several major findings are noted: (1) BPRMF performs the worst among all methods as it fails to model sequential behavior; (2) deep learning based SRSs (e.g., SASRec) generally achieves better performance than conventional SRSs (i.e. FPMC); (3) the fact that SASRec and GC-SAN are superior to GRU4Rec and Caser implies the usefulness of highlighting relevant input items with attention mechanism; (4) BERT4Rec and HGN are the best baselines, indicating that considering bidirectional context and long-term preference assists in reducing the negative impact of unreliable instances; and (5) BERD consistently achieves the best performance, demonstrating its superiority against all the counterparts. Moreover, BERD trained under Train-Origin view defeats all baselines under the Train-Filter view as depicted in Fig. 4, which further confirms that eliminating unreliable instances w.r.t. recommendation loss and uncertainty via our end-to-end BERD is more efficient than the heuristic reliability measurement.

Ablation Analysis. To examine the efficacy of each module of BERD, six variants are compared as shown in Table 3. First, (a) without uncertainty modelling is significantly surpassed by (f) - our final BERD, verifying the necessity of modelling both loss and uncertainty to eliminate unreliable data. Second, (b) abandons uncertainty regularizer and performs closely to (a), as it fails to allocate uncertainty properly. Third, (c) replaces UGCN with a classic GCN; (d-e) enhance GCN with the propagation (Equ. 3) and fusion (Equ. 4) operation of UGCN, respectively. The performance comparison among (c-f) firmly validates the superior design of UGCN.

Hyper-parameters Analysis. The filter ratio α, sample size Z, sampled loss weight λ and uncertainty margin γ are four vital hyper-parameters in BERD. Their impacts are illustrated by Figs. 5 and 6, which indicate that the best choice for α is around 0.05∼0.10 depending on the datasets; Z > 3 is sufficient to help reach good performance; the optimal setting for λ ranges from 0.01 to 0.1; and BERD is less sensitive to γ in comparison with other hyper-parameters.

Visualization. Fig. 7 depicts the instance distribution w.r.t. Lrec (Equ. 9) and hu,t (Equ. 8) at different training epochs on ML-1M, where the red points denote unreliable instances. As the epoch increases from 1 to 50, the ratio of detected unreliable instances climbs up from 5.04% to 5.88% while the number of high-uncertainty instances gradually drops (the maximum of hu,t decreases from 20 to 10). It exhibits that BERD is capable of progressively mining unreliable instances from high-uncertainty ones.

6 Conclusion

This paper first verified that SRSs can be misguided by the unreliable instances (i.e. targets mismatch inputs). To ease this issue, we propose a novel neural model BERD guided by two insightful observations to eliminate unreliable data for performance-enhanced SRS. BERD modelled both loss and uncertainty via a Gaussian distribution, whereby instances with high loss but low uncertainty can be cautiously removed as unreliable instances; meanwhile the novel UGCN is designed to help mine unreliable instances from the high loss & high uncertainty ones by lowering instance uncertainty. Extensive experiments verified the superiority of BERD.

Acknowledgments

The work is partially supported by the National Key Research and Development Program of China (2020YFB1707900), National Natural Science Foundation of China (62072088), Ten Thousand Talent Program (ZX20200035), and Liaoning Distinguished Professor (XLYC1902057).