HPDM: A Hierarchical Popularity-aware Debiased Modeling Approach for Personalized News Recommender

1 Introduction

News recommender systems have emerged as a fundamental technological approach for tackling information overload [Wu et al., 2019b; Wu et al., 2019a; An et al., 2020]. While these systems primarily aim to precisely model user interests based on historical click behaviors, they face significant challenges stemming from the inherent nature of news distribution. Specifically, user interaction data typically exhibits a pronounced long-tail distribution [Zheng et al., 2021;

∗Corresponding author

Wei et al., 2021], where a majority of user engagements concentrate on a small subset of popular news. This skewed pattern would introduce potential biases in user modeling, as models may overly emphasize popular news when learning user interests, thereby compromising the effectiveness of personalized recommendations.

While progress has been made in alleviating the long-tail phenomenon across various recommendation domains [Wei et al., 2021], such as PDA [Zhang et al., 2021b] which could effectively de-confound negative bias effects during training while leveraging positive popularity signals at inference time, adapting these methods to news recommendation poses unique challenges:

First, news popularity manifests in a hierarchical manner across different granularity (e.g., titles and categories shown in Figure 1), with heterogeneous effects on user behaviors. Some users may be more susceptible to trending titles, while others are primarily influenced by popular categories. Furthermore, user interactions with news of varying popularity levels often stem from distinct motivations, i.e., clicks on popular news may largely reflect conformity effects, whereas engagement with less popular news typically indicates genuine interests. However, existing approaches typically adopt a simplified view of popularity and treat all user interactions uniformly, failing to capture nuanced relationships between hierarchical popularity patterns and user behaviors.

Second, modeling user interactions with news of different popularity levels needs to be treated differently. As shown in Figure 2, for popular news, user's real interest and conformity could both determine clicks on the news. Hence, the key challenge lies in disentangling genuine user interests from the confounding effects of social influence. In contrast, unpopular news interactions, though sparse, often provide more reliable signals of personal preferences and how to leverage these valuable but limited data to model user is important. Current methods typically employ a uniform modeling strategy regardless of news popularity, overlooking these distinct characteristics and modeling requirements.

In this paper, we propose a hierarchical causal debiasing framework to address these challenges. First, we design a hierarchical popularity-aware user interest modeling approach that captures user preferences across different news granularities (from titles to categories). At each granularity level, we introduce a selective masking mechanism that distinctively processes popular and unpopular news interactions, enabling

our model to construct more comprehensive user representations that reflect how popularity differently impacts individual users. For instance, when modeling user interests in popular news, we mask unpopular news from the user's click history, and vice versa. Second, we develop a dual-view causal debiasing structure to effectively capture genuine user interests. For popular news interactions, we employ counterfactual reasoning to identify and mitigate conformity effects by comparing observed user behaviors against hypothetical scenarios where clicks are solely influenced by popularity. For unpopular news interactions, we implement an inverse propensity weighting framework to address the exposure bias, which helps appropriately weight these valuable but potentially underrepresented interaction signals. During inference, our framework eliminates both the popularity-induced effects and inverse propensity weights used in training, enabling more accurate capture of genuine user interests while reducing the impact of hierarchical popularity biases.

The main contributions of this work are:

1. We propose a hierarchical causal debiasing framework for news recommendation that effectively addresses long-tail distribution challenges while preserving genuine user interests.
2. We propose a hierarchical framework that alleviate the popularity bias at different news granularities (from titles to categories) equipped with a selective masking mechanism. Besides, we design a differential causal debiasing structure that employs counterfactual reasoning for popular news to alleviate the conformity effect and inverse propensity weighting for unpopular news to capture genuine user interests.

(3) Extensive experiments on two widely-used benchmarks (MIND and Adressa) demonstrate our framework's superior performance over existing news recommendation and causal recommendation baselines.

2 RelatedWork

2.1 News Recommendation

In the early stages of news recommendation research, scholars predominantly relied on hand-crafted feature engineering to construct user profiles [Li et al., 2010; Garcin et al., 2013; Son et al., 2013].

Unlike manual features, Deep Neural Networks (DNNs) [Vaswani et al., 2017] have the ability to learn textual features, which have been widely used in modeling user interest [Wu et al., 2019b; Peng et al., 2020; Peng et al., 2022;

M

(b) Only user's real interest influence user click for unpopular news.

Figure 2: Causal graph for the popular and unpopular news. Each node represents: (N) News, (U) User, (P) Popularity, (I) Interest Matching, (C) Conformity Matching, (M) User Click.

Xu et al., 2023]. For example, Wu et al. [Wu et al., 2019b] introduced an innovative neural architecture leveraging multi-head self-attention mechanisms to capture the nuanced interplay between news content and user behavior; Recent studies have investigated the role of popularity in news recommendation systems. For instance, PP-Rec [Qi et al., 2021a] integrated personalized matching with popularity scores through a unified model for popularity prediction, incorporating a personalized aggregator and knowledge-aware news encoder to enhance ranking performance.

However, these approaches treat news popularity uniformly, failing to distinguish between conformity effects in highly popular news interactions and genuine user interests reflected in low-popularity news engagement. This undifferentiated treatment leads to suboptimal user interest modeling.

2.2 Causal Recommendation

Real-world interactions are fundamentally governed by causeand-effect relationships rather than simple correlations, as correlated events may not necessarily indicate causation [Gao et al., 2022]. Recent research has shifted focus toward understanding various confounding effects and counterfactual analysis [Zhang et al., 2021b]. For example, In addressing popularity bias for general recommendation systems, Zhang et al.[Zhang et al., 2021b] introduced PDA (Popularity-bias Deconfounding and Adjusting), which removes bias during model training and adjusts recommendations through causal intervention at inference time. Additionally, Wang et al. [Wang et al., 2022] proposed UCI (User-Controllable Inference), a framework that integrates causality to enable user alerts, control inputs, and flexible recommendation adjustment.

While these causal approaches have demonstrated promising results in general recommendation scenarios, their direct application to news recommendation systems faces two critical challenges: 1) User-News interactions typically involve multiple engagement points, such as tags and titles. Existing methods lack the capability to differentiate and quantify how popularity at each level influences user interactions; 2) User interactions with news of varying popularity levels carry different implications. Interactions with popular news may primarily stem from conformity effects or social influence, while engagement with less popular news might better reflect

3 ProblemDefinition

Given a user u and a news n, the news recommender system aims to calculate the matching score s to evaluate the likelihood of user u clicking on news n. Subsequently, the system ranks all candidate news based on the matching score s and recommends the top-ranked news to the user n. Each news n consists of title nt = [w1,w1, · · · ,wL] with L words and a specific category, denoted as nc . Each user u has a history list of behaviors ub = [n1, n2, · · · , nM] with the list length of M, representing the news he/she has previously clicked on.

4 ProposedMethod

In this section, we introduce our model's overall structure in detail. Specifically, the proposed method consists of three core modules: a Hierarchical Popularity-aware User Interest Modeling to learn hierarchical user interests with different news popularity, shown in Figure 3, a Dual-track Causal Debiasing to capture genuine user interests, shown in Figure 5, and an Adaptive Score Aggregator to predict the final user-news matching score.

4.1 Hierarchical Popularity-aware User Interest Modeling

In this section, we propose to capture personalized popularityaware user interests. Our method consists of two key components: (1) a hierarchical news encoder that learns comprehensive news representations by incorporating both title and category information, and (2) a popularity-aware user modeling mechanism that leverages a selective masking strategy to distinguish between popular and unpopular news interactions across different news granularities.

Hierarchical News Encoder In our method, news encoder aims to learn news representations. Previous works [Wu et al., 2023] usually use a variety of combinations of news information (e.g., title, category, sub-category, entity, etc.) to learn news representations, which have demonstrated superior performance. In fact, different aspects of news content, specifically titles and categories, represent distinct perspectives of the same news, each with notably different popularity distributions (as illustrated in 1). These varying popularity patterns would significantly influence user engagement in distinct ways. For instance, some users might be drawn to news based on popular categories, while others might be more influenced by trending titles.

Hence, we design a hierarchical news encoder that utilizes the Word2Vec [Mikolov et al., 2013] (denoted as encw) to learn the news category embedding and the BERT [Devlin et al., 2018]) (denoted as encb) to learn the news title embedding. Thus, the x-th news embedding of title and category with d dimension can be formulated as:

In this way, the model could capture news features at different granularities, which helps model users' diverse interests driven by the varying popularity of different news content.

Popular-aware User Representation Based on the learned news embedding nt and nc , a straightforward approach is to encode the category-aware user representation and the titleaware user representation separately and then concatenate them, which is denoted as:

where [n1, · · · , nM] denotes the embeddings of news in the user's click history and [·; ·] is the concatenation, and encu(·) is the user encoder.

However, this approach overlooks a critical aspect: the varying importance of news with different popularity levels in modeling user preferences. User engagement patterns vary fundamentally based on news popularity, i.e., clicks on popular news often result from a combination of conformity effects and genuine interest, while engagement with unpopular news typically indicates authentic personal preferences. Moreover, users exhibit different sensitivity to popularity across content granularities. Some users are more likely to click news due to trending titles, while others are primarily influenced by the popularity of news categories. This heterogeneous popularity influence necessitates separate modeling of user interests at both title and category levels.

To alleviate this, we propose to learn user representations by considering popularity differences at both title and category levels. Specifically, we employ a selective masking strategy that separates popular from unpopular content within each granularity. For example, for news's title embeddings, we could categorize the news in click history [n1, n2, n3, n4] into two classes:

in which pop and unp denote the news classified as popular or unpopular based on news popularity.

Then, based on the popularity of news title, we can obtain the title-aware hierarchical user interest, which could be calculated as follows:

𝑛∗

𝑢∗

𝑝∗

M

C

𝑀 = 𝑃(𝑀|𝑛∗ , 𝑢∗ , 𝑝∗ )

(a) The causal effect, with

the variables at the refer-

ence status: news n∗ , user

u∗ and popularity p∗ .

(b) The counterfactual world, in which user click-through intentions are solely influenced by popularity.

Figure 5: Model training and inference.

where nti denotes the title embeddings of the i-th news.

Similarly, we can obtain the category-aware hierarchical user interest representations upop c and uunp c .

Finally, four user's interest representations from four different views would be obtained: (1) popular news title view upop t ; (2) unpopular news title view uunp t ; (3) popular news category view upop c ; (4) unpopular news category view uunp c .

4.2 Dual-viewCausal Debiasing

While our previous modeling approach successfully captures user interests across diverse news content and popularity levels, challenges remain in obtaining unbiased interest representations. Specifically, two challenges persist: highly popular news can introduce conformity bias through herd effects, while less popular news often receives insufficient attention during model training due to limited user interactions. To effectively address these contrasting challenges, we propose a novel differential causal debiasing framework including: (1) counterfactual debiasing for popular news to mitigate conformity effects, and (2) inverse propensity weighting for unpopular news to enhance the influence of limited interactions.

I

Counterfactual Debiasing for Popular News In fact, the news contents and user interest determine the user-news interest matching. Similarly, the interplay between news popularity and user conformity shapes user conformity matching. And the final user-news clicking probability is determined by user conformity and interest. Notably, popularity's influence extends beyond immediate interactions, it plays a crucial role in shaping the user's historical click behavior, so there is an edge from Popularity to User. The causal graph is illustrated in Figure 2 (a).

Within our causal framework, the counterfactual framework aims to mitigate the impact of users' herd mentality on their clicks under the popular view. A feasible solution is to eliminate the influence of popularity within the causal graph. This is because when popularity no longer affects users' clicks, the causal effect generated by path (U, P) → C → M is determined solely by the users (which can be regarded as the average click-through rate of users). In the ranking task, for all candidate news items, this influence brought solely by users is the same and will not affect the final ranking result. Therefore, eliminating the effect of Popularity (P) would result in the ranking not being interfered with by users' herd mentality.

There are two distinct conformity-related pathways: indirect path P → U → C and direct path P → C. To analyze conformity causal influence, we first investigate the direct causal effect of all factors (i.e., P, N, U). We compare the aggregated score with the reference world, shown in Figure 4 (a), where all factors are set to the average of corresponding vectors. Hence, the Total Effect (TE) is employed to measure the causal effects of all factors, which is defined as:

Then, according to the definition, TE can be decomposed into two components: Total Indirect Effect (TIE) and Natural Direct Effect (NDE). NDE represents the influence by the path P → C, which means the difference between the origin variable p and reference variable p∗ without the influence of user and news:

And hence, TIE could be computed by subtracting NDE from TE:

where TIE represents the indirect effect (P → U → C) without the influence of popularity.

We can find that P(M|u, n, p) represents the final prediction score, and P(M|u∗ , n∗ , p) is a counterfactual result, as demonstrated in Figure 4 (b), which means whether the user will click the news only based on the popularity. As news content and user interest at the reference status, we could assume that the counterfactual result is only determined by news popularity and user conformity:

where popn is the real popularity of the news n. u and n denote the mean of all the user and news vectors, which could be ignored. To quantitatively assess the causal impact of conformity on user clicks, we apply the MLP to predict the conformity score popu of user u, formulated as:

In this way, we can enable the estimation of model parameters for both P(M|u, n, p) and P(M|u∗ , n∗ , p) through supervised training, as detailed in Section 4.4. Through the counterfactual learning on the user popular-view histories, the model capture more genuine interests without conformity.

Inverse PropensityWeighting for Unpopular News In the context of unpopular news, user engagement patterns differ significantly from those of popular content. As depicted in Figure 2 (b), users' interactions with unpopular news tend to be driven more by genuine interest rather than conformity bias. However, this presents a challenge in our training: unpopular news receives limited exposure in our training data, potentially leading to under representation in the model learning process. Besides, the popularity influences exposure rates, creating an imbalance between exposed (treatment) and unexposed (control) news.

To address this bias, we adopt an Inverse Probability Weighting (IPW) approach. We propose a weighting mechanism that adjusts for this imbalance by calculating an exposure-based propensity score for each user:

where imprni denotes the impression number of news ni, γ is the hyper-parameter, Du unp and D represent the unpopular news set for the user u's click and total news dataset.

Finally, each user-news match score calculated from the unpopular news embeddings would be re-weighted by the following approach:

In this way, the model could effectively compensate for the exposure bias by giving more weight to interactions with unpopular news items, thereby improving the model's ability to capture genuine user interests.

4.3 Adaptive Score Aggregator

Given the four different view user embeddings, upop t , uunp t , upop c , uunp c , we compute two title-awarematching scores based on the candidate news title representation ncandidate t and two category-aware matching scores based on the candidate news category representation ncandidate c . Then, we design an adaptive score aggregator to personalized fuse these four scores to obtain the final user-news matching score. For simplicity, we denote the representations of different view user representations as u, which could be written as:

The final fusion matching score P(M|u, n, p) could be denoted by:

where Ws(u) is a MLP and softmax layer to compute the fusion weights. Wu is obtained via Eq.13 when u ∈ {uunp t , uunp c } for unpopular-view news and equals 1 when u ∈ {upop t , upop c } for popular-view news.

4.4 Training & Inference

Training As denoted above, the model parameters for both P(M|u, n, p) and P(M|u∗ , n∗ , p) need to be trained.

For P(M|u, n, p), we utilize the cross-entropy objective following the previous works" which is denoted as:

where s+Agg and s-Agg final matching scores of positive and negative samples. D and Du represent the training dataset and candidate negative news set for the user u.

Similar to Eq. 16, we still apply the cross-entropy objective for training P(M|u∗ , n∗ , p), which is denoted as:

Then, we utilize a multi-task learning paradigm with a hyperparameter η to train these two loss functions, which is defined as:

Inference As previously analyzed, the popularity effect (path P → C → M) for the popular news should be removed for inference. The actual score without the popularity effect could be accessed via Eq. 9. P(M|u, n, p) is the click probability determined by user u, news n, and popularity p, which is the model output SAgg. Besides, P(M|u∗ , n∗ , p) is affected only by popularity with the user and news at reference. It can be simplified like Eq. 10, denoted as:

where C is an invariant constant derived from the reference status of user and news, which could be ignored in inference.

Moreover, since the IPWaims to re-weight the loss function to mitigate the sample bias, we remove the IPW weight in Eq. 15 for inference. The aggregating score for inference could be written as:

Hence, the final matching score without popularity interference can be calculated by the following:

where τ is the hyper-parameter.

4.5 Time Complexity

The HPDM user modeling framework introduces an MLP layer and an aggregator module (with time complexity O(5d2 ), where d denotes the embedding dimension) to the base user encoder. Consequently, the overall time complexity of our framework depends on the choice of news and user encoders. For instance, the NAML architecture exhibits O(Md +Nd2 ) complexity; NRMS demonstrates O(Md + 3Nd2 + Nd2 ) complexity; Our HPDM framework achieves O(Md+3Nd2 + Nd2 + 5d2 ) complexity, where M represents the candidate news number, and N denotes the length of the user's historical news sequence. Therefore, we argue that HPDMmaintains acceptable inference efficiency.

5 Experiments

5.1 Experiment Setting

We evaluate our model on two real-world benchmark datasets: MIND [Wu et al., 2020] and Adressa [Gulla et al., 2017]. Detailed statistics of both datasets are presented in Table 2.

We compare our method with several benchmark baselines. We select two representative groups of baselines for evaluation. The first group is the traditional news recommendation models, including: (1) NRMS [Wu et al., 2019b]; (2) NAML [Wu et al., 2019a]; (3) LSTUR [An et al., 2020]; (4) FIM [Wang et al., 2020]; (5) Hie-Rec [Qi et al., 2021b]; (6) PP-Rec [Qi et al., 2021a]; (6) UNBERT [Zhang et al., 2021a]; (7) NRMSBERT [Wu et al., 2021].

The second group consists of several model-agnostic methods of popularity debias in the recommender system: (1) IPS [Liang et al., 2016]; (2) MACR [Wei et al., 2021]; (3) PPAC [Ning et al., 2024]; (4) PA & PAD [Zhang et al., 2021c].

Following the previous works, we choose four widely used metrics to evaluate the performance of our method, including AUC, MRR, NDCG@5, and NDCG@10.

5.2 Main Results

The main results of our model are illustrated in Table 1. Based on these results, we have the following observations:

Firstly, the bert-based model (e.g., UNBERT, NRMSBERT) usually demonstrates better performance than other models without bert (e.g., NRMS, NAML). This is because the extensive pre-training on large-scale corpora enables bert to capture rich linguistic knowledge and semantic representations, which serve as a strong foundation for understanding news and user preferences.

Secondly, our experimental results demonstrate that incorporating popularity debiasing mechanisms into news recommendation frameworks yields substantial performance improvements. Specifically, by augmenting the baseline models (e.g., NRMS-BERT) with various popularity debiasing frameworks (IPS, MACR, PA, and PAD), we observe consistent and significant enhancements across all evaluation metrics on both the MIND and Adressa datasets. This is because these models could help calibrate the training signals by reweighting samples based on their popularity, preventing the model from overfitting to popularity patterns and focusing more on news intrinsic content and user preferences.

Thirdly, our proposed HPDM framework demonstrates superior performance compared to existing popularity debiasing approaches across both MIND and Adressa datasets. This superior performance can be attributed to two key modules: This is because: 1) Unlike existing approaches that rely on single-dimensional popularity metrics, our method introduces hierarchical popularity patterns (title-level, category-level), enabling the model to capture more nuanced user interests. For instance, when a user clicks on a news with an unpopular title but popular category, HPDM can better discern whether this interaction stems from genuine topic interest or category-level conformity effects. 2) Rather than applying uniform popularity adjustments, HPDM utilizes the dual-view causal debiasing structure that considers the nuanced popularity patterns of both popular and unpopular news news. This granular approach enables more precise calibration of recommendation signals, leading to better alignment with users' genuine interests.

5.3 Ablation Study

In this section, we conduct ablation studies to evaluate the contribution of each component within our framework. For experiments, we use the MIND dataset as the example dataset, and design six variants: (1) w/o PV removes the popular-view user representations ( upop t and upop c ) and the counterfactual module; (2) w/o UV removes the unpopular-view user representations (uunp t and uunp c ) and the inverse propensity weighting module; (3) w/o CF maintains user representations with different views but removes the counterfactual module; (4) w/o IPWmaintains user representations with different views but removes the inverse propensity weighting module; (5) w/o NE represents removing the title-aware news encoder and only employing the category-aware news encoder; (6) w/o CE represents removing the category-aware news encoder and only employing the title-aware news encoder.

The experimental results are shown in Table 3. We can find that: 1) The removal of hierarchical popularity level architecture components leads to more substantial performance degradation, with the unpopular view (UV) and popular view (PV) modules showing significant AUC decreases to 67.92 and 68.13, respectively, while the removal of counterfactual reasoning (CF) and inverse probability weighting (IPW) modules resulted in relatively moderate decreases to 69.51 and 69.28. This hierarchical pattern suggests the fundamental importance of our dual-view architecture in capturing user preferences, where UV effectively extracts genuine interest signals from less popular news items, and PV models the conformity influence in clicking popular news; 2) The notable performance drops associated with the removal of debiasing strategies underscore their crucial role in mitigating popularity bias, with CF simulating counterfactual user behaviors to reduce conformity effects and IPW ensuring balanced attention distribution across the popularity spectrum; 3) The performance degradation of w/o NE and w/o CE indicate the different importance of title perspective and category perspective in modeling news representation and user interest respectively. In all, these findings validate our framework's architectural, which could effectively combine different view user representation for capturing hierarchical popularity patterns with debiasing mechanisms.

5.4 Hyper-parameter Analysis

We investigate two critical parameters in the framework: γ for scaling exposure rate in inverse probability weighting, and τ for adjusting the debiasing weight in the counterfactual framework. The optimal value of γ was observed around

0.1, striking a balance between mainstream and niche user interests, while τ achieved peak performance at approximately 0.12. Notably, in our experiments, we find that performance degrades at τ = 0, which could demonstrate the necessity of counterfactual debiasing in news recommendation. Hence, we carefully set the γ as 0.1 and the τ as 0.12.

5.5 Popularity Debias Visualization

We use visualization to more intuitively analyze the debiasing effect of popularity from different views. In Figure 6, the standard (grey) represents the real click behaviors. Since these news have an interactive relationship with users, they need to be ranked first in terms of standard results. In the popularityview shown in Figure 6 (a), we observe that "pop-view rank without cf" (red) exhibits excessive sensitivity to popularity signals, potentially due to the confounding effects. As the popularity decreases, the recommended ranking is lower, and the recommendation effect is worse compared with the standard results. Our approach (orange) achieves more balanced ranking scores across the popularity spectrum. Similarly, in the unpopularity view shown in Figure 6(b), "unp-view rank without ipw" approaches struggle with sparse interaction signals from less popular news. The integration of inverse propensity weighting in our framework (blue) could effectively address this challenge. These observations validate our motivation to design a dual-track framework.

6 Conclusion

In this paper, we investigated the critical issue of popularity bias in news recommendation systems, with emphasis on its hierarchical manifestation across different news granularities, from fine-grained titles to coarse-grained categories. Our framework introduces two key modules: a multi-granular user interest modeling approach with selective popularity-aware masking mechanism that distinctively processes interactions at different granularity levels to capture more comprehensive user preferences, and a dual-track causal debiasing structure that employs counterfactual reasoning for popular news while leveraging inverse propensity weighting for unpopular news to effectively disentangle genuine user interests from popularityinduced behaviors. Extensive experiments on the MIND and Adressa datasets demonstrated the superior performance, validating its effectiveness in capturing genuine user interests while mitigating popularity bias at different view levels.