Towards Hierarchical Policy Learning for Conversational Recommendation with Hypergraph-based Reinforcement Learning

1 Introduction

Conversational recommendation systems (CRS) aim to dynamically learn user preferences by iteratively interacting with the user. Existing works have explored various settings

∗Corresponding author

(b)

(a) Framework of SCPR (b) Framework of Unicorn (c) Framework of DAHCR

Figure 1: Illustration of policy learning frameworks for CRS, including a framework of the outsourcing strategy (SCPR), a unified framework (Unicorn), and our proposed Director-Actor framework.

of CRS from the perspective of either dialogue systems [Li et al., 2018] or recommendation systems [Lei et al., 2018]. This work focuses on the setting of multi-round conversational recommendation (MCR) [Sun and Zhang, 2018], which aims to recommend the target item to the user by iteratively asking attributes and recommending items in the limited turns.

For each turn in CRS, the system naturally includes two essential decision-make procedures, when to recommend (i.e., ask or recommend), and what to talk about (i.e., the specific attribute/items). Early works [Lei et al., 2020a; Sun and Zhang, 2018] develop policy learning for a subset of decision procedures and outsource the other procedures to heuristic rules (SCPR as illustrated in Figure 1 (a)). These works isolate strategies for different decisions and make policy learning hard to converge due to their lack of mutual influence during training. To solve this problem, Deng et al. [2021] and Zhang et al. [2022] develop unified policy learning frameworks (Unicorn as illustrated in Figure 1 (b)) which unify the aforementioned two separated decision-make processes as a selection from the action space consisting of items and attributes. Despite effectiveness, the unified strategy brings out issues to be solved: (i) The unified strategy complicates the action selection of the CRS strategy by enlarging the action space and introducing data bias into the action space due to the imbalance in the number of items and attributes. As illustrated in Figure 1 (b), the action space is enlarged with all the items and attributes, and the strategy will prefer to select items when the number of items is larger. (ii) The unified strategy ignores the different roles of the two decision procedures, leading to the sub-optimal CRS strategy.

In the real scenario of CRS, the two decision procedures have different roles which are mutually influenced. As illustrated in Figure 1 (c), the decision procedure of when to recommend works as a director, which should select the option (i.e., ask or recommend) that is more effective for reducing the action space and acquiring user preferences (e.g., avoid recommending when the user's preference is not certain enough) to guide the latter procedure. The latter procedure works as an actor, which should accordingly choose the primitive action (i.e., the specific attribute/items) that satisfies the user's preference and gives feedback to evaluate the effectiveness of the director's option. The director's option limits the actor's action space to either attributes or items, which reduces the action space and avoids the data bias introduced by the imbalance in the number of items and attributes.

There remain three challenges in modeling these two roles and their mutual influence. The first challenge is weak supervision. The extrinsic rewards from the environment in each turn estimate the user's preference for the actor's primitive actions, but fail to estimate the effectiveness of the director's option, which is weakly supervised by the final-turn result (i.e., success or failure). The second challenge is user preference modeling. In the scenario of CRS, the user likes/dislikes items since they satisfy some attributes, which is a three-order relation (i.e., user-attribute-item). To specify the attributes that motivate the user to like/dislike the item, we should model user preferences with such high-order relations. The third challenge is the bad effect of model bias [Battaglia et al., 2018; Tarvainen and Valpola, 2017] on the mutual influence between director and actor. Specifically, the director's bias may lead to bad options that will filter out more efficient actions for the actor. And the actor's bias can result in false feedback that will disturb the convergence of the director.

To overcome the aforementioned challenges, we propose a Director-Actor Hierarchical Conversational Recommender (DAHCR) with the director to select the option (i.e., ask or recommend) that is more effective for reducing the action space and acquiring user preferences, followed by the actor accordingly choosing primitive actions (i.e., specific items/attributes) that satisfy user preferences. To train from weak supervision over the director's option effectiveness, we develop and introduce an intrinsic motivation (i.e., the actor's feedback) [Chentanez et al., 2004] into our Director-Actor framework to estimate the effectiveness of director's options. Furthermore, to model user preferences, we develop a dynamic hypergraph [Feng et al., 2019; Jiang et al., 2019] with each high-order relation (i.e., user-attribute-item) specifying an attribute that motivates the user to like/dislike the item. Finally, to alleviate bad effects of the model bias in the mutual influence, we model the director's option by sampling from a categorical distribution with Gumbel-softmax [Pei et al., 2022]. Extensive experiments on real-world datasets show that our method outperforms the state-of-the-art methods.

In a nutshell, this work makes the following contributions:

• We emphasize the different roles in two decision procedures for CRS, and the mutual influence between them.
• We propose a novel Director-Actor Hierarchical conversational recommender with intrinsic motivation to train from weak supervision and a dynamic hypergraph to learn user preferences from high-order relations. To alleviate the bad effect of model bias on the mutual influence between director and actor, DAHCR models the director's options by sampling from a categorical distribution with Gumbel-softmax.
• We conduct extensive experiments on two benchmark datasets, and DAHCR effectively improves the performance of conversational recommendation.

2 RelatedWork

Different from traditional recommendation systems [Zhao et al., 2022; Wang et al., 2022; Wei et al., 2023] that predict the user's preference based on his/her historical behaviors, conversational recommendation systems (CRS) [Priyogi, 2019; Xie et al., 2021; Zhou et al., 2020] aim to communicate with the user and recommend items based on the attributes explicitly asked during the conversation. Various efforts have been conducted to explore the challenges in CRS which can mainly be categorized into two tasks: dialogue-biased CRS studies the dialogue understanding and generation [Chen et al., 2019; Kang et al., 2020; Liu et al., 2020], and recommendationbiased CRS explores the strategy to consult and recommend [Christakopoulou et al., 2016; Christakopoulou et al., 2018; Sun and Zhang, 2018; Lei et al., 2020a]. This work focuses on the multi-round recommendation-biased CRS (MCR) [Lei et al., 2020a] which focuses on the setting where the MCR aims to recommend the target item to the user by iteratively asking attributes and recommending items in limited turns.

For each turn in CRS, the system naturally includes two make-decision procedures, when to recommend (i.e., ask or recommend), and what to talk about (i.e., which attribute/item to inquire/recommend). Early works for the MCR improve the strategies of when and what attributes to ask, while the decision of which item to recommend is made by external heuristic rules. EAR [Lei et al., 2020a] utilizes latent vectors to capture the current state of MCR, and employs policy gradient to improve the strategy of deciding when to ask questions about attributes and which attribute to ask. To reduce the action space in policy learning, SCPR [Lei et al., 2020b] improves the strategy to only decide whether to ask or recommend and develops external path reasoning methods to decide which attribute to ask or which item to recommend. These works, however, isolate strategies for different problems and make the policy learning of these strategies hard to converge. To solve this problem, Unicorn [Deng et al., 2021] unifies the two decision procedures as the selection from the candidate action space consisting of items and attributes. Specifically, Unicorn proposes a graph-based Markov Decision Process (MDP) environment to choose actions from the candidate action space. MCMIPL [Zhang et al., 2022] further considers the user's multiple interests in the unified strategy and develops a multi-interest policy learning module. Despite effectiveness, these works ignore the variant roles of different decision procedures, which may lead to sub-optimal CRS strategies.

3 The ProposedModel

We first introduce the problem definition of multi-turn conversational recommendation (MCR). Next, we introduce the framework and the model of our proposed Director-Actor Hierarchical Conversational Recommender (DAHCR).

3.1 ProblemFormulation

In this section, we formulate the problem of multi-turn conversational recommendation (MCR), which aims to recommend the target item to the user by asking attributes and recommending items in the limited turns of the conversation.

Specifically, let V = {v1, v2, · · · , vM} denotes the item set. For each item v, there exists an attribute set Pv associated with the item. At the beginning of each conversation, a user u initializes the conversation session with a target item v∗ and an attribute that belongs to the target item p0 ∈ Pv∗. The candidate item set Vcand is formed with the items associated with p0 and the candidate attribute set Pcand is constructed by the attributes associated with the items in the candidate item set Vcand. Then at each turn t, MCR can either ask the user an attribute pt ∈ Pcand or recommend a certain number of items (e.g., the top ten items) Vt ⊆ Vcand to the user. According to the target item v∗ and its associated attributes Pv∗ , the user will choose to accept or reject the proposal of MCR. Based on the user's feedback, MCR will update the candidate attribute set Pcand and the candidate item set Vcand. The conversation will continue until the max turn T and the recommendation is successful if the target item v∗ is recommended within T.

3.2 DAHCR Framework

As illustrated in Figure 1 (c), we propose the Director-Actor hierarchical conversational recommendation policy Learning, a novel framework for MCR. At each time step t, the director chooses an option ot ∈ O, and the actor chooses the primitive action at ∈ At|ot accordingly. Consequently, the state is updated to st+1 with the transition T(st+1|st, ot, at). The user's feedback ft ∈ {acc, rej}, the extrinsic reward ra t ∈ Ra , and intrinsic motivation ro t ∈ Ro are given according to st, ot and at. Specifically, the main components of DAHCR ⟨S, O, A, T, Ro , Ra ⟩ are defined as:

State S. The current state contains three components, including the interactive history I t , the related nodes N t , and the hypergraph G t among the user and related nodes:

where I t = [(a1j , fj)|j = 1, 2 · · · t - 1]. The related nodes N t = {u} ∪ P t acc ∪ P t rej ∪ V t rej ∪ V t cand contains the user u, the accepted attributes P t acc , the rejected attributes P t rej , the rejected items V t rej , and the candidate items V t cand .

Options O. Based on the state st of the current turn, the director should choose an option ot ∈ O, where O = {ask, rec} denotes whether to ask or recommend.

Primitive actions A. Based on the director's option ot and the state st, the actor selects the primitive action at ∈ At|ot , where At|ask = P t cand is the candidate attributes to ask and At|rec = V t cand means the candidate items to recommend.

Transitions T. As previous works [Deng et al., 2021], the state st+1 updates based on the user's response:

where VP t acc denotes the items that satisfy all the accepted attributes and PV t cand denotes all the attributes that belong to the candidate items. Follows Lei et al. [2020b], items that have rejected attributes are not eliminated from V t cand .

Extrinsic rewards Ra . Extrinsic rewards are special signals to guide the agent to select user-preferred actions. Five kinds of rewards are designed [Deng et al., 2021]: (1) ra acc|rec , a strongly positive reward for successful recommendation; (2) ra rej|rec , a slightly negative reward for rejected recommendation; (3) ra acc|ask , a slightly positive reward when the asked attribute is accepted; (4) ra rej|ask , a slightly negative reward when the asked attribute is rejected; (5) ra quit , a strong negative reward when the maximum turn reaches.

IntrinsicmotivationRo . The intrinsicmotivation is passed from the actor to the director to estimate the effectiveness of the director's option. Since recommending is an inefficient action when the user's preference is not certain enough (i.e., the target item is outside the top ten in the actor's ranking list of items), we assign a positive reward ro+ to the option of ask and a negative reward ro- to the option of rec in this situation. Inversely, when the user's preference is certain, ro+ and ro- are assigned to the option of rec and ask, respectively.

3.3 DAHCR Policy Learning

State Encoder

The interactive history I t = [(oj , fj)|j ∈ {1, 2 · · · t-1}] between DAHCR and the user contains the director's historical options oj and the user's feedback of accepting or rejecting fj ∈ {acc, rej}. With the embedding of the interactive history {X1h ,X2h , · · · ,Xt-1 h }, the interactive state sth is obtained with GRU networks [Cho et al., 2014]:

The interactive state sth encodes the historical interaction between the agent and the user, and is excepted to guide DAHCR to learn CRS strategy to decide whether the user's preference is certain enough (e.g., the recommended attribute is accepted for several turns) for recommending items.

To learn the user's preference for the specific attributes and items, we build a dynamic hypergraph G (t) u = (N (t) ,H(t) ,A(t) ), including: (1) the set of related nodes N t = {u} ∪ P t acc ∪ P t rej ∪ V t rej ∪ V t cand ; (2) a hyperedge set H(t) , whose element h ∈ H(t) denotes a hyperedge between the user, an attribute and items. In our case, for each attribute p ∈ N (t) , we define a hyperedge hp corresponding to the attribute p; (3) a |N (t) | × |H(t) | adjacent matrix A(t)

State Encoder

Hierachical Action Selection

which denotes the weighted edge between each node and hyperedge, with entries denoted as:

where phj denotes the attribute corresponding to the hyperedge hj , and V (t) hj indicates items that satisfy the attribute phj .

To take advantage of the connectivity information from the dynamic hypergraph, we employ hypergraph neural networks [Xia et al., 2022] to refine the node representation with structure and connectivity information. Firstly, we aggregate information propagated from nodes to related hyperedges:

where E ∈ R|N (t) |×d denotes the initial embedding of related nodes N (t) , Wn ∈ Rd×d is the weight matrix, and Dh is the diagonal matrix denoting the degree of the hyperedges, which is defined as the number of nodes connected by hyperedges.

During the conversation, the hyperedges are successively generated when the user accepts or rejects the asked attribute. Moreover, the higher-level interactions between different hyperedges are also important in learning user preferences. To model the sequential information and hyperedge-wise feature interactions, higher-level hypergraph layers further pass messages through the interactions between hyperedges as:

where f ∈ {acc, rej}, and MHSA(·) indicates the multihead self attention [Vaswani et al., 2017]. Finally, we aggregate the information from the hyperedges to refine the nodes' representations Γl and then obtain the connectivity state stg :

With the interactive state sth and the connectivity state stg , the final state is obtained by:

where ni denotes the node in the hypergraph, stlen encodes the size of the candidate item and attribute set by dividing the length |Vcand| and |Pcand| into ten-digit binary features [Lei et al., 2020a], since it is also an important basis for deciding to ask or recommend (e.g., the recommendation is easier to be successful when the candidate items' size is small).

Hierarchical Action Selection Strategy

After obtaining the state encoding the interactive history and the user's preference, we design a novel dueling Q-network to conduct policy learning under the hierarchical structure. Following the basic assumption that the delayed rewards are discounted by a factor of γ, we define the Q-value Qo(st, ot) and Qa(st, at) as the expected reward for the director's option ot and the actor's action at based on the state st:

where the value function f o θV (·) f a θV (·), and the advantage function f o θA (·) f a θA (·) are four separate multi-layer perceptions (MLP). P ∗ (st, ot) controls the action space of the actor's actions by masking the Q-value of actions at according to the director's option ot. To realize a differentiable discrete sample of the director's option and alleviate the bad effect of model bias on the mutual influence between the director and actor, we model ot by sampling from a categorical distribution with Gumbel-softmax as:

where ϵ = -log(-log(x)) and x is sampled from Uniform(0, 1). The temperature parameter τ controls the bias and variance of the likelihood distribution. When τ is larger, the likelihood is smoother with more variance and less bias. P(st, ot) is calculated with the softmax function as:

where O = {ask, rec} denotes the space for the director's option. Gumbel-Softmax is used to alleviate the bias when lacking policy learning in the mutual influence between Director and Actor: (1)Biases (e.g., bad options of Director) caused by Director may filter out of efficient actions for the Actor. Hence, we consider sampling options from a Gumbel distribution, which may pave the way to explore the right action for Actor; (2)Bias (e.g., false feedback) caused by Actor may affect the convergence of Director, and thus to alleviate the issue, we consider to optimize the Gumbel distribution, rather than the deterministic function of Director's option.

The optimal Q-function with the maximum expected reward Q∗o (st, ot) and Q∗a (st, at) for the director's option and the actor's primitive action are achieved by optimizing the hierarchical policy πo and πa, follows the Bellman function [Bellman and Kalaba, 1957] as:

where At+1|ot+1 denotes the action space of primitive actions according to the director's option (i.e., At+1|ask denotes candidate attributes and At+1|rec denotes candidate items).

Model Training

For each turn, the agent gets the intrinsic motivation ro t to the director's option, the extrinsic reward ra t to the actor's primitive action, and the candidate actions space At+1 is updated according to the user's feedback. We define a replay buffer D following Deng et al. [2021], which stores the experience (st, ot, at, ro t , ra t , st+1, At+1). For the training procedure, we sample mini-batch from the buffer and optimize the model with loss function as follows:

L1 and L2 are alternatively optimized to teach DAHCR efficiently interacts with the user and predict the user's preference, where θQ = {θV , θA}, and yo t , ya t are target values for the director's options and the actor's actions, which are based on the optimal value function as:

To alleviate the problem of overestimation bias, we adopt the double Q-learning [Van Hasselt et al., 2016] to employ target networks Q′o and Q′a as period copies from the online networks to train the network following previous works [Deng et al., 2021; Lei et al., 2020b].

4 Experiments

To fully demonstrate the superiority of our method, we conduct experiments to verify the following three research questions (RQ):

• RQ1: Compared with the state-of-the-art methods, does our framework achieves better performance?
• RQ2: What are the impacts of key components on performance?
• RQ3: How do hyper-parameters settings (such as the layer number of hypergraph neural networks) affect our framework?

4.1 Experiment Setting

Datasets

For better comparison, we follow previous works to conduct experiments 1 on LastFM, LastFM* for music artist recommendation and Yelp, Yelp* for the business recommendation. The statistics of datasets are illustrated in Table 1.

• LastFM [Lei et al., 2020a]: LastFM is designed to evaluate the binary question scenario for the music artist recommendation, where the user gives preference towards an attribute using yes or no. Following Lei et al. [2020a], we manually merge relevant attributes into 33 coarse-grained attributes.
• Yelp [Lei et al., 2020a]: Yelp is designed for enumerated questions for the business recommendation, where the user can select multiple attributes under one category.
• LastFM* and Yelp* [Lei et al., 2020b]: Following Lei et al. [2020b], we construct the datasets with original attributes and pruning off the attributes with frequency < 10, named LastFM* (containing 8438 attributes) and Yelp* (containing 590 attributes) separately.

Metrics

Following previous studies [Lei et al., 2020a; Lei et al., 2020b; Deng et al., 2021], we adopt three widely used metrics for conversational recommendation: SR@t, AT, and hDCG. Success rate (SR@t) is adopted to measure the cumulative ratio of successful recommendations by the turn t. Average turns (AT) is used to evaluate the average number of turns for all sessions. hDCG@(T, K) is used to additionally evaluate the ranking performance of recommendations. Therefore, the higher SR@t and hDCG@(T, K) indicate better performance, while the lower AT means an overall higher efficiency.

1 https://github.com/Snnzhao/DAHCR

Implementation Details

Following previous works [Deng et al., 2021; Lei et al., 2020b], we adopt TransE [Bordes et al., 2013] from OpenKE [Han et al., 2018] to pretrain the embedding of nodes in the constructed graph with the training set. The embedding size and the hidden size are set as 64 and 100. The temperature parameter τ is set to be 0.7 for Yelp* and 0.3 for the other three datasets. The layer number of hypergraph neural networks is selected from 1, 2, 3, and 4. We set the intrinsic motivations as: ro+ = 1, ro- = -1. The settings of the extrinsic rewards are the same as previous works [Lei et al., 2020a; Lei et al., 2020b; Deng et al., 2021]: ra rec acc = 1, ra rec rej = -0.1, ra ask acc = 0.01, ra ask rej = -0.1, ra quit = -0.3. In the training procedure, the size of the experience replay buffer is 50,000, and the batch size is 128. The learning rate and the L2 norm regularization are set to be 1e-4 and 1e-6, with Adam optimizer. The discount factor γ is set as 0.999. Our experiment is conducted on Nvidia RTX 3090 graphic cards equipped with an AMD r9-5900x CPU (32GB Memory).

4.2 Baselines

To demonstrate the effectiveness of the proposed DAHCR, we choose state-of-the-art methods for comparison. Specifically, we first choose two rule-based methods, three reinforcement learning-based methods that outsource part of decision procedures, and then two reinforcement learning (RL)- based method that unifies two decision procedures.

• Max Entropy [Lei et al., 2020a]. This method employs a rule-based strategy to ask and recommend. It chooses to select an attribute with maximum entropy based on the current state, or recommends the top-ranked item with certain probabilities.
• Abs Greedy [Christakopoulou et al., 2016]. This method only makes the item-recommendation actions and updates the model based on the feedback. It keeps recommending items until the successful recommendation is made or the pre-defined round is reached.
• CRM [Sun and Zhang, 2018]. A RL-based method that records the users' preferences into a belief tracker and learns the policy deciding when and which attributes to ask based on the belief tracker.
• EAR [Lei et al., 2020a]. This method proposes a three-stage solution to enhance the interaction between the conversational component and the recommendation component.
• SCPR [Lei et al., 2020b]. This method models CRS as an interactive path reasoning problem. It prunes irrelevant candidate attributes by traversing attribute vertices on the graph based on user feedback.
• UNICORN [Deng et al., 2021]. A RL-based approach that unifies the two decision strategies. It learns graphenhanced state representations for RL via graph neural networks.
• MCMIPL [Zhang et al., 2022]. A state-of-the-art approach to CRS that extends the unified conversational recommendation strategy with multi-interest representations of the user.

4.3 Performance Comparison (RQ1)

Overall Performance

From the overall performance of all methods reported in Table 2, we make the following observations:

• Our proposed DAHCR achieves the best performance. DAHCR significantly outperforms all the baselines by achieving a higher success rate and hDCG, and less average turn on four datasets. The reason for such improvement can be attributed to the following aspects: i) With the hierarchical strategy, our proposed DAHCR can reduce the action space and avoid the data bias introduced by the imbalance in the number of items and attributes. ii) The director and actor in DAHCR can well play their roles of choosing the effective option (i.e., ask or recommend) and learning the user's preference over the candidate items and attributes. iii) The intrinsic motivation can well deal with the problem of weak supervision on the director's option effectiveness. iv) Modeling user preferences with dynamic hypergraph, DAHCR could specify the attributes that motivate the user to like/dislike the itemwith high-order relations. v) Modeling the director's option by sampling from a categorical distribution can well alleviate the bad effect of model bias on the mutual influence between the director and the actor.

• Mutual influence of different decision procedures and the user's dynamic preference in the policy learning is important for CRS. From Table 2, DAHCR, MCMIPL and UNICORN surpasses CRM, EAR and SCPR in terms of three metrics over four datasets. There are two reasons for this performance: (i) CRM, EAR and SCPR isolate strategies for different decision procedures during the training process, which makes the conversational recommendation strategy hard to converge. This demonstrate the importance of the mutual influence between different decision procedures in the training process. (ii) Compared with other methods, DAHCR, MCMIPL and UNICORN use RL to update the user preferences on the attributes and items, which leads to better performance. This proves the importance of learning the user's dynamic preference for CRS.

Training Efficiency

Figure 3 shows the performance curves of the different methods tested on LastFM* and Yelp*, respectively. The test performance curves for the unsupervised methods Max Entropy and Abs Greedy are shown as two horizontal lines for comparison. It can be seen that DAHCR is far superior in converge speed and stability to all the baselines. The reason is that the hierarchical conversational recommendation strategy of DAHCR can realize more effective action choice and better learn the user's preference. In the Yelp∗ , where the action candidate space is larger, the performance of EAR and CRM does not improve much or even gets worse as the training process iterates. These results demonstrate the efficiency and effectiveness of the proposed DAHCR.

4.4 Study of DAHCR (RQ2&RQ3)

Next we investigate the underlining mechanism of our DAHCR with four ablated models that remove the hierarchical framework, dynamic hypergraph, Gumbel-softmax, and intrinsic motivation, respectively. From the results in Table 3, we observe that:

• The performance of DAHCR suffers a significant degradation when replacing the hierarchical framework with the unified framework. This demonstrates the importance of modeling the variant roles of different decision procedures in CRS.
• The performance of DAHCR drops when replacing the dynamic hypergraph neural networks with graph neural networks. We attribute this to the importance of high-order relations (i.e., user-attribute-item) in modeling user preferences for CRS.
• The model performs worse when removing the Gumbelsoftmax. This result suggests that alleviating the bad effect of model bias on the mutual influence between the director and the actor is necessary. Our method that models the director's option by sampling from the categorical distribution with Gumbel-softmax can reasonably deal with such an effect.
• The performance of the model drops when removing the intrinsic motivation, which indicates the necessity of intrinsic motivation to train DAHCR from weak supervision on the director's effectiveness.

Figure 4 shows the experimental results by varying the layer number of hypergraph neural networks. Two-layer DAHCR performs better than one-layer DAHCR since it can capture high-order collaborative information in the dynamic hypergraph. But the performance of DAHCR does not always increase when the layer number increases. We attribute this to the noise that increases along with the hop of neighbors.

5 Conclusion

In this work, we propose a Director-Actor Hierarchical Conversational Recommender (DAHCR) with the director to select the most effective option (i.e., ask or recommend), followed by the actor accordingly choosing primitive actions that satisfy user preferences. The intrinsic motivation is designed for training from weak supervision on the director's effectiveness, a dynamic hypergraph is developed to learn user preferences from high-order relations, and Gumbel-softmax is employed to alleviate the bad effect of model bias on the mutual influence between director and actor. Experimental results on two real-world datasets show that the proposed DAHCR outperforms the state-of-the-art methods.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant No.62276110, Grant No.61772076, in part by CCF-AFSG Research Fund under Grant No.RF20210005, and in part by the fund of Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL). The authors would also like to thank the anonymous reviewers for their comments on improving the quality of this paper.