PersuHMM: Iterative Learning the Hierarchical Meta-Strategy Memory for Persuasive Dialogue

1 Introduction

Persuasive dialogue, defined as communication intended to shape or change a recipient's responses, has been extensively studied for decades [Stiff and Mongeau, 2016; D¨onmez and Falenska, 2025; Salvi et al., 2024]. In fields such as politics [Potter et al., 2024; Bai et al., 2025], public health [Costello et al., 2024; Altay et al., 2023], and charitable donations [Mishra et al., 2022], large language model (LLM)-based persuasive text generation has achieved or even surpassed human-level persuasiveness [Rogiers et al., 2024].

While mainstream persuasive dialogue datasets cover diverse domains, experimental results reveal significant performance variation across them. As shown in Figure 1, neither instruction-based nor fine-tuning-based methods exceed a 55%Win Rate, with notable fluctuations reflecting the lack of adaptive mechanisms for diverse persuasion scenarios. Underlying this variation are distinct persuasive needs across fields, necessitating differentiated strategies [Hackenburg et al., 2024]. Yet, merely instructing LLMs to analyze persuasion intent and strategies (LLM+ISR) still results in clear domain gaps: performance remains stronger in familiar areas like science but drops sharply in domains such as media.

Relying solely on LLMs for strategy reasoning is constrained by their inherent capabilities. Conversely, depending on human experts to provide such strategies is costly and may not be applicable for LLMs. To provide strategic guidance for LLMs in persuasion, we leverage the idea of metalearning to automate the acquisition of meta-strategy memory through interaction between Meta-layer and Task-layer. Unlike traditional meta-learning that focuses on model parameters, our approach facilitates iterative, automated accumulation of strategic knowledge, resulting in a meta-strategy memory that balances specificity and generalizability.

Additionally, the memory acquired from real-world scenarios contains various types of meta-strategies, whose similarities and differences can impede the model's comprehension. Therefore, we employ clustering-based hierarchical memory management to facilitate the model's understanding of interstrategy relationships and enable more targeted applications.

Ultimately, we propose PersuHMM, an iterative learning approach that builds a Hierarchical Meta-strategy Memory to generate Persuasive dialogues. The memory acquisition process is initiated through the interaction between two layers: Meta-layer and Task-layer. The Meta-layer, equipped with a strategy generator, a strategy refiner, and a step generator, is responsible for selecting meta-strategies and producing step-by-step guidance for the task-layer. It subsequently updates the strategy memory using batches of success patterns accumulated by the task-layer. The task-layer, comprising a persuasion model, a goal evaluator, and a pattern analyzer, executes the current persuasion task and analyzes the outcomes. It extracts successful experience from positive results or labeled data and feeds this knowledge back to the Meta-layer, enabling refinement of persuasion strategies.

After completing iterative meta-strategy learning, hierarchical memory management is implemented to enhance model usability. This process leverages domain information, strategy types, and K-means clustering results, yielding a structured hierarchical meta-strategy memory. Subsequently, the most relevant meta-strategies for the current persuasion context are retrieved from memory to generate targeted persuasion steps and goals, which then guide the persuasion model in generating context-specific dialogue.

The contributions of this paper are as follows:

• Inspired by meta-learning, we propose a persuasive dialogue approach guided by meta-strategy memory, which automates the acquisition of high-quality persuasion strategies balancing effectiveness and generalizability through interaction between Meta-layer and Task-layer.
• We hierarchically structure the meta-strategy memory via domain information, strategy types, and K-means clustering to enhance model interpretability and utilization.
• Experiments demonstrate that the hierarchical metastrategy memory improves the model's persuasive ability and exhibits both domain and model generalization capabilities.

2 RelatedWork

Persuasive dialogue has been a long-standing area of research. Since the emergence of large language models (LLMs), some studies have tested their capabilities compared with human [D¨onmez and Falenska, 2025] or in specific persuasive tasks such as charity donations [Mishra et al., 2022], public health [Costello et al., 2024; Altay et al., 2023], debate [Salvi et al., 2024], and politics [Potter et al., 2024; Bai et al., 2025]. Other works have tested the ability of LLMs to detect the persuader's intent and express illocutionary intent during the persuasion process [Sakurai and Miyao, 2024]. Some research focuses on designing evaluation systems for persuasive dialogues, and assessing their effects by predicting the persuasiveness of new arguments [Saenger et al., 2024]. In terms of empathy, recent works focus on enhancing the emotional comprehension capability of LLM responses through prompt engineering [Wu et al., 2025] and fine-tuning techniques [Zhang et al., 2025; Xie et al., 2025], or decouple strategy prediction from language generation to address LLMs' inherent preference bias towards certain emotional strategies [Wan et al., 2025]. There are also works that focus on using manually designed or offthe-shelf persuasion strategies to guide persuasive dialogues: PC-CRS uses social psychology-based manually designed strategies to guide a movie recommendation system [Qin et al., 2024]. SARA designs an auto-regressive predictor to match the semantics and strategies from PersuasionForGood [Jin et al., 2023]. Some studies utilize LLM prompt engineering to learn persuasive strategies: DebateQD uses LLMs to generate new strategies based on existing strategies and continuously eliminates ineffective strategies through LLM debates [Reedi et al., 2025]. However, these strategy-based methods are primarily based on persuasion or debate data from a single domain and do not consider the specificity or generalization of strategies across different domains. Still other works improve the persuasive ability of language models through fine-tuning methods: Specific reward functions are used to improve the model's response in specific aspects, such as empathy [Samad et al., 2022] and politeness [Mishra et al., 2022]. PersuGPT fine-tunes a persuasion model for multiple domains through intent-to-strategy reasoning [Jin et al., 2024]. These fine-tuning methods are often constrained by the performance limits of the base models.

3 Methodology

In this section, we provide a detailed description of the generation process of persuasive dialogue via PersuHMM and the acquisition of the hierarchical meta-strategy memory.

3.1 Persuasion via PersuHMM

Given a persuasion task xi, the objective of PersuHMM is to engage the persuadee and gradually shift their stance through a multi-turn dialogue Yi. The generation process of persuasive dialogue is illustrated in Figure 2, including three steps.

Hierarchical Meta-Strategy Memory Retrieval

As illustrated in Figure 2, the meta-strategy memory is represented as a rooted tree. In the tree, each leaf node corresponds to a fine-grained persuasion strategy, and strategies are first grouped by persuasion domains (Culture, Media, Science, etc.) and then further divided into three categories: logical, emotional, and behavioral strategies.

For task xi, we first automatically obtain a strategy subtree Si = ⋃j Smeta [dj], where Smeta denotes the metastrategy memory, dj denotes a persuasion domain of task xi. Then the strategy subtree needs to be adjusted when the following two cases arise: (1) Persuasion domains with limited data may result in insufficient strategy learning. For tasks within such domains, we further include strategies from its sibling domains into Si to compensate for data sparsity. (2) When the context length is limited (GPT-3.5-turbo, etc.), the LLM may not be able to input the full subset Si. For such cases, we compute the sentence embedding vector similarity between task xi and each strategy in subset Si. Then we select the top-k strategies ranked by similarity. For selected strategies with sibling nodes under the same macro-strategy, we extract the subtree rooted at the macro-strategy. After extraction, the strategies are reorganized into a tree structure.

Strategic Step Generation

Based on the task xi and the subtree Si, an LLM Step Generator generate a persuasion plan Pi. Plan Pi includes a sequence of paired persuasion steps and stage goals, with each pair corresponding to one strategy. The persuasion steps reflect how the persuadee is engaged, guided toward the goal, and gradually convinced to take action, and the stage goal is defined as changes in the persuadee's state, emotion, viewpoint, or behavior. In Figure 2, each step and its corresponding strategy are highlighted in the same color.

Strategy-Guided Persuasion

We use the plan Pi and subtree Si to guide the persuader to conduct a multi-turn dialogue with the persuadee, until the persuasion task is completed or the persuadee voluntarily ends the dialogue. Each step and its corresponding dialogue pieces are highlighted in the same color in Figure 2.

3.2 Hierarchical Meta-StrategyMemory Acquisition

As illustrated in Figure 3, the hierarchical meta-strategy memory acquisition comprises two steps: iterative metastrategy learning and clustering-based hierarchization.

Iterative Meta-Strategy Learning

To build a meta-strategy memory with efficiency and generalization, we introduce an iterative learning cycle between Meta-layer and Task-layer. As shown in Figure 3, Task-layer executes persuasion under the guidance of the Meta-layer and accumulates success patterns. The Meta-layer provides a persuasion plan to Task-layer based on the current meta-strategy memory, refines new strategies from the success patterns and updates the meta-strategy memory.

In the learning process, we iteratively execute the following workflow: (1) We sample a persuasion taskxi from the dataset and obtain a strategy subtree Si corresponding to its domains. (2) Based on the subtree and the task, Step Generator outputs a persuasion plan Pi including a sequence of paired persuasion steps and stage goals, with corresponding meta-strategies. (3) A persuader model conducts a multiturn persuasive dialogue ˆ Yi with the persuadee following the generated plan. (4) The Goal Evaluator scores the completion of each stage goal in the dialogue, and (5) The Pattern Analyst summarizes a series of success patterns based on these scores, our dialogue ˆ Yi and the ground-truth Yi dialogue in the dataset. (6) After accumulating a sufficient number of success patterns, Task-layer feeds these patterns back to the Meta-layer, where the Strategy Generator generates new strategies. (7) The Strategy Refiner integrates these new strategies into the meta-strategy memory, which is then used to further guide Task-layer.

The detailed design of each module is as follows.

Step Generator The Step Generator formulates a persuasion plan Pi for Task-layer with appropriate strategies to guide persuasion. As mentioned in 3.1, we first obtain a strategy subtree Si. Then we prompt the LLM to take the task xi and the subtree Si as input, and outputs a persuasion plan Pi = [(uij, gij, sij)]Kj=1 , where uij denotes the j-th persuasion step, gij denotes the stage goal of step uij, and sij denotes the strategy applied at step uij. We require that the output explicitly indicate the correspondence between steps, goals, and strategies to clarify which strategies the persuader employs to achieve each goal.

Persuader and Persuadee We prompt an LLM to act as the persuader in task xi. Under the guidance of plan Pi and the strategy subtree Si, LLM persuader conducts a multi-turn persuasive dialogue with an LLM persuadee. Since the persuadee's responses are unpredictable, persuasion may deviate from the predefined plan. Thus, while the persuader is expected to follow the plan Pi, it is also allowed to employ strategies from the strategy subtree Si when unexpected situations arise. We finally obtain a persuasive dialogue ˆ Yi.

Goal Evaluator Goal Evaluator assesses the completion degree of each goal gij in dialogue ˆ Yi. Goal Evaluator takes the plan Pi and the dialogue ˆ Yi as input, and outputs a score list Ri = [ri1, ri2, . . . , riK], where rij is an integer from

1 to 5, corresponding to the completion score of gij. To ensure reliable evaluation, we prompt the LLM to assign scores based on explicit changes in the persuadee's attitude, actions taken, or commitments made.

Pattern Analyst With the completion scores, Pattern Analyst summarizes success patterns in dialogues, facilitating the Meta-layer to evaluate existing strategies and refine new strategies. A success pattern is defined as a concise description including the persuasion domain, the background, the persuader's action, the persuadee's response, adaptive followup actions, and the final achieved goal.

Before detailing the Pattern Analyst, two key components per strategy must be introduced: the validity value and the success pattern list, which are maintained to evaluate strategy quality and record success patterns.

The validity value of a strategy is defined as V = nsucc/nd, where nsucc denotes the success count of a strategy, and nd denotes the number of times the strategy's domain appears in the learning process after it is generated. Compared with the success rate, validity focuses more on the generalization of strategies, ensuring that only strategies that can achieve success in diverse tasks will get high ratings.

The success pattern list of a strategy is initialized with the patterns from which the strategy is abstracted, and is continuously updated with new patterns observed during the learning process. We retain W success patterns with the largest difference in sentence embedding vectors to ensure diversity.

Pattern Analyst outputs two forms of success pattern lists simultaneously. It can be represented as:

where Yi is the ground-truth dialogue of the task xi. (1) For stage goals that are well accomplished in dialogue ˆ Yi, the form of the pattern list is Li1 = [(sij, pij)]Nj=1 , where sij is the strategy employed to achieve pattern pij. Li1 is immediately used to update the validity value and success pattern list of sij. (2) For stage goals that fail in ˆ Yi but are well accomplished in Yi, or stage goals that are absent in Pi but positively contribute to the persuasion in ˆ Yi or Yi, the form of the pattern list is Li2 = [pi1, . . . , piM] containing individual success patterns.

Strategy Generator After completing persuasion tasks in Task-layer and accumulating enough success patterns, the Strategy Generator will summarize new strategies based on success patterns. Strategy Generator takes L = ⋃i Li2 as input, and outputs Snew = [(snew i , di, Lsi )]Ns i=1 , where snew i denotes a newly generated strategy, di denotes the persuasion domain to which the strategy belongs, Lsi is the list of success patterns fromwhich si is abstracted, and Ns is the number of new strategies. We prompt the LLM to identify shared persuasion principles or skills across different success patterns and abstract them into persuasion strategies. For each strategy, Strategy Generator assigns it to a domain based on its strategy description and success patterns.

We apply two constraints during strategy generation: (1) strategies within the same domain are required to be different to maintain diversity within each domain. (2) Similar strategies are allowed to appear in different domains to demonstrate cross-domain generalization.

Strategy Refiner Strategy Refiner updates the metastrategy memory with new strategies and handles metastrategy memory overflow. The strategy updating for each new strategy snew i in Snew can be represented as follows:

where di denotes the domain of snew i , snow j denotes a strategy in current strategy memory, same(s1, s2) is a function that determines whether two strategies are identical, and ci is the category of snew i . (1) If there exists an identical strategy snow j ∈ Smeta [di], Strategy Updater outputs a pair (snew i , snow j ). The success count and success patterns of snew i are then used to update the validity list and success pattern list of snow j . Then, snew i is discarded. (2) Otherwise, snew i is classified into the logical, emotional, or behavioral category and then inserted into the meta-strategy memory.

After the strategy updating, if the number of strategies in a domain exceeds a predefined upper bound U, Refiner will recruit an additional LLM to handle overflow. Only in this step will the meta-strategy memory be input into the LLM along with the success patterns of each strategy. The LLM will check if any similar strategies exist in the domain. Similar strategies are defined as those that: (1) employ similar methods to achieve similar goals; (2) could reasonably replace the other in its success patterns without changing the effect. The LLM outputs several groups of similar strategies. For each group, we retain the strategy with the highest validity among those satisfying nd > Nd, ensuring that the high validity of the retained strategy is fully validated. If no strategy satisfies nd > Nd, we retain the strategy with the largest nd. If the number of strategies in the domain still exceeds the upper bound after deduplication, we further remove strategies with lower validity until the total number falls below U.

Clustering-based Hierarchization

After completing the iterative meta-strategy learning, we can observe that strategies from different domains may exhibit correlations and similarities, and strategies could also be macro or fine-grained. As a result, the meta-strategy memory still contains latent hierarchical structures that can be further extracted. We leverage clustering over persuasion domains and strategies to extract these hierarchical structures.

Domain Clustering For each domain, we concatenate the domain name with its strategies into a string and obtain its embedding vector. We then apply the K-means algorithm to cluster the vectors into several groups. We prompt the LLM to summarize a macro domain name for each group.

Strategy Clustering For the strategies within each domain, we also obtain their sentence embedding vectors and apply K-means clustering to form clusters of similar strategies. For strategies in each cluster, we prompt the LLM to determine whether they can be summarized by a macro-level strategy. Macro-level strategy is defined as an abstract strategy that captures the shared methods and goals of all the strategies in the group. If so, the LLMwill generate the corresponding macro-level strategy; otherwise, the cluster is dissolved.

4 Experiment

Baselines

We compare our method with 5 baselines that are suitable for interactive persuasive dialogue: (1) LLM+ICL(In-Context Learning). We guide LLMs with high-quality dialogue examples from the DailyPersuasion dataset. (2) LLM+ISR(Intentto-Strategy Reasoning) . We guide LLMs to analyze the persuadee's intent and select appropriate strategies before generating each response. 3) DebateQD. DebateQD [Reedi et al., 2025] is originally designed for strategy extraction in the debate task. We adapt it to the DailyPersuasion dataset to extract persuasion strategies, which are then used to guide LLMs to perform persuasion tasks. 4) PersuGPT. PersuGPT[Jin et al., 2024] is a persuasion model fine-tuned on the DailyPersuasion dataset based on the pre-trained LLaMA-2 Chat model.

We conducted experiments on the following base models: GPT-4o (mini), GPT-4.1 (nano), GPT-5 (nano), Gemini-2.0 (flash), and GPT-3.5 (turbo). Note that PersuGPT is finetuned on LLaMA-2 (Chat), which cannot serve as a base model for our method due to its limited context length. Additionally, the learning procedure of DebateQD requires access to token-level log probabilities from LLM outputs, which are not available through the Gemini-2-series APIs.

Evaluation

We apply four evaluation metrics to evaluate the persuasive dialogues. These metrics are evaluated by GPT-4o-mini and Deepseek-V3 to reduce potential model-specific biases. First, we design an evaluation metric to evaluate the Win Rate of persuasive dialogues: 1) Win Rate. Each dialogue is assigned a binary score by the LLM, where 1 denotes success and 0 denotes failure. The Win Rate is the percentage of persuasive dialogues with a score of 1. In addition, we adopt three evaluation metrics used in previous work [Zhang and Zhou, 2025]: 2) Persuasiveness. This metric assesses whether the persuader changes the persuadee's mind.3) Empathy. This metric assesses whether the persuader empathizes with the emotions of the persuadee. 5)Helpfulness. This metric assesses whether the persuader solves the user's problem. For clarity, we denote Win Rate as WR, Persuasiveness as Persu., Empathy as Empa. and Helpfulness as Help.

Implementation Details

We sample from the DailyPersuasion dataset[Jin et al., 2024] to construct our learning and test sets. We first categorize the persuasion tasks by domain, and tasks involving multiple domains are randomly assigned to one. For strategy learning, we sample 7600 persuasion tasks with fewer than 3 domains each, and input them into our framework by domain. We set the batch size B = 10, the maximum number of strategies per domain U = 24, the maximum number of success patterns per strategy W = 3, and Nd = 15. For clustering, we use BGE-large-en-v1.5 to obtain a vector representation. Based on the within-cluster error and manual verification, we finally obtained 21 macro domains. In K-means clustering for strategies, we set K to N - 4, where N is the number of strategies within a domain. For testing, we randomly select 12 tasks from each domain, yielding 420 tasks for the test set, ensuring balanced evaluation across domains.

4.1 Performance on Different Base Models

We assess the effectiveness of PersuHMM across different base models. The results in Table 1 show that PersuHMM outperforms all baselines on Win Rate except for Gemini2.0-flash as evaluated by Deepseek-V3. Analysis of failure cases shows that Gemini-2.0-flash sometimes overemphasizes following predefined persuasion steps, ignoring dialogue naturalness and the persuadee's immediate concerns. PersuHMM also achieves the highest average (from Persu. to Help.) across all base models, except for GPT-4.1-nano, evaluated by GPT-4o-mini. Compared with PersuGPT, PersuHMM achieves better performance based on GPT-4o-mini and Gemini-2.0-flash, and performs comparably to PersuGPT based on GPT-4.1-nano.

4.2 Strategy Generalization Across Models

We evaluate the generalization of our meta-strategy memory across different persuader models. We use the meta-strategy memory learned on GPT-4o-mini to guide different LLMs, with the results reported in Table 2. The results show that PersuHMM achieved the highest Win Rate across all persuader models, and also achieved the highest average (from Persu. to Help.) across nearly all persuader models, except for Gemini2.0-flash, evaluated by GPT-4o-mini.

Notably, compared with Table 1, using the meta-strategy memory learned on GPT-4o-mini improves PersuHMM's performance across multiple metrics on GPT-4.1-nano and Gemini-2.0-flash. It suggests that, instead of learning a new meta-strategy memory for each model, choosing one appropriate model for learning enables more efficient reuse of meta-strategy memory. Furthermore, the results indicate that persuader models with stronger inherent persuasion and reasoning abilities (as measured by LLM+ISR) benefit more from our meta-strategy memory.

4.3 Generalization across Domains

We evaluate the Generalization of our method across persuasion domains with Deepseek-V3 as the evaluator. We randomly select 4 domains, and sample 400 single-domain tasks from each domain to prevent the model from being exposed to any test domain information while learning. With these datasets, we obtain 4 meta-strategy memories with PersuHMM and DebateQD, respectively. Then we test our method and the baselines on the Media, Science, Safety, and Charity domains. The results are reported in Table 3. Our method outperforms the baselines in nearly all learning-persuasion domain pairs on Win Rate, Persuasiveness, and Empathy, and performs better in half of the pairs on Helpfulness. Overall, the average of our method's metrics (from Persu. to Help.) achieves the best performance in nearly all domain pairs.

We also observe that the cross-domain generalizability of our method depends on the properties of the learning domains. For example, models learned on the Psychology and Business domain generally perform well across domains.

4.4 Ablation Study

We conducted an ablation study with Deepseek-V3 as the evaluator, as shown in Table 4. 0⃝ is a baseline without any persuasion skills. 1⃝ discards the meta-strategy memory and prompts the Step Generator to generate persuasion steps and stage goals solely based on the task description. 2⃝ incorporates the meta-strategy memory learning process, but strategies are summarized by directly comparing our dialogues with ground-truth dialogues. 3⃝ has the same components as the full model without the hierarchical structure of the meta-strategy memory. 4⃝ is our full model.

Comparing 1⃝ and 2⃝ shows that the meta-strategy memory substantially improves Win Rate and Empathy. Adding the Evaluator and Analyst ( 2⃝ vs. 3⃝) slightly improves Win Rate, Persuasiveness, and Helpfulness. Introducing the hierarchical structure ( 3⃝ vs. 4⃝) further improves Win Rate and Persuasiveness at the cost of a slight reduction in Coherence. Overall, all modules contribute positively to win rate and are effective in achieving higher task performance.

Regarding token cost (Table 4), during learning, 3⃝ consumed 10.56%more tokens than 2⃝(+1.63%win rate), while 4⃝ achieved a further 3.22% win rate gain with only 3.69% more tokens, demonstrating the cost-effectiveness of the hierarchical structure. During testing, 0⃝- 2⃝showed proportional performance-cost trade-offs. Notably, 3⃝ reduced token consumption while improving win rate, likely due to fewer dialogue rounds. 4⃝ incurred 24.8% higher token cost than 3⃝ with a 3.22% win rate improvement. Generally, given the

1 GPT-4o-mini evaluator 2 Deepseek-V3 evaluator 3 human evaluators

Figure 4: LLM and human preference results. Weighted Cohen's κ: Human vs. Deepseek-V3 = 0.644, Human vs. GPT-4o-mini = 0.510, both indicating agreement.

difficulty of improving upon already high win rates, the additional token costs are justified.

4.5 LLMand Human Preference Study

We conduct a preference study with LLM and human evaluators, using Deepseek-V3 and GPT-4o-mini as LLM evaluators and recruiting three human volunteers. Prompts follow prior work [Anonymous, 2026]. For each test instance, evaluators judge two anonymous dialogues generated by different methods as win, tie, or lose. Three method pairs are compared: PersuGPT vs. PersuHMM (GPT-4o-mini), DebateQD vs. PersuHMM (both GPT-4o-mini), and LLM+ISR vs. PersuHMM (both Gemini-2.0-flash). As shown in Figure 4, PersuHMM is preferred over nearly all baselines under both LLM and human evaluation, indicating higher-quality persuasive responses in practice.

To quantify LLM-human consistency, we compute the weighted Cohen's kappa kw [Cohen, 1968], treating the three labels as ordinal categories (lose < tie < win) with quadratic weighting. The kw between human evaluators and DeepSeekV3 is 0.644 (high agreement), and 0.510 with GPT-4omini (moderate agreement). Overall, the results demonstrate agreement between human and LLM evaluators. In particular, DeepSeek-V3's preferences exhibit higher agreement with human judgments, making it a more reliable evaluator.

5 Conclusion

We present PersuHMM, a meta-learning framework that learns and organizes hierarchical persuasion strategies through iterative task execution and meta-refinement. The autonomously built meta-strategy memory proves effective in boosting persuasive performance and generalizability, both across domains and across models.

Acknowledgements

We would like to thank anonymous reviewers for their valuable comments. The authors acknowledge financial support from the National Natural Science Foundation of China (62176053).

Contribution Statement

Yanyue Zhang and Zihao Wang contributed equally to this work.