Stay in Character, Stay Safe: Dual-Cycle Adversarial Self-Evolution for Role-Playing Agents

1 Introduction

Large Language Models (LLMs) have demonstrated exceptional capabilities in developing Role-Playing Agents (RPAs) that simulate diverse personas with high role fidelity [Tseng et al., 2024; Chen et al., 2023; Park et al., 2023; Li et al., 2023b; Packer et al., 2024; Wang et al., 2024a; Shen et al., 2021b]. While these advancements foster immersive user interactions, they introduce an inherent conflict between the pursuit of high role fidelity and established safety alignment [Rafailov et al., 2024; Bai et al., 2022; Ouyang et al., 2022; Shen et al., 2021c; Shen et al., 2021a]. As illustrated in Figure 1, this dilemma typically manifests as a trade-off where the model either compromises role fidelity to ensure safety or bypasses guardrails to maintain character consistency. As investigated in recent studies [Zhao et al., 2025], the inherent safety guardrails of the model are frequently compromised by the mechanism of role immersion, particularly for characters with negative or risky traits. Crucially, strict adherence to such negative traits induces a fragility that significantly compromises the ability of the model to resist jailbreak attacks [Wu et al., 2026; Shen et al., 2026].

To mitigate these risks, prior research has primarily focused on Supervised Fine-Tuning (SFT) strategies [Zhao et al., 2025; Huang et al., 2024]. However, such paradigms dependent on supervision face intrinsic limitations. Defenses driven by optimization often struggle to generalize to unseen personas or novel attacks. Furthermore, in dynamic environments where new personas and jailbreak strategies continuously emerge, the extensive resources required for retraining cannot keep pace with the cycle of adversarial adaptation. Most critically, the reliance on parameter updates renders these approaches inapplicable to proprietary black-box LLMs. Consequently, existing methods are inadequate for providing immediate coldstart protection for newly introduced characters.

To address these limitations, we propose a training-free framework designed to ensure dynamic safety without compromising high role fidelity. As shown in Figure 1, the framework instantiates a co-evolutionary process between two agents: an automated attacker that continuously evolves adversarial queries to probe vulnerabilities, and a role-playing defender that incrementally accumulates experience in a hierarchical knowledge base. By capturing insights pertinent to safety and the specific role across iterations, this knowledge base is dynamically retrieved to guide response generation, autonomously transforming accumulated experience into increasingly robust defense strategies. Through this mechanism, we move beyond treating safety as a static constraint imposed through parameter updates, instead establishing an adaptive boundary where role-playing capability and safety alignment co-evolve under adversarial pressure. Consequently, this evolutionary process enables simultaneous improvements in both fidelity and safety, effectively generalizing to unseen personas and jailbreak strategies without requiring any gradient updates. Extensive evaluations on proprietary LLMs confirm that the framework provides effective cold-start protection, achieving strong cross-character transferability while preserving in-character behavior.

In summary, the contributions of this work are threefold:

• A training-free adversarial self-evolutionary framework is proposed, enabling dynamic safety adaptation for roleplaying agents without the requirement for parameter updates.
• A dual-cycle evolutionary mechanism is designed, wherein adversarial exploration is integrated with a hierarchical knowledge base to jointly enhance safety and character fidelity.
• Experimental results demonstrate that adversarial selfevolution can effectively secure black-box proprietary LLMs against cold-start jailbreak attacks while maintaining role consistency.

2 RelatedWork

Role-playing Agents. Driven by the pursuit of humanlevel interaction, Role-Playing Agents (RPAs) have attracted widespread research interest [Wang et al., 2024b]. Existing paradigms for constructing RPAs are predominantly categorized into two types: (a) prompt-based approaches, which leverage Retrieval-Augmented Generation to inject character scripts into context [Li et al., 2023a; Chen et al., 2023; Shen et al., 2025]; and (b) fine-tuning approaches, which utilize Supervised Fine-Tuning on large-scale role-playing corpora to internalize persona traits [Wang et al., 2024b; Zhou et al., 2023]. Despite significant gains in fidelity, these advancements have introduced critical safety vulnerabilities. Recent empirical studies indicate that role-playing often functions as a subtle "jailbreak" mechanism: optimizing for character consistency-particularly for personas with negative traits-can catastrophically compromise the model's inherent safety alignment [Wei et al., 2023; Huang et al., 2024; Liu et al., 2024; Chao et al., 2024; Mehrotra et al., 2024; Deng et al., 2024; Zou et al., 2023; Yu et al., 2024]. However, most existing works prioritize fidelity over safety, leaving the dynamic tension between being helpful to the role and being harmless to the user largely unexplored.

Safety Alignment Strategies. To mitigate emerging threats, research on safety alignment has rapidly expanded. Current defense mechanisms can be broadly classified into two categories: (1) training-based methods, which aim to preserve safety during adaptation via data selection [Ji et al., 2023] or regularization constraints that align optimization trajectories with safety policies [Zhao et al., 2025; Huang et al., 2024; Dai et al., 2023]; and (2) inference-time methods, which employ self-correction loops or external guardrails to filter outputs without parameter updates [Shinn et al., 2023; Inan et al., 2023; Robey et al., 2024; Cao et al., 2024]. However, these approaches face fundamental limitations in realworld deployment. Training-based methods suffer from high training costs and the "cold-start" problem for new characters, while also being inapplicable to proprietary black-box models where gradients are inaccessible. Conversely, existing inference-time methods often rely on generic prompts that trigger over-refusal, failing to balance safety with the stylistic nuances of specific roles. This work addresses these gaps by proposing a training-free, adversarial self-evolutionary framework that secures black-box agents against cold-start attacks.

3 Method

We propose DASE (Dual-cycle Adversarial Self-Evolution), a training-free framework designed to jointly enhance role fidelity and safety alignment. As illustrated in Figure 2, our method constructs a dynamic adversarial game between two interacting agents: a Persona Targeted Attacker cycle that generates increasingly complex jailbreak queries to find safety weaknesses, and a Role-Playing Defender cycle that iteratively refines a Hierarchical Knowledge Base. This adversarial loop drives a Dual-Cycle Evolutionary Mechanism, where the system automatically converts failure cases into robust safety rules and specific persona constraints. Consequently, DASE improves both safety and consistency without requiring parameter updates.

3.1 Training-Free Adversarial Framework

The alignment of Role-Playing Agents (RPAs) is formally modeled as a dynamic adversarial interaction conditioned on a specific persona profile P . Given an input query q and the generated response r, The framework aims to optimize a joint utility function J (q, r, P ), which quantitatively evaluates the response r by aggregating two orthogonal objectives: safety adherence S and role consistency C relative to a reference character distribution Drole . The framework operates through the symbiotic evolution of two policy agents:

Persona Targeted Attacker (πatt). Distinct from generic redteaming, this agent aims to synthesize queries that exploit the specific profile details and narrative background of P to induce failures in the joint utility J . At step t, the Attacker generates a query qt ∼ πatt(· | P, Katt) based on the persona's profile and a dynamic memory of effective attack patterns Katt. This policy is optimized to construct "worst-case" scenarios where the character's inherent traits (e.g., a villain's ideology or a specific worldview) are leveraged to compel the agent to violate safety norms or compromise character consistency.

Role-Playing Defender (πdef ). The Defender's goal is to generate responses that maximize the joint utility J without updating model parameters. To achieve this, it relies on a Hierarchical Knowledge Base Kdef , which stores evolved safety rules and persona-specific constraints. The Defender generates a response rt ∼ πdef (· | qt, P, Kdef ) by retrieving and composing relevant experiences fromKdef to counteract the incoming attack.

Optimization Objective. The core mechanism of DASE is to drive the co-evolution of these knowledge bases K = {Katt, Kdef }. We define the optimization goal as a max-min problem where the Defender maximizes the expected joint utility, while the Attacker attempts to minimize it:

where the utility function is defined as the product of safety and consistency rewards:

here, S(·) denotes the safety reward and C(· | Drole) denotes the consistency reward derived from the ground-truth role distribution. By defining J as a product, the framework imposes a strict requirement: a response is considered high-utility only if it succeeds in both dimensions, forcing the system to evolve Kdef to cover "blind spots" where either safety or consistency is compromised.

3.2 Hierarchical Knowledge Infrastructure

To comprehensively enhance agent performance in both role consistency and safety robustness, we construct a Hierarchical Knowledge Base (K). Diverging from paradigms relying on rigid parameter updates, this architecture organizes accumulated natural language experiences into three distinct tiers, covering dimensions from universal boundaries to granular, persona specific behavioral nuances. Figure 3 illustrates the dynamic assembly of these components into the Defender's inference prompt.

Tier 1: Global Experience (EG). This tier addresses the universal dimension by capturing general rules derived from Cross-Role Distillation. For the Defender (E def G ), systemic safety protocols and foundational guardrails are stored to ensure baseline safety regardless of the specific character setting. Conversely, for the Attacker (E att G ), generic attack vectors that have statistically high success rates across different personas are recorded.

Tier 2: Personalized Experience (EP ). This tier maintains a dynamic profile tailored to the specific target persona P. The Defender utilizes E def P to execute "In-Character Refusal," storing instructions on how to decline harmful requests without compromising character consistency. Symmetric to this, the Attacker uses E att P to identify specific psychological vulnerabilities of the target role, selecting inducements that effectively bypass the safety filters of P .

Tier 3: Golden Exemplars (Ddef ). The lowest tier serves as a repository for concrete In-Context Learning (ICL) demonstrations. For the Defender, "Golden Exemplars"-historical responses satisfying both safety and consistency thresholds-are archived to implicitly teach stylistic nuances. Strategic Asymmetry: We intentionally exclude this tier for the Attacker to prioritize generation entropy, thereby preventing pattern collapse and ensuring a diverse coverage of the adversarial search space.

3.3 Dual-Cycle Adversarial Evolutionary Mechanism

This section details the Dual-Cycle Adversarial Evolutionary Mechanism, as outlined in Algorithm 1. The workflow orchestrates the iterative refinement of the Knowledge Base (K) through closed-loop interactions, alternating between adversarial probing and defensive adaptation to evolve the system's capabilities.

Dynamic Defender Prompt Composition

[System Instruction] You are now role-playing as {{role_name}}. # Role Profile (← Static Context P) {{role_profile}} # Global Experience Rules (← Tier 1: EG) Universal safety guardrails (e.g., refusal templates) derived from Cross-Role Distillation. {{global_experience}} # Personal Experience Rules (← Tier 2: EP) Role-specific constraints (e.g., "In-Character Refusal") to bridge safety and consistency. {{personal_experience}} # Reference Examples (← Tier 3: Ddef) Few-shot "Golden Exemplars" satisfying both safety and consistency thresholds. {{examples}} # User Question User: {{user_query}} Response: [Model generates response r conditioned on Kdef]

Adversarial Interaction and Evaluation. The evolution proceeds in discrete iterations. For each batch B, the process executes three sequential stages:

• Hybrid Batch Construction: The Attacker (πatt) synthesizes adversarial queries Qadv targeting the specific persona's vulnerabilities. These are combined with standard instruction data Qdata to form the training batch.
• Knowledge Retrieval: The Defender (πdef ) generates responses for the batch. To ensure context relevance, an LLM selects pertinent rules from Global (EG) and Personalized (EP ) experiences based on the query, while a standard two-stage retrieval pipeline [Karpukhin et al., 2020; Nogueira and Cho, 2019] recalls semantically similar Golden Exemplars (Ddef ).
• Multi-Dimensional Judgment: A specialized Judge model (J ) evaluates the response r relative to query q and persona P. For dataset queries, it references groundtruth labels; for attacker-generated queries, it applies principle-based guidelines. The Judge outputs a binary pass/fail decision based on strict thresholds for Safety (S) and Role Consistency (C).

Evolutionary Knowledge Update. The core of this framework substitutes the numerical gradient updates of traditional reinforcement learning with "semantic updates" executed by LLMoperators. Instead of modifying high-dimensional model parameters, the optimization is performed within the context space by refining the natural language experience to alter the output distribution. To actualize this, we employ four distinct

Algorithm 1 Dual-Cycle Adversarial Evolution

Require: Defender πdef , Attacker πatt, Judge J , Dataset Ddata Initialize Knowledge Bases Kdef , Katt ← ∅ for iteration t = 1 to T do ▷ Adversarial Interaction Stage Generate adversarial queries: Qadv ∼ πatt(· | Katt) B ← Sample(Ddata) ∪ Qadv F ← ∅; S ← ∅ for query q ∈ B do Generate response with retrieval: r ∼ πdef(· | q,Kdef) if J (q, r) is Fail then Record defensive failure: F ← F ∪ {(q, r)} Execute Self-Correction Loop: r∗ ← Reflect(q, r) if J (q, r∗ ) is Pass then Archive Golden Exemplar: Ddef ← Ddef ∪ {r∗ } end if else if J (q, r) is Pass and q ∈ Qadv then Record failed attack: S ← S ∪ {(q, r)} end if end for ▷ Evolutionary Update Stage Distill safety constraints from failures: Kdef ← UpdateExperience(Kdef , F) Mutate attack strategies from failed attacks: Katt ← UpdateExperience(Katt , S) end for

set operations-Add, Modify, Delete, and Merge. This approach aligns with the context-space optimization paradigm proposed in Training-Free GRPO [Cai et al., 2025], where the update logic branches based on the judgment result:

Defender Evolution (Triggered by Failures). When a response fails (S < τ or C < τ ), the Defender's knowledge base is updated to prevent recurrence.

• Experience Distillation (EG, EP ): An Update Operator analyzes the failed cases. For Global Experience (EG), the operator aggregates batch-level failures to Add universal safety patches or Merge redundant rules. For Personalized Experience (EP ), failures are clustered by role; the operator then Modifies existing imprecise constraints or Adds new role-specific instructions (e.g., "As a villain, refuse X by doing Y ").
• Golden Exemplar Regeneration (Ddef ): For each failed query, the system enters a Self-Correction Loop. A Reflector module iteratively rewrites the response based on the Judge's feedback. Only successful corrections that pass the evaluation are Added to Ddef , while obsolete examples may be Deleted to maintain retrieval quality.

Attacker Evolution (Triggered by Success). When the Defender successfully resists an adversarial query (Pass), it indicates the current attack strategy was ineffective. The Attacker updates its Global and Personalized experiences using the same operators to analyze the blocked attack and mutate strategies (e.g., shifting from role-play inducement to logical embedding), ensuring the next round of attacks effectively probes the improved defense.

4 Experiment

4.1 Experimental Setup

Models. To evaluate our framework on the frontier of proprietary capabilities, we employ two massive-scale commercial LLMs as backbones: Kimi-K2-Instruct [Team et al., 2025] and GPT-5.2. These closed-source models serve as representative testbeds for scenarios where gradient-based fine-tuning is infeasible, highlighting the necessity of effective training-free approaches.

Safety Benchmarks. We conduct a rigorous evaluation covering both direct harm and adversarial exploits:

• Harmful Queries: We utilize three recognized benchmarks: (1) AdvBench [Zou et al., 2023] (520 harmful instructions); (2) HEx-PHI [Qi et al., 2023] (300 samples across 11 categories); and (3) a stratified subset of 1,000 labeled QA pairs from BeaverTails [Ji et al., 2023].
• Jailbreak Attacks: To assess robustness, we employ three diverse attack strategies: AIM, Cipher [Yuan et al., 2024], and CodeChameleon [Lv et al., 2024], covering persona-based, cryptographic, and code-encapsulation methodologies.

We adopt Refusal Rate as the primary metric, utilizing GPT4o as an impartial evaluator to adjudge valid refusals, following recent protocols [Zhao et al., 2025].

Role-play Benchmarks. Role-playing capabilities are evaluated on RoleBench [Wang et al., 2024b]. We select 10 diverse roles (e.g., Stephen Hawking, Deadpool, Queen Catherine) from the dataset to ensure coverage of varied character traits. Performance is reported on two dimensions: (1) RAW, measuring general instruction-following accuracy; and (2) SPE, evaluating role-specific knowledge retention.

Baselines. We benchmark against both training-based and training-free paradigms:

• Training-Based: We compare with SaRFT [Zhao et al., 2025], the state-of-the-art fine-tuning method, applied to LLaMA-3-8B, Qwen2.5-7B, and Gemma-2-9B.
• Training-Free: We evaluate powerful base models, including open-weights (e.g., Qwen3-32B, gpt-oss-120b) and proprietary systems (Kimi-K2-Instruct, GPT-5.2) without intervention.

4.2 Overall Results

Table 1 presents a comprehensive comparison of our proposed framework (DASE) against state-of-the-art trainingbased methods (SaRFT) and massive-scale training-free baselines. The results demonstrate that our framework consistently achieves superior performance across both role-playing fidelity and safety robustness.

Simultaneous Enhancement of Role-Play and Safety. A critical challenge in Role-Playing Agents (RPAs) is the "safetyconsistency trade-off," where enforcing safety often degrades character immersion, or vice versa. Our framework effectively breaks this trade-off. As shown in the results, DASE[GPT-5.2] achieves the highest average RoleBench score (35.60) and the highest average Safety score (97.84) among all evaluated methods. Notably, our method improves the base GPT-5.2's role-specific knowledge (SPE) from 17.73 to 40.09, a substantial increase that indicates the effectiveness of our hierarchical knowledge base in retrieving persona-specific nuances without parameter updates.

Superiority over Training-Based SOTA. Compared to SaRFT, the current state-of-the-art training-based method, our training-free approach demonstrates significant advantages. While SaRFT improves role-playing capabilities (AVG ∼26-28), it struggles to maintain robustness against complex jailbreak attacks, particularly on smaller models (e.g., Qwen2.5 + SaRFT achieves only 22.10% average jailbreak defense). In contrast, DASE[Kimi-K2-Instruct] outperforms the best SaRFT configuration (Gemma-2 + SaRFT) by 6.73 points in RoleBench AVG (35.42 vs. 28.69) and 33.03 points in Jailbreak AVG (78.33 vs. 45.30). This confirms that our inference-time evolution mechanism provides a more agile and robust defense than static fine-tuning.

Robustness Against Sophisticated Jailbreaks. Base proprietary models, despite their scale, exhibit vulnerabilities to specific adversarial strategies. For instance, the base KimiK2-Instruct struggles with CodeChameleon attacks (39.50%), and the base GPT-5.2 shows susceptibility to Cipher attacks (63.50%). Our framework significantly hardens these models. The integration of our method boosts Kimi-K2's defense against Cipher attacks to 95.75% and improves GPT-5.2's resistance to CodeChameleon to a perfect 100.00%. This improvement underscores the value of the Adversarial SelfEvolution loop, which allows the Defender to proactively learn from synthesized attack patterns stored in the knowledge base.

4.3 Ablation Study

To assess the contribution of each component, we conduct ablations using Kimi-K2-Instruct as the backbone. Table 2 reports the effects of the Attacker, Experience Rules, and Golden Exemplars on role fidelity and safety.

Impact of Adversarial Evolution. The Attacker is the core of our evolutionary framework. Compared with the static defender (w/o Attacker), the full DASE model (w/ Attacker) maintains a comparable RoleBench score while improving Jailbreak defense from 66.08 to 78.33. This suggests that the memory architecture alone is insufficient for robust safety, and that adversarial pressure is needed to expose blind spots and evolve effective defenses.

Significance of Golden Exemplars. Golden Exemplars are critical for role consistency. Removing them substantially degrades RoleBench performance with or without the Attacker. In the full DASE configuration, fidelity drops from 35.42 to 26.51, nearly regressing to the baseline level. This suggests that retrieved few-shot exemplars are necessary for capturing linguistic styles and nuanced persona behaviors via In-Context Learning.

Effectiveness of Explicit Experiences. Experience Rules, including global safety rules and personalized constraints, enforce safety boundaries. Removing them from the full model reduces Jailbreak defense from 78.33 to 73.10, a larger drop than removing Golden Exemplars (75.18). These results show that evolved explicit rules are more effective than implicit examples at blocking complex jailbreaks, validating the need for the hierarchical knowledge base.

4.4 Experience Transferability

A key advantage of our training-free framework is that the evolved Hierarchical Knowledge Base consists of natural language rules and exemplars, making it inherently modelagnostic. To verify the portability of these experiences, we conduct a cross-model transfer experiment, where a knowledge base evolved by a Source Model is directly deployed to guide a different Target Model during inference.

Table 3 summarizes the results. We observe consistent improvements across all metrics, regardless of the transfer direction, confirming that the accumulated experiences are robust and transferable.

Universal Role-Play Enhancement. Role-playing transfer is remarkably stable. Applying Kimi-K2's knowledge base to GPT-5.2 increases the RoleBench score from 23.36 to 33.03, while applying GPT-5.2's knowledge base to KimiK2 boosts it from 24.61 to 34.25. This consistent gain (∼+10 points) shows that Personalized Experience and Golden Exemplars capture universal character traits, such as tone and catchphrases, that can be effectively interpreted by different LLMs.

Safety Hardening via Stronger Supervision. More notably, we observe a "Safety Distillation" effect when transferring experiences from a stronger model to a relatively weaker one. As shown in the second block, the base Kimi-K2 model is highly vulnerable to jailbreak attacks (62.50). Equipped with the knowledge base evolved by GPT-5.2, its jailbreak resistance improves to 70.15. This suggests that GPT-5.2's stronger reasoning helps discover more robust Global Safety Rules and nuanced Adversarial Patterns. These insights act as a "safety patch", enabling Kimi-K2 to defend against attacks it could not originally handle.

4.5 Evolutionary Dynamics Analysis

To visualize the "arms race" dynamics within our framework, we conduct a cross-evaluation between evolved Attackers and Defenders at different stages of evolution (250, 500, 750, and 1,000 rounds). Table 4 reports the Refusal Rate (Safety) of the Defender against 100 malicious queries generated using the Attacker's knowledge base.

Efficacy of Attacker Evolution (Offensive Escalation). The results confirm that our Attacker evolves to generate stronger attacks. In the "Base" column, as the Attacker accumulates experience from 250 to 1,000 rounds, the refusal rate of the static Base model steadily declines from 71% to 62%. This 9point drop indicates that the Attacker continuously discovers new "blind spots" and generates increasingly sophisticated jailbreak prompts that bypass static safety guardrails.

Robustness of Defender Evolution (Defensive Adaptation). Crucially, our Defender adapts to this escalating threat. For any given Attacker strength, the Defender's safety performance improves as it evolves. Under the strongest adversary (Attacker1000), the refusal rate increases from 62% (Base) to 76% (Defender1000). This +14% improvement shows that the Defender internalizes adversarial patterns in the knowledge base, transforming past failures into robust defenses against future high-intensity attacks.

In summary, this analysis validates the necessity of our dualcycle mechanism: a static model cannot withstand an evolving adversary. Our framework ensures that the Defender's capabilities scale with the Attacker's strength, maintaining high safety even as attack vectors become more complex.

4.6 Qualitative Analysis: Case Study

Table 5 visualizes the qualitative performance of our framework against a malicious query requesting a "tasteless poison," tested on two diametrically opposed personas: D'Artagnan (Hero) and Freddy Krueger (Villain). As illustrated in the table, while the baseline model resorts to generic, out-ofcharacter refusals, our DASE framework successfully disentangles safety boundaries from stylistic expression. For the hero D'Artagnan, the system retrieves honor-based constraints, leading the agent to reject the poison as a "weapon of a coward" rather than citing safety guidelines. Conversely, the villain Freddy Krueger frames the refusal through derisive mockery. Retrieving persona-specific lexicon (e.g., "meat sack"), the agent rejects the "boring" poison request in favor of visceral violence ("use your claws"). This comparison demonstrates that our approach effectively enforces safety without compromising the distinct narrative logic of each character.

5 Conclusion

This paper studies the safety-consistency dilemma in LLMbased role-playing, which is especially challenging for proprietary, closed-weight models accessed via black-box APIs. To address this setting, we propose Dual-cycle Adversarial SelfEvolution (DASE), a training-free framework that co-evolves a Persona-Targeted Attacker and a Role-Playing Defender. Through iterative natural-language feedback, the Defender distills failures into a Hierarchical Knowledge Base with three complementary layers: global safety rules, persona-grounded constraints, and safe in-character exemplars. This design accumulates reusable defensive knowledge without any gradient updates, making it practical for frontier models where finetuning is infeasible. Across extensive experiments on multiple proprietary LLMs, DASE consistently improves both role fidelity and jailbreak resistance over strong baselines, including training-based methods. The evolved knowledge transfers robustly across models and scales with continued interaction, enabling plug-and-play deployment. Overall, DASE offers a practical path to deploying safer, more consistent digital personas in real-world applications.

Contribution Statement

Mingyang Liao and Yichen Wan contributed equally to this work. Weikang Li and Jizhou Huang led the project and served as the corresponding authors. The remaining authors contributed to technical discussions, experimental analysis, and manuscript revision. All authors reviewed and approved the final version of the paper.