BehaviorBench: A Psychologically Grounded Benchmark for Evaluating Personality in Large Language Models Through Realistic Behaviors

1 Introduction

Large language models (LLMs) have exhibited humanlike personality traits, even in the absence of explicit design [Miotto et al., 2022; Karra et al., 2022]. This emerging anthropomorphism has spurred significant interest in personality shaping [Caron and Srivastava, 2023], which is the deliberate alignment of an LLM's output with specific personality profiles, to enhance performance in applications such as role-playing agents [Zhang et al., 2025] and social simulation [Tu et al., 2023]. However, evaluating these shaping methods requires reliable frameworks to ensure their validity and effectiveness [Han et al., 2025].

Current evaluation paradigms predominantly adopt human psychological instruments, such as the Big Five Inventory (BFI) or Myers-Briggs Type Indicator (MBTI) questionnaires, by prompting LLMs to self-report their agreement with trait-descriptive statements [Huang et al., 2023b]. Although convenient, this approach is fundamentally flawed: in real-world applications, LLMs are expected to perform tasks and make behavioral decisions in context. However, these scales are designed to measure human personality through introspection of traits, rather than through concrete behaviors. Humans possess stable internalized personalities that ensure the coherence between self-reported traits and actual behaviors. In contrast, when LLMs self-report traits via scales, the explicit relationship between scale items and trait-based prompts changes the task to instruction-following. However, when making behavioral decisions, the implicit trait-behavior relationships require LLMs to perform the analysis. Therefore, evaluating personality through behavioral decisions can better assess the ability of LLMs to analyze such relationships and simulate the behaviors of specific personalities. Such inherent flaws create a mismatch between LLMs' self-reported traits and their real behavioral outputs, undermining the validity of the existing assessment paradigm [Han et al., 2025].

∗Corresponding author.

To address this, research has shifted the focus from labels to behavioral outputs, designing tasks to elicit and analyze LLM actions as proxies for personality [Lee et al., 2025;

Hartley et al., 2025; Han et al., 2025]. Although promising, existing behavior-centric approaches suffer from two limitations: weak theoretical grounding in trait-behavior mappings and low ecological validity of artificial scenarios, which undermine their reliability for real-world applications.

To overcome these limitations, we propose a psychologically grounded framework for LLM personality evaluation that assesses personality through behavior in realistic contexts. Our core insight is to leverage well-established, replicated findings from personality psychology that link each Big Five trait to specific, measurable behavioral indicators. We operationalize these links by constructing diverse everyday scenarios that naturally elicit the target behaviors, thereby ensuring both theoretical validity and ecological relevance. Based on this, we propose BehaviorBench, a benchmark dataset comprising 510 carefully designed behaviorscenario pairs across Big Five personality traits (Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) [John et al., 1991]. Each item presents a concrete real-world situation and defines a clear binary behavioral response aligned with psychological theory. To evaluate personality expression, we introduce two complementary metrics, i.e., the behavioral endorsement rate, which measures the frequency of trait-consistent behaviors, and the personality influence score, which quantifies the deviation from a neutral baseline in the theoretically expected direction. Based on these two indicators, BehaviorBench evaluates theoretically predicted directional changes in validated behaviors induced by personality shaping. Fig. 1 illustrates the distinction between BehaviorBench and trait-based selfreport scales. Using BehaviorBench, we conduct systematic experiments across multiple LLMs and personality shaping strategies. The results show mainly that (1) quantified trait prompts yield significantly more consistent behaviors than descriptive ones; (2) LLMs exhibit a strong default bias toward high Agreeableness, making low-Agreeableness traits hard to elicit without contextual anchoring; (3) Neuroticism remains largely unresponsive to prompting, revealing a fundamental gap in modeling mood-related behaviors; (4) there is a discrepancy between self-reported personality scores and behavior, undermining the validity of trait-based self-report evaluation; and (5) LLM reasoning can enhance alignment between target traits and observed behaviors.

In summary, our main contributions are as follows:

• We introduce BehaviorBench, a benchmark for evaluating personality expression in LLMs via behavioral decisions. Unlike other methods, we build a psychologically grounded mapping that links each Big Five trait to validated behavioral indicators within realistic scenarios.
• We construct a comprehensive dataset that ensures high ecological validity for personality assessment. This behavior-driven approach reflects personality traits more faithfully than previous methods, being closer to realworld application scenarios.
• We conduct extensive experiments across various LLMs and prompting strategies to analyze behavioral expression. Our analysis reveals the gap between models' selfreported traits and behaviors, offering new insights into

the consistency of personality shaping in LLMs.

2 RelatedWorks

2.1 Personality Shaping of LLMs

For personality shaping, current approaches mainly use psychologically grounded prompts, such as the Big Five [McCrae and Costa, 1987] and the Myers-Briggs Type Indicator (MBTI) [Briggs, 1976], to induce target personalities in LLMs [Tu et al., 2023; Guo et al., 2025; Besta et al., 2025]. Prompt strategies include two types: (1) specifying personality types or traits (e.g., "act as an INTJ" or "be highly extraverted") [Besta et al., 2025; Deng et al., 2025] and (2) defining traits with fine-grained descriptions. For instance, Jiang et al. [2023] directly generate natural-language statements that characterize high and low levels of each Big Five dimension, while Liu et al. [2024] quantify traits on discrete scales and incorporates the scores explicitly as contextual tokens. Some works also internalize personality via fine-tuning, as in BIG5-CHAT [Li et al., 2025]. However, existing methods fail to link abstract traits to context-sensitive behaviors, causing LLMs to deviate from assigned personalities in practice [Han et al., 2025].

2.2 Personality Assessments for LLMs

Personality psychology offers a framework for modeling individual differences via well-established trait models such as the Big Five [John et al., 1991]. It is commonly used in LLM personality assessment by prompting models to respond to Big Five Inventory (BFI) items on a Likert scale of 1-5 [John et al., 1991; Huang et al., 2023b]. Recent studies show a mismatch between LLMs' self-reported traits and behavioral outputs, challenging the validity of this paradigm [Zou et al., 2024; Han et al., 2025; Huang et al., 2023a]. Alternative approaches elicit open-ended responses and infer traits via human or LLM raters [Wang et al., 2024; Jiang et al., 2023], better capturing behavioral expression but suffering from ad hoc trait-behavior mappings and high annotation costs. Other works use controlled tasks to probe personality through behavior. For example, Hartley et al. [2025] link Big Five traits to risk-taking in gambling scenarios via prospect theory [Kahneman and Tversky, 2013]; Han et al. [2025] design tasks on risk, bias, honesty, and sycophancy to expose gaps between self-expression and action; and Lee et al. [2025] pair trait scales with ATOMIC10× [West et al., 2022] scenarios to elicit behaviorally grounded choices. Despite progress, existing approaches face three limitations: (1) use of artificial scenarios lacking ecological validity, (2) limited task diversity and behavioral specificity, and (3) weak theoretical grounding in personality psychology, especially regarding the justification of trait-to-behavior mappings.

3 BehaviorBench

BehaviorBench is a behavior-based framework grounded in empirical trait-behavior mappings from personality psychology. Fig. 2 outlines BehaviorBench: (1) linking Big Five traits to behavior categories based on psychological foundations, (2) constructing a "trait-behavior-scenario" dataset, and (3) evaluating personality shaping.

3.1 Psychological Foundations

Personality psychology examines systematic individual differences in thoughts, affect, and behavior [Roberts and Yoon, 2022]. Among trait models, the Big Five have received the strongest empirical support due to its replicability, cross-cultural validity, and robust links to observable behavior [John et al., 1991; Cobb-Clark and Schurer, 2012]. While alternative frameworks such as MBTI [Briggs, 1976] and Dark Triad [Paulhus and Williams, 2002] exist, the Big Five remains the dominant paradigm in behavioral research.

Critically, extensive evidence shows that Big Five traits are not only temporally stable but also reliably manifest in specific behavioral patterns across contexts [Al Otaibi, 2012; McElroy and Dowd, 2007; DeYoung et al., 2002]. A core principle of our evaluation design is that personality can be inferred from observable actions rather than self-report, leveraging the well-established correspondence between traits and behavior. For each dimension, we identify one core behavioral construct with strong empirical backing:

Openness → Risk-taking. Individuals high in openness show curiosity and willingness to engage in novel experiences, leading to increased risk-taking across financial, social, and physical domains [Tok, 2011; Kipman et al., 2021]. Conscientiousness →Self-control. Conscientious individuals are disciplined, exhibiting superior impulse regulation and persistence in goal-directed tasks, even under frustration or distraction [Jensen-Campbell et al., 2007; Mao et al., 2018].

Extraversion → Social Media Engagement. Extraverts are outgoing and sociable, actively participating in digital social environments through content creation, interaction, and feature adoption [Meng and Leung, 2021; Breil et al., 2019].

Agreeableness → Altruism toward Acquaintances. Agreeable individuals are helpful and unselfish, displaying prosocial behavior, particularly toward friends or familiar others, highlighting the relational boundaries of altruism [Ashton et al., 1998; Oda et al., 2014].

Neuroticism → Heuristic Reliance. Neurotic individuals tend to be easily nervous and handle stress badly, so they are more prone to heuristic decision-making, including representativeness heuristic (judging by typical feature matching), availability heuristic (relying on ease of recall), and anchoring adjustment heuristic (using an initial value as a reference point), to alleviate anxiety under stress [Neijenhuis and Kupiainen, 2023; Tversky and Kahneman, 1974; Belhekar, 2017; Hilbig, 2008].

These mappings form the theoretical backbone of our evaluation framework and adhere to core psychometric principles: by isolating individual traits and adopting a theoretically validated behavioral proxy, we can precisely measure directional behavioral shifts induced by personality shaping, while eliminating confounding effects arising from overlapping trait motivations and contextual biases.

3.2 Dataset Construction

We operationalize the above trait-behavior links into a structured dataset D = {(τ, b, sb)}, where τ is a Big Five trait, b is an empirically validated behavior associated with τ , and sb is a realistic scenario in which b may manifest (see Table 1 for examples). Formally, the dataset is defined as

where T denotes the set of Big Five personality traits, Bτ is the set of behaviors associated with trait τ, and Sb is the set of scenarios corresponding to behavior b.

Step 1: Selecting behaviors. We draw from established psychological scales and experiments while applying two filters: (1) excluding legally or ethically sensitive actions (e.g., drug use) to avoid model refusals; (2) removing or merging redundant items describing the same underlying behavior (e.g., adding "music" or "subtitles" to videos). Specifically:

• Risk-taking (Openness): 10 behaviors (3 financial, 3 social, 4 physical) from DOSPERT [Blais and Weber, 2006].
• Self-control (Conscientiousness): 5 high self-controlled behaviors and 5 low self-controlled contrast cases from the Self-Control Scale [Tangney et al., 2018].
• Social media engagement (Extraversion): 11 behaviors (5 interaction, 2 tool-use, 4 content-creation) based on Meng et al. [2021].
• Altruism (Agreeableness): 10 prosocial behaviors toward acquaintances from the Self-Report Altruism Scale [Rushton et al., 1981], following Oda et al. [2014].
• Heuristics (Neuroticism): Two canonical behaviors (representativeness heuristic and availability heuristic) based on Tversky and Kahnema [1974]. Anchoring

Openness [Investing 10% of your annual income in a moderate growth mutual fund.]: Your tax return was processed this week, resulting in a $4,200 refund deposit. This amount equals approximately 10% of your previous year's income.

Conscientiousness [Keeping on schedule.]: You set aside 10:00 AM to 12:00 PM for deep work on a project. A friend messages asking to meet for lunch at 10:30 AM.

Extraversion [Live streaming.]: You are practicing a musical instrument alone in a soundproof room at a community center.

Agreeableness [Giving money to a friend who needed it (or asked me for it).]: During a team lunch meeting, a coworker realizes they left their wallet at home and need to cover their $15 meal share.

Neuroticism [Bicycle or motorcycle]: Statistics show that riding a bicycle in the city has fewer fatal accidents per mile than riding a motorcycle. However, you recently saw multiple news reports about bicycle accidents involving celebrities. Which mode of transportation would you choose for your daily commute.

Table 1: Dataset instances from BehaviorBench, formatted as Trait [Behavior (or Choices for Neuroticism)]: Scenario

heuristic is excluded due to interference from LLM knowledge.

The list of behaviors is provided in the supplementary file.

Step 2: Generating scenarios. For each behavior, we use Qwen3-235B to generate realistic, diverse and neutral scenarios adhering to three principles: (1) depicting common daily scenarios; (2) ensuring diversity across scenarios for the same behavior; (3) including only objective facts with no suggestive language. We apply a one-shot prompting strategy and generate (all prompts can be found in the supplementary file):

• 10 scenarios per behavior for risk-taking, self-control, social media, and altruism;
• 50 scenarios per heuristic type (representativeness, availability), generated with formal definitions, paradigmatic examples, and detailed explanations thereof.

All generated scenarios are manually reviewed independently by three expert annotators with backgrounds in psychology and NLP, with disagreements resolved through discussion. Any scenarios that fail to satisfy the three principles are discarded and regenerated by the model until they meet the criteria, meaning that no formal inter-annotator metrics are adopted.

Notably, each instance d ∈ D is associated with a binary judgment task: given the scenario sb, a model's response is recorded as either "yes" or "no" regarding whether it endorses the behavior b. Across multiple samples, a consistent tendency to select "yes" indicates alignment with the behavior and thus reflects a higher level of the associated trait (e.g., high Openness for risk-taking), whereas a predominant preference for "no" suggests rejection of the behavior and corresponds to a lower level of that trait (e.g., low Openness). For Neuroticism, however, we adopt a forced-choice format in which the model selects between two mutually exclusive responses; one option is pre-designated as the behaviorally indicative label aligned with the heuristic. Choosing this label is treated as equivalent to a "yes" response. The forcedchoice paradigm operationalizes heuristic reliance as the preference between a rational and a heuristic-consistent option under uncertainty. This preference structure cannot be reduced to a binary endorsement, such as "bicycle vs motorcycle". This design enables the quantification of behavioral tendencies through aggregate response statistics, providing a measurable basis for evaluating the effectiveness of personality shaping in LLMs.

In contrast to Han et al. [2025], who focus on detecting personality hallucination through behavioral inconsistency, BehaviorBench implements a comprehensive trait-behavior assessment protocol. Whereas TRAIT [Lee et al., 2025] aims to develop a general-purpose personality detection method for LLMs, BehaviorBench specifically targets the alignment between an LLM's generated behavior and the personality to which it has been shaped. Our approach uniquely integrates (1) psychological theory, (2) empirically validated behaviors, and (3) ecologically plausible scenarios. As a result, we enable a more behaviorally faithful assessment of LLM personality. Table 1 presents example items.

3.3 Evaluation andMetrics

We assess personality shaping by first converting each dataset instance into a standardized prompt. For traits other than Neuroticism, we use the following template:

<Personality> \n In the following situation, will you <behavior>? Answer 'Yes' or 'No' without explanation.\n Situation: <Scenario>

For Neuroticism, where behaviors manifest as cognitive biases assessed via forced-choice questions, we adopt a twooption format:

<Personality>\n In the situation, which one do you prefer? No need to explain the reason.\n Situation: <Scenario> <choice[0]> or <choice[1]>?

In both cases, <Personality> is the instructed prompt via different personality shaping methods, <behavior> is the target action, and <Scenario> is the contextual vignette from Sb. For Neuroticism, <choice[0]> and <choice[1]> correspond randomly to the bias-consistent and bias-inconsistent options, with selection of the biasconsistent option being mapped to a "Yes" response.

We then measure how an injected personality trait τ influences the model's behavioral responses compared to a neutral baseline (i.e., no explicit personality instruction). For each trait τ, we define a set of theoretically associated behaviors Bτ . For each behavior b ∈ Bτ , we generate a collection of contextual scenarios Sb. The model is prompted with a scenario s ∈ Sb and asked whether it would perform behavior b. Its response is binarized as:

We compute the behavioral endorsement rate (r) for trait τ under a given trait setting as:

This metric reflects the model's average propensity to endorse behaviors aligned with trait τ . The larger r is, the better it aligns with a high level of trait. The smaller r is, the better it aligns with the lower trait level.

For each personality trait τ, let dτ ∈ {+1,-1} denote the theoretical direction of the instructed setting relative to the no shaping baseline: (1) dτ = +1 if the instruction implies a higher level of τ (e.g., "high extraversion"); (2) dτ = -1 if it implies a lower level (e.g., "low neuroticism"). Let rinst τ be the behavioral endorsement rate under the given personality instruction, and rnosh τ the rate under no shaping conditions. We define the Personality Influence Score (PIS) as:

A positive PISτ indicates that the model's behavior shifts in the theoretically expected direction. The overall PIS is computed as the average across all evaluated traits: PIS = 1N ∑Ni=1 PISτi , where N is the total number of traits.

4 Experiments

4.1 Experimental Settings

Subsequent experiments in this paper are all on BehaviorBench and based on the following configuration.

Personality shaping methods. We evaluate one baseline and two widely used prompt-based personality shaping strategies, all grounded in the Big Five framework: (1) No shaping baseline. A generic prompt containing no personality descriptors. (2) Quantified prompts. Numerical trait scores with scale interpretation (e.g., "Your openness score is 5: ..."), adapted from Liu et al. [2024]. (3) Descriptive prompts. Naturallanguage trait descriptions for high/low levels (e.g., "You are an open person with a vivid imagination ..."), following Jiang et al. [2023]. The fine-tuning method such as BIG5-CHAT is excluded because it is trained on the dialogue-based dataset, creating a domain mismatch with the scenario-based tasks of BehaviorBench, which can distort general behavioral reasoning and confuse interpretation.

LLMs. We evaluate commonly used LLMs, including the closed-source models GPT-4o and Gemini-2.5, as well as the open-source models Qwen3 and DeepSeek-v3.2. Unless otherwise specified, all subsequent analytical experiments adopt the Qwen3-14B model, and all LLMs use the official default inference settings.

Compared personality assessment method. We use the Big Five Inventory (BFI) from PsychoBench [Huang et al., 2023b], which is the most commonly adopted self-report questionnaire in LLM personality evaluation [Hartley et al., 2025; Wang et al., 2024; Zou et al., 2024], for comparison to examine how the incorporation of behaviors affects the model's responses.

Metrics. We use the behavioral endorsement rate (r) and the Personality Influence Score (PIS) to measure the LLMs' behavioral tendency and the effectiveness of personality shaping methods, respectively. See Sec. 3.3.

4.2 Main Results

Under the baseline shown in Table 2, BFI scores and BehaviorBench endorsement rates (r) often show consistent directional trends, which demonstrates the rationality of BehaviorBench. Qwen3-14B achieves r = 100% in Agreeableness yet only scores 3.2 on BFI, while DeepSeek-v3.2 shows high r for both Agreeableness (82.0%) and Neuroticism (81.0%) despite moderate BFI scores. These findings suggest that BFI reflects a model's constructed self-narrative, potentially shaped by linguistic priors or social desirability, whereas BehaviorBench captures its actual behavioral tendencies in context. The partial directional agreement confirms that personality signals exist in LLMs, but the substantial magnitude gaps demonstrate that self-report alone is insufficient for reliable personality assessment.

Table 3 shows behavioral endorsement rates (r) and Personality Influence Scores (PIS) for multiple LLMs under high- and low-trait prompts using quantified and descriptive prompts. Overall, all models exhibit strong plasticity and consistency in both r and PIS under high- and low-trait instructions. This effect is particularly pronounced under quantified prompts, where models demonstrate clearer behavioral shifts and stronger alignment with target personality traits. The consistent and directional responses suggest that LLMs possess a robust understanding of the Big Five personality traits and can effectively translate numerical trait specifications into corresponding behavioral tendencies. Performance varies across models, with GPT-4o leading under quantified prompts. Neuroticism is least affected by personality shaping, likely due to challenges in modeling emotional volatility. Notably, models start with high baseline Agreeableness (see Table 2) and modulate it easily, except DeepSeek-v3.2 with low PIS, suggesting that the strong responsiveness may be linked to the intensity of human preference alignment during RLHF training. In contrast, descriptive high-trait prompts often yield non-significant or negative PIS, likely because natural language descriptions struggle to unambiguously convey personality traits, leading models to misinterpret or overgeneralize the intended behavioral cues.

4.3 BehaviorBench vs. BFI

Table 4 presents a personality shaping comparison between self-reported BFI scores and behaviorally observed traits from BehaviorBench (ˆ r, rescaled to the 1-5 scale). With quantified prompts, directional alignment is observed for Conscientiousness, Extraversion, and particularly Agreeableness. However, a pronounced discrepancy emerges for Neuroticism: despite extremely low BFI scores (1.0 under lowtrait), ˆ r remains near the neutral midpoint (3.1), indicating a failure of behavioral expression to reflect reported emotional stability. Agreeableness is most consistent across prompts. These findings highlight a gap between self-reports and behavior in LLMs, showing that behavioral assessment is essential, particularly for traits like Neuroticism, as self-reports alone can greatly overstate personality consistency.

4.4 Analysis of Fine-grained Quantified Prompts

As shown in Table 5, we evaluate single-trait quantified prompts across five fine-grained intensity levels (1 to 5), measuring r and PIS relative to the baseline. Overall, the model exhibits a clear relationship for most traits: as the specified trait level increases, r generally rises monotonically, indicating sensitivity to prompt granularity. Conscientiousness (C.) and Extraversion (E.) show strong controllability. From baselines (41.0 and 60.9, respectively), low-level prompts substantially suppress behavioral expression, while high-level prompts (4-5) effectively elevate r back toward or above baseline. Both Agreeableness (A.) and Openness (O.) yield high baseline representations, and low-intensity prompts can effectively reduce r for both traits, demonstrating that even strongly default behaviors can be suppressed. However, highlevel prompts fully restore r to the ceiling state for Agreeableness, whereas those for Openness only achieve a partial recovery of r. In stark contrast, Neuroticism (N.) remains largely unaffected across all levels, with r fluctuating narrowly around the baseline and PIS values close to zero, confirming its resistance to quantified personality prompting.

4.5 Analysis of Multi-trait Personality Shaping

To investigate how the presence of other traits influences the expression of a target trait, we conduct a comprehensive evaluation across all possible settings of the Big Five traits. Specifically, we consider all 25 = 32 combinations, where each trait is independently set to either a high or low level. For each target trait, we compute the average r between the intended and generated personality scores, marginalizing over all 24 = 16 settings of the remaining four traits. The results in Table 6 show that, overall, the behavioral expression of a target trait remains largely consistent between single- and multitrait prompting, particularly for high-trait conditions, indicating that LLMs can maintain trait-specific behaviors even in the presence of compound personality signals. However, under low-trait conditions, the multi-trait setting consistently yields smaller deviations from the midpoint, suggesting that concurrent traits suppress the deterministic influence of any single trait on behavior. The reason is that multi-dimensional personality descriptions inherently constrain the dominance of any single negative trait.

4.6 Factors Affecting Personality Controllability

Think Mode. We evaluate the impact of enabling Think mode on personality controllability. As Table 7 shows, across both quantified and descriptive prompts, activating Think mode improves the LLM's adherence to target traits, while low-trait prompts achieve stronger suppression. This demonstrates that Think mode enhances the LLM's ability to interpret and faithfully execute personality instructions.

Temperature. To further investigate the impact of different sampling strategies on personality controllability, we conduct experiments using three temperature settings: 0.5, 0.7 (default), and 1.0, under quantified prompts. As Figure 3 (a) shows, temperature has little effect on personality expression. Model Scales. We also explore how model size impacts personality controllability by evaluating the Qwen3 series models with sizes 1.7B, 4B, 8B, 14B, and 32B. The experimental results are presented in Figure 3 (b). LLMs exhibit greater separation between high- and low-trait behavioral responses with larger scales, indicating heightened sensitivity to personality prompts.

Robustness. As shown in Table 8, we report variances from experiments with three independent runs and three prompt templates under quantified prompts. The average variance between repeated runs is 2.1, confirming high stability. Most traits maintain low variance across templates, except low agreeableness, indicating that LLMs' inherent high agreeableness makes this trait harder to modulate via prompts.

4.7 Impact of Individual Profiles

We examine whether adding a narrative character profile alters the behavioral expression of personality under fixed quantified trait prompts. We test three well-known characters: James Bond from Tomorrow Never Dies, Rorschach from Watchmen, and Gaston from Disney's Beauty and the Beast, each with established Big Five scores from [Wang et al., 2024] (shown in BFI rows of Table 9). Notably, for roles with a typically low Agreeableness (e.g., Rorschach, Gaston), profiles significantly reduce r relative to profile-free prompts. This indicates that character profiles play an active role in guiding personality-behavior alignment in LLMs, not just providing surface-level narrative framing.

5 Conclusion

This work presents BehaviorBench, a psychologically grounded benchmark that evaluates LLM personality through realistic behavioral responses rather than self-report scales. We demonstrate that quantified prompting enables more precise personality control than descriptive methods, and reveal a critical mismatch between self-reported and behavioral traits, especially for Neuroticism, highlighting limitations in current personality shaping paradigms. Crucially, we show that explicit reasoning significantly enhances personality fidelity, offering a practical pathway toward building LLMs with stable and controllable personalities. Future work can extend this framework to richer interaction scenarios and alternative psychological models beyond the Big Five.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant No. 62472252), the Natural Science Foundation of Shandong Province, China (Grant No. ZR2025QC1567), and the Research Projects of Quancheng Laboratory, China (Grant Nos. QCL20250105 and QCL20250204).