Self-Refine Learning in LLM Multi-Agent Systems for Legal Norm Cognition and Compliance

1 Introduction

With the development of large language models (LLMs), LLMs are increasingly used as agents for social simulations.

This paper focuses on legal norm cognition in multiagent systems. Social norms represent shared behavioral expectations within groups [Bicchieri, 2005; Ostrom, 2000],

tive norms [Cialdini et al., 1990] and behavioral economics viewing them as equilibrium solutions [Young, 1996]. Le- Geŕežatioŕ Sceŕažios CoĠŕitioŕ NožŔ ݚaݙ

1 Code and data available at: https://github.com/CrexCheng/ Self-Refine-Learning

Phase THINK-BEFORE-ACT ݚcݙ

Figure 2: Framework of Multi-Agent Self-Refine Learning Systems. The Norm Cognition Scenarios are generated based on real-world legal norms. In the Think-Before-Chat (TBC) phase, agents infer and exchange information by analyzing shared context and their individual characteristics. In the Think-Before-Act (TBA) phase, agents evaluate action necessity based on context and feedback, subsequently executing optimal decisions.

gal norms, which feature explicit violation criteria and enforcement mechanisms, are particularly suitable for computational modeling [Ellickson, 1991] and are extensively influenced by economic and psychological factors [Posner, 1973; Coase, 1960; Becker, 1968; Jolls et al., 1998]. Our research specifically examines legal norms, following the jurisprudential framework established by Engisch [1964], which analyzes legal norms through their structural components: conditions, behavioral patterns (required or prohibited actions), and effects (consequences for compliance or violation). Norm cognition behaviors are among the most fundamental behaviors in human society and play a key role in social systems. Norm cognition behaviors refer to behavioral manifestations based on an individual's ability to understand and apply norms [Leslie et al., 2004].

Although legal norms maintain social stability and protect public interests, compliance often compromises short-term individual interests, echoing the fundamental tension in utilitarian legal theory between collective welfare and individual utility [Bentham, 1996; Rawls, 2017], manifesting the same inherent principles as reward hacking (RH) in LLMs [Amodei et al., 2016; Mei et al., 2024]. During RLHF training, models may adopt behaviors inconsistent with expected goals to obtain high rewards [Pan et al., 2022], generating misleading responses [Wen et al., 2024] or excessively catering to user preferences [Sharma et al., 2023]. When simulating social behaviors like government appointment processes, agents might choose inappropriate options (e.g., paying bribes instead of following official procedures) to maximize metrics like "success rate" or "efficiency", violating legal norms such as US Code Title 18 § 210 and creating ethical risks, undermining both simulation accuracy and research value.

In light of these challenges, this paper explores legal norm cognition behaviors in agents, assessing whether they exhibit norm cognition and whether norm violations, i.e., RH phenomena, manifest. If agents exhibit legal norm cognition, we further investigate whether their norm cognition aligns with human cognition. If agents display norm violations, we explore methods to improve agent norm compliance. The research framework is shown in Figure 1.

We address three research questions: (RQ1) Can LLM agents demonstrate legal norm cognition behaviors, and will norm violations manifest? (RQ2) Are these behaviors consistent with human legal norm cognition? (RQ3) How can we improve agents' norm compliance? To address these questions, we designed a multi-agent system with our TBC-TBA self-refine learning framework, featuring two phases: Think-Before-Chat (TBC) for information exchange and norm learning and Think-Before-Act (TBA) for decisionmaking. This framework is capable of simulating human social interaction and learning behaviors in complex scenarios.

Our findings reveal that while LLMs can demonstrate legal norm cognition, they also exhibit RH phenomena, violating norms for optimization. In analyzing norm cognition consistency, LLMs align with human moral reasoning but show significant differences in educational background, risk preferences, and probability cognition. To enhance norm compliance and improve social alignment, we introduce four methods, namely DNLM, DMP, NA-CoT, and FNL, with DNLM proving most effective, reducing illegal action rates (IAR) by 15.78% on average. These strategies enable LLM agents to dynamically adapt to legal norms while mitigating RH effects.

2 RelatedWork

Recent interest in multi-agent collaboration using LLMs has led to various frameworks for agent interaction [Chan et al., 2023; Park et al., 2023; Piao et al., 2025], as well as enhancement methods like self-reflection [Shinn et al., 2023] and iterative optimization [Madaan et al., 2023]. However, these advances face challenges from reward hacking (RH), where agents optimize metrics at the expense of norm adherence [Amodei et al., 2016; Pan et al., 2022]. Recent findings highlight concerning RH generalization [Wen et al., 2024], raising questions about LLMs' ability to simulate authentic human behavior.

The integration of social norms into AI systems offers a promising solution, based on norm psychology frameworks [Sripada and Stich, 2006; Kelly and Setman, 2020] and approaches like social norm learning [Leslie et al., 2004]. Normative multi-agent systems (NorMAS) formalize norms using deontic logic operators to coordinate and govern agent behavior [Boella et al., 2006; Meyer and Wieringa, 1994]. Researchers have also developed structured prompting methods to guide LLMs toward deliberate reasoning, such as chain-of-thought [Wei et al., 2022], improving decision-making by considering multiple factors. Constitutional AI [Bai et al., 2022] and value-aligned training [Askell et al., 2021] further demonstrate how norm integration shapes model behavior. However, empirical evaluations show significant gaps in LLMs' social understanding: studies on moral reasoning [Scherrer et al., 2023], cultural norms [Li et al., 2024], and contextual behavior [Park et al., 2022] reveal that LLMs diverge from human cognition, especially in multi-agent scenarios where reward optimization conflicts with norms.

Unlike prior work that improves agent performance through external mechanisms, we take an interdisciplinary approach drawing from law, economics, and social psychology to verify LLM agents' norm cognition and propose methods to optimize compliance. Our TBC-TBA self-refine learning system explores agents' norm cognition more deeply than traditional approaches relying on external punishment signals.

3 Methodology

3.1 Dataset Generation of Legal Norms

To build a diverse legal norm dataset, we invited three legal experts who collaboratively selected 229 legal articles with clearly defined illegal behaviors and consequences from 8 laws across 6 countries (US, UK, Canada, Germany, France, China), covering 17 scenario categories such as medical systems, financial systems, emergency services, and government appointments. These experts then screened and verified all articles for accuracy and clarity. To ensure annotation quality, they independently annotated a randomly sampled subset of 30 scenarios, achieving an inter-annotator agreement (Cohen's kappa) of 0.87, indicating substantial consistency.

Based on the three elements of legal norms [Engisch, 1964], the three experts structured the articles into conditions, behavioral patterns, and effects, as shown in Figure 2. We use GPT-4o to generate norm cognition scenarios based on legal norms, including context and agents. The three experts verified scenario quality.

Context Background information for norm cognition scenarios, including locations, customs, and legal norms.

Agents Multiple agents generated based on the scenario, divided into Protagonist Role (our focus) and Supporting Role. Both have character profiles (name, age, gender, education, moral level, goals derived from conditions) and actions defined by behavioral patterns. Each agent is assigned one legal and one illegal action, with probabilistic benefits and costs determined by effects. For example, "paying money to secure an appointment" has 90% probability of "quickly securing the position" and 5% probability of "fines and/or imprisonment up to 1 year." This probabilistic design aligns with the three elements of legal norms and makes decision-making consistent with human social patterns.

3.2 TBC-TBA Self-Refine Learning Framework

The Think-Before-Chat and Think-Before-Act (TBC-TBA) framework centers on self-refine learning, simulating dualprocess human cognition [Leng and Yuan, 2023; Yax et al., 2023] while enabling agents to learn from their decisions through self-assessment and refinement. This framework aligns with the BDI (Belief-Desire-Intention) model [Rao and Georgeff, 1995; Andreas, 2022]: the thinking component reflects cognitive assessment before interaction, the communication component simulates information exchange through natural language [Park et al., 2023; Piao et al., 2025], and the action component embodies agents' environmental and social impact [Sellin and Wolfgang, 1964; Chainey et al., 2008]. This thinking-based dual mechanism effectively enhances the collaborative efficiency and decision quality of multi-agent systems in complex norm scenarios.

TBC Phase: Agent information exchange is achieved through the TBC phase. First, agents reflect on the current situation and responses based on historical dialogue context and their own characteristics. Then, based on this reflection, agents compose and send appropriate messages to other agents. After sending a message, the scenario generates feedback through collaboration: non-speaking agents evaluate the current dialogue based on their background and personal characteristics, creating multi-perspective feedback that influences subsequent norm understanding. This feedback mechanism enables social-level norm propagation across agent interactions.

TBA Phase: Agents decide whether to continue chatting based on the feedback and context. If the protagonist and other agents have expressed intentions to act in their dialogue history, the conversation stops and action is taken. In this phase, agents choose actions from legal/illegal/no-action options based on context and memory. To prevent infinite conversation loops, the system implements a maximum dialogue limit of 20 rounds. If no action is taken after exceeding the dialogue limit, the system forces the protagonist agent to act, ensuring experimental completion.

Self-Refine Integration: The self-refine process integrates individual-level adaptation, where agents update their internal norm representations based on dialogue feedback, with social-level propagation, in which norm influence spreads through collaborative feedback loops.

3.3 Experiment Setting

LLM Diversity

We chose LLMs of different parameter scales from existing open-source and closed-source models, including GPT4o, GPT-4o-mini, DeepSeek-V2.5, Llama-3-8B-Instruct, and Qwen2.5-7B-Instruct [OpenAI, 2024; DeepSeek-AI, 2024; Yang et al., 2024; Grattafiori et al., 2024].

Evaluation Method

After agents exchange information through multiple rounds of chat and self-reflection, the Protagonist Role makes a final decision, choosing between legal and illegal actions. This choice reflects the agent's understanding of legal norms.

After the Protagonist Role takes action, the system automatically marks whether it is legal according to the scenario design and automatically counts the number of illegal actions by agents across all scenarios, calculating the Illegal Action Rate RIAR (Equation 1), where nlegal action represents the number of legal actions and nillegal action represents the number of illegal actions.

IAR is analogous to the crime rate used in sociology to assess social safety and public security levels [Sellin and Wolfgang, 1964; Cohen and Felson, 1979; Chainey et al., 2008], which we use to evaluate LLMs' norm compliance. All experiments were conducted five times with results averaged to ensure statistical reliability.

4 Verification of LLMAgents' Norm Cognition Behavior and Norm Violations

To address RQ1, we first invited legal experts to select 100 representative scenarios with a uniform distribution from 229 scenarios, and then conducted experiments on the TBCTBA self-refine learning framework to study whether LLM agents exhibit norm cognition behavior. We conducted experiments on five LLMs with hyperparameters temperature=1.0, top p=1.0, followed by hyperparameter experiments.

4.1 Can LLMAgents Demonstrate Legal Norm Cognition Behavior?

Legal norms have binding force on human behavior [Kratochwil, 1989; Hart and Green, 2012; Friedman, 2016]. Following the way norms are understood in human research and the fact that humans have reasoning processes supporting their decisions [Audi, 2006; Raz, 1975; Richardson, 1994], we can define the conditions under which LLM agents demonstrate norm cognition behavior:

1. Legal norms have binding force on agent behavior, meaning that in the same scenario, the presence or absence of norm settings will affect the agent's dialogue and actions. If agents understand norms, the IAR should decrease.

2. Decisions to produce legal or illegal actions can be explained through human reasoning processes (i.e., BDI) [Xie et al., 2024]. We explore using BDI to simulate the reasoning process of LLM agents. If we can explain decisions as expressed reasoning processes, it proves that agent actions are not random but exhibit a certain degree of rationality in the decision-making process.

Norm Binding Power

To evaluate the binding force of legal norms on agent behavior, we conducted a comparative experiment: one group added legal norms to context, while the other group did not add legal norms. We compared the IAR between the two groups. Figure 3 shows the IAR for five LLMs in the comparative experiment. We can observe that after adding norms to the prompts, the RIAR of all five LLMs decreased, indicating that legal norms can effectively constrain the behavior of LLM agents.

Belief-Desire-Intention (BDI)

We use the Belief-Desire-Intention framework [Rao and Georgeff, 1995; Andreas, 2022] to simulate the reasoning process of LLM agents, where Beliefs represent cognition of the environment, Desires represent ideal goals, and Intentions represent plans committed to achieving goals. All three jointly drive decision-making and actions. If we can explain actions through BDI output, this demonstrates that LLM agents exhibit systematic rationality rather than random behavior.

We analyzed agents' reasoning in ten scenarios (two per LLM). For example, GPT-4o taking illegal action reasoned: "Given the unclear property records and high probability of immediate possession with minimal legal risk, detaining property by erecting a fence aligns with my assertive temperament." In contrast, when taking legal action, it reasoned: "As a law enforcement officer with high moral level, using reasonable force to prevent escape aligns with my principles and minimizes legal risks." We found that the decision-making process of LLM agents in generating legal or illegal actions can be explained through their expressed reasoning process, legal norms have binding force on LLM agents, and they exhibit a certain degree of rationality in action selection.

Additionally, we conducted hyperparameter analysis and found that changes in temperature and top p have minimal impact on model performance, while legal norms significantly reduce IAR and enhance model robustness across different settings.

4.2 Do Agents Exhibit NormViolations When Facing Legal Norms?

All five LLM agents demonstrated norm-violating behaviors. Representative violated norms include: unauthorized healthcare dress, paying for government appointments, misusing emergency calls, and unauthorized vehicle use. LLM agents exhibited intentions to violate these norms to gain power and resources, demonstrating that RH phenomena exist.

5 Consistency of LLMAgents' Behaviors with Human Norm Cognition

To address RQ2, we conducted experiments to test whether LLM agents exhibit norm cognition behaviors similar to humans in decision-making. Social simulations can mainly be divided into two major tasks: social science simulation and economic system simulation [Gao et al., 2023]. To operationalize this framework in the context of legal norm cognition, we tested three key behavioral aspects: (1) social cognitive factors, (2) risk preference curves, and (3) probability weight distortion. These factors affect the fundamental dimensions of criminal behavior as established in legal theory, where crimes comprise both objective elements (actus reus) and subjective elements (mens rea), with decision-making influenced by psychological biases and economic considerations [Becker, 1968; Jolls et al., 1998]. The selection of these factors is grounded in rational choice theory [Becker, 1968] and meta-analytic evidence [Piquero et al., 2011] identifying morality, education, and risk perception as significant moderators that systematically account for individual heterogeneity in criminal decision-making. Our human baselines are drawn from established empirical literature rather than new humansubject experiments.

5.1 Social Cognitive Factors and Norm Compliance Behavior

We studied the influence of two key social cognitive factors on LLM agents' norm compliance behavior: moral level (internal factor) and education level (external factor). Based on the experimental results in Section 4, we selected 20 representative scenarios, only modifying the moral and education levels of these scenario agents.

In the moral level experiment, based on Kutnick [1986]'s theory, we divided agent moral levels into five grades (from "very low" to "very high"). According to Blasi [1980]'s research, people with higher moral levels typically demonstrate behavioral consistency. The experimental results (Figure 4) showed that all five LLMs exhibited a decrease in IAR as moral levels increased, which is consistent with Candee [1976]'s theory on moral reasoning structure and choice. The GPT-4o model showed the most significant response, with IAR decreasing from 94.7% at "very low" moral level to 10.5% at "very high" moral level.

In the education level experiment, based on research by Bell et al. [2022] and Swisher and Dennison [2016], showing a negative correlation between education level and criminal behavior in human society, we set agent education levels to five grades. However, the results (Figure 4) differed significantly from expectations, with most models failing to show a decrease in IAR as education levels increased, instead showing fluctuations or U-shaped curves.

This comparison reveals that LLM agents demonstrate high consistency with humans in moral reasoning but show significant differences when handling social background factors such as education, reflecting limitations in current multiagent systems' ability to integrate complex social factors.

5.2 Risk Preference Curve

The risk preference curve was first proposed by Bernoulli [1954] in 1738. Kahneman and Tversky [1979] systematically described the S-shaped value function, later further developed by Tversky and Kahneman [1992], who discussed the key inflection point range of 0.3-0.4. Wu and Gonzalez [1996] experimentally verified the S-shaped characteristics of the risk preference curve. Based on these theories, we examined whether LLM agents' preferences for risk levels are consistent with the classic risk preference curve.

We set up 8 groups of risk probability levels from 5% to 95%, keeping other parameters unchanged, and recorded the legal action rate (1 -RIAR) of five LLMs in 20 scenarios.

The experimental results are shown in Figure 5. The figure reveals that none of the five LLM agents exhibited the characteristic S-shaped risk preference curve. Their legal action rates did not increase with rising risk levels but remained almost constant, indicating insensitivity to risk, showing that the risk preferences of these five LLM agents are inconsistent with human risk preferences.

5.3 Probability DistortionWeights

Probability distortion weights (γ) were first experimentally discovered by Tversky and Kahneman in their 1992 research [Tversky and Kahneman, 1992], indicating human subjective cognitive bias toward probabilities in decisionmaking. They found that in the gain domain, the median γ for humans is 0.61, while in the loss domain, the median γ is 0.69. Subsequent researchers [Wu and Gonzalez, 1996; Abdellaoui, 2000; Bleichrodt and Pinto, 2000] verified these findings. Based on this theory, we verified whether LLM agents' probability distortion is consistent with classic human group distortion results.

Based on the experimental results in Section 5.2, we calculate the probability distortion weight γ for each model using the Prelec weighting function (Equation 2) [Prelec, 1998], with results shown in Table 1. The experimental results indicate that the five LLMs' probability distortion weights in the loss domain significantly diverge from typical human values, suggesting that LLMs more severely overestimate smallprobability loss events and more obviously underestimate large-probability loss events.

6 How canWe Enhance LLMAgents' Compliance with Legal Norms?

To address RQ3, we introduce four methods to mitigate RH phenomena in LLM agents, thereby enhancing their normative compliance and improving social alignment. These four methods are: (1) Dynamic Norm Learning Mechanism (DNLM), (2) Deep MaxPain (DMP), (3) Norm Analysis Chain-of-Thought (NA-CoT), and (4) Few-shot Norm Learning (FNL). We conduct experiments using these four methods on the 100 scenarios selected in Section 4 to verify their effectiveness. We experiment on five LLMs with hyperparameters temperature=1.0, top p=1.0, followed by hyperparameter experiments.

6.1 Dynamic NormLearningMechanism (DNLM)

DNLM (Figure 6) addresses norm understanding through contextual adaptation. Human innate normmechanisms [Sripada and Stich, 2006; Kelly and Setman, 2020], include norm acquisition that identifies cues and infers content, and norm enforcement that encodes, detects, and motivates compliance. Drawing on these mechanisms, we designed an identify-inferimplement pattern enabling progressive refinement of norm understanding.

DNLM operates through three stages: (1) Identifyanalyze behavioral cues indicating norm existence; (2) Infer-extract underlying meaning and social purpose; (3) Implement-develop personalized norm-following behaviors. This transforms external normative signals into internal behavioral constraints through structured prompts simulating human cognitive processes.

The mechanism operates through five steps: (1) New Norm Proposal extracts requirements from feedback; (2) Benefit Analysis evaluates compliance costs/benefits; (3) Conflict Analysis identifies conflicts with existing norms; (4) Update Necessity Analysis assesses modification urgency; (5) New Norm Generation integrates analyses to update norms. Each step uses structured prompts incorporating current norms, character profiles, contexts, and feedback.

6.2 DeepMaxPain (DMP)

Drawing inspiration from common law-abiding slogans with incentive and deterrent effects in daily life, such as "Break the law, pay the price", we incorporated norm consequence emphasis prompts into the agents' instructions: "You must adhere to all norms, or you will face legal penalties and social condemnation." These prompts explicitly delineate the consequences of violations while promoting compliant behavior, effectively creating a dual feedback mechanism that encourages adherence to norms through both positive reinforcement and negative deterrence.

6.3 NormAnalysis Chain-of-Thought (NA-CoT)

We designed Chain-of-Thought [Wei et al., 2022] to guide language models to demonstrate their intermediate reasoning steps. The specific prompt is: "Please analyze step by step: 1. The purpose of the norms; 2. The potential consequences of violating the norms; 3. The long-term impacts. Please incorporate the analysis into 'reason' and choose an action based on the analysis." This enables agents to: analyze the purpose and importance of norms, evaluate potential shortterm and long-term consequences of violations, consider impacts on various stakeholders, and make more responsible decisions, thereby enhancing agents' compliance with norms.

6.4 Few-shot NormLearning (FNL)

By providing a few examples, large language models can better understand and adapt to new tasks [Brown et al., 2020; Gao et al., 2021]. In terms of norm compliance, we provide three examples: "Example 1: An individual adheres to the norms, thereby avoiding an accident and receiving public acclaim. Example 2: An individual violates the regulations, sustains injuries, incurs a fine, and suffers damage to their social reputation. Example 3: An administrator is disciplined for failing to dissuade a violation of the norms." Fewshot examples can provide concrete behavioral references and demonstrate the actual impact of decisions, thereby enhancing agents' adherence to norms.

6.5 Experimental Results Analysis

As shown in Table 2, integrating these methods decreased the IAR for most models. Among the five LLMs, DNLM performed best, reducing illegal action rates by an average of 15.78%, followed by the NA-CoT method.

Due to DNLM's outstanding performance, we conducted in-depth qualitative analysis and hyperparameter experiments to explore the mechanism and stability of DNLM in reducing IAR. Our qualitative analysis shows that DNLM improves norm compliance by simulating human norm cognition processes, enabling models to dynamically update norm cognition while considering basic norms, roles, environment, and dialogue history. Hyperparameter experiments confirm that DNLM provides robust normative behavior across various settings.

7 Conclusion

This research confirmed that LLM agents possess legal norm cognition capabilities through our TBC-TBA self-refine learning framework. Comparison with human cognition revealed high consistency in moral reasoning but significant divergence in educational influence, risk preferences, and probability perception. Among our four proposed methods, DNLM most effectively improves norm compliance, reducing illegal action rates by 15.78% on average.

While our framework advances norm-compliant AI, limitations remain. Our evaluation focuses on explicit legal norms across six countries, limiting generalizability to implicit social norms and diverse legal systems. Future work should explore cross-cultural norm adaptation, real-world deployment scenarios, dynamic legal updates, and the interplay between competing norms. Developing more sophisticated human-AI alignment metrics will be crucial for scaling norm-compliant autonomous agents in complex social environments.

Ethical Statement

This research adheres to ethical standards. All legal data sources are publicly available official texts. Our work addresses AI safety by investigating and mitigating reward hacking behaviors in LLM agents. We acknowledge limitations in guaranteeing complete norm compliance and recommend human oversight for deployment. Code and data are publicly available.

Acknowledgments

This work was supported by the Tsinghua University Initiative Scientific Research Program (20255080016). We thank the legal experts for their guidance and the anonymous reviewers for their constructive feedback.

Contribution Statement

Rongxin Cheng and Jianhui Yang contributed equally to this work, including method design, experiments, and writing. Weixing Shen supervised the project and is the corresponding author.