Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing

1 Introduction

As Large Language Models increasingly serve as the cognitive core of autonomous agents [Minaee et al., 2025; Luo et al., 2025], the ability to maintain and update their parametric memory becomes paramount [Zhang et al., 2024b; Wu et al., 2025]. Unlike static models, an agent operating in a dynamic environment must continuously internalize new factual information [Hu et al., 2026]. In this context, knowledge editing serves as the critical technique for the agent's parametric memory, aiming to efficiently update a small set of factual associations encoded in LLMs while exerting minimal side effects [Wang et al., 2024b].

∗Zhengxiao Liu and Zheng Lin are corresponding authors.

Mainstream in-parameter locate-then-edit methods have demonstrated impressive efficacy in rewriting specific factual associations [Zhang et al., 2024a; Meng et al., 2022a; Meng et al., 2022b; Fang et al., 2024; Li and Chu, 2025]. However, they suffer from a severe side effect that we term Subject-Dominant Memory Interference: modifying a specific attribute of a subject inadvertently corrupts the LLM's structural knowledge of the same subject [Liu et al., 2025]. For instance, an edit targeting the fact that Lionel Messi is a citizen of Germany can lead to the severe corruption of his associated knowledge, causing the model to incorrectly assign him a hallucinatory nationality or alter his family relations (Fig. 1). Such uncontrollable over-generalization makes the edited model untrustworthy and unreliable.

While recent studies have begun to acknowledge this issue, existing solutions remain superficial. Prior attempts diagnose the issue statistically and primarily rely on onesided or heuristic adjustments without diving into the root cause, leading to limited improvement or compromised performance [Liu et al., 2025; Zhang et al., 2025]. In this work, we attribute Subject-Dominant Memory Interference to a fundamental optimization pathology within the editing process. Through gradient saliency analysis and causal tracing, we reveal that standard editing objectives drive the model towards a form of shortcut learning [Geirhos et al., 2020]: SubjectDominant Shortcut. Specifically, during the optimization of the edited weights, the model exhibits a strong bias towards overfitting the high-magnitude representation of the subject (e.g., "Messi") while neglecting the relational context (e.g., "is a citizen of"). Consequently, the optimization trajectory bypasses the internal causal mechanism that gates knowledge retrieval based on relations, establishing a direct, unconditional mapping from the subject to the new target.

To rectify this, we propose Causal Path Alignment (CPA), a principled framework designed to anchor the optimization trajectory to the model's valid causal pathways. Unlike previous heuristic patches, CPA reframes knowledge editing not merely as minimizing a loss function, but as navigating a constrained optimization landscape. Our approach operates in two stages: First, we anchor the relational context by identifying an intermediate hidden state that encodes the specific relation, independent of the target change. Second, we steer the parameter updates to strictly route through this anchored state. By forcing the information flow to traverse the correct relational feature, CPA re-engages the model's contextual gating mechanisms, ensuring that the update is specific to both the subject and the relation (Subject, Relation→Target) rather than the subject alone.

Our contributions are summarized as follows:

• We identify shortcut learning as the root cause of memory interference in knowledge editing, framing the problem as a misalignment between the optimization objective and the internal causal structure of LLMs.
• We introduce Causal Path Alignment (CPA), a twophase optimization framework that explicitly anchors the editing process to relational features to eliminate the shortcut path.
• Comprehensive experiments on diverse LLM backbones and benchmarks demonstrate that CPA achieves optimal performance in relation specificity while exhibiting minimal side-effects. Crucially, CPA acts as a modelagnostic plug-in, seamlessly enhancing the reliability of existing methods, paving the way for controllable and trustworthy knowledge editing.

2 RelatedWorks

2.1 Memory-based andMeta-learning Editing

Early approaches to knowledge editing often circumvent direct modification of the internal parameters of LLMs. One line of work adopts a memory-based strategy [Mitchell et al., 2022b; Hartvigsen et al., 2023; Wang et al., 2024a], utilizing external caches to route inputs to edited counterparts. Another line employs meta-learning networks to predict weight updates via a trained hyper-network [De Cao et al., 2021; Mitchell et al., 2022a; Li et al., 2025], enabling rapid adaptation. While effective at altering behavior, these methods function as external patches rather than updating the LLM's internal parametric worldview. This precludes deep integration and inevitably incurs overhead in inference latency and memory growth.

2.2 Locate-then-Edit Knowledge Editing

Locate-then-edit works achieve knowledge editing through knowledge neurons localization and the direct modification of located parameters. ROME [Meng et al., 2022a] pioneers this by employing Causal Mediation Analysis to identify the decisive neuron activations for processing factual associations. They reveal that the Multi-Layer Perceptron (MLP) modules at the last-subject token position (i.e., the subject aggregation site) play a pivotal role in mediating the prediction of factual prompts. To scale this capability, MEMIT [Meng et al., 2022b] distributes the update across multiple layers to enable batch editing, while subsequent works have extended this paradigm to lifelong editing scenarios [Fang et al., 2024; Cai and Cao, 2024; Li and Chu, 2025], aiming to update parameters sequentially without catastrophic forgetting.

Despite their efficacy, these subject-focused editing methods suffer from a critical Subject-Dominant Memory Interference flaw. As mentioned by recent studies [Liu et al., 2025; Zhang et al., 2025], editing the subject aggregation site often inadvertently changes every fact linked to that subject, rendering the editing process uncontrollable. Current mitigation strategies primarily rely on heuristic modifications. For instance, RETS [Liu et al., 2025] attempts to alleviate interference by relocating the edit target to the last-relation token with subject constraints, while LTI [Zhang et al., 2025] introduces a data-augmentation approach, constraining the optimization neighborhood guided with enhanced samples. However, these approaches still omit the root cause. RETS suffers from a drastic degradation in generation fluency due to excessively stringent constraints, while LTI's reliance on data augmentation obtains limited improvement. In this work, we provide an in-depth exploration of the optimization dynamics that causes subject-focused editing to fail, and propose a principled optimization trajectory alignment to address this issue from the root cause.

3 Deconstructing the Optimization Pathology: The Subject-Dominant Shortcut

To understand the root cause of the uncontrollable overgeneralization observed in locate-then-edit methods, we conduct a rigorous investigation into the optimization dynamics of ROME-like approaches. We propose that the failure is not due to the efficacy of the parameter update itself, but rather a structural pathology in how the target representation is derived.

3.1 Preliminaries: The Locate-then-Edit Objective

Knowledge editing aims to alter factual associations within an auto-regressive language model M. Given a fact triplet ⟨s, r, o⟩, where s is the subject, r is the relation, and o is the object, the goal is to update the model's prediction to a new target ⟨s, r, o∗ ⟩ while minimizing interference with unrelated knowledge ⟨s, r′ , o′ ⟩ or ⟨s′ , r, o′ ⟩.

While the second step (i.e., the weight update) is a solved constrained least-squares problem (often yielding a rank-one update ˜W l s = W l s + ∆W), our investigation identifies the first step as the source of the subject-dominant memory interference issue. The target vector v∗ s is obtained by optimizing the latent representation v to maximize the probability of the target object o∗ given the prompt. Formally, v∗ s = argminvL(v) is obtained as the minimizer of the following loss function:

Locate-then-edit methods operate under the Linear Associative Memory hypothesis [Geva et al., 2022; Geva et al., 2023]. They identify specific mid-to-early MLP layers (e.g., layer l) at the last subject token index as the decisive location for recalling subject attributes. The down-projection matrix W l s of layer l can be treated as a set of associative key-value memories that store the map between input key vectors K = [ k1 | k2 | k3 | ... ] to output value vectors V = [ v1 | v2 | v3 | ... ]. A specific fact is edited by injecting the corresponding (k∗ s , v∗ s ) pair into the associative memories. The editing process is decomposed into two distinct steps: (1) determining the optimal target vector v∗ s that encodes the new fact, and (2) updating the weight matrix W l s to map the subject's key k∗ s to this new value v∗ s .

Here, PM(o∗ | hL(v; pt)) denotes the prediction probability for o∗ when the hidden state at the edit layer is intervened with vector v, given a set of prefixed prompts pt. The KL divergence term KL(v, vold) = DKL(PM(·|v)∥PM(·|vold)) acts as a regularizer to constrain the distribution shift of v from the original output vector of W l s .

Crucially, this optimization process assumes that finding a vector v that satisfies the likelihood of o∗ is sufficient for robust knowledge editing. However, as we demonstrate in the subsequent analysis, this objective function creates a Subject-Dominant Shortcut: it encourages the vector v∗ s to over-encode the mapping to o∗ directly, effectively bypassing the relational computational path inherent in the inference mechanism of LLMs.

3.2 Gradient Analysis: The Divergence of Information Pathways

To characterize the feature prioritization inherent in the optimization of the target vector v∗ s , we analyze the gradient landscape of the loss function L(v) with respect to the model's internal representations. Since ROME introduces perturbations at a specific mid-early layer l, we track the back-propagated gradients through the hidden states of subsequent layers. Formally, we define the Gradient Saliency S for a hidden state hi,ℓ at token position i and layer ℓ > l as the L2-norm of the gradient vector:

S(hi,ℓ) = ∥∥∇hi,ℓ L(v)∥∥2 (2) We focus particularly on the last subject token (pls) and the last relation token (plr) 1 , as prior interpretability studies suggest these positions serve as aggregation hubs for subject attributes [Geva et al., 2023] and relational attributes [Liu et al., 2025], respectively.

Figure 2 visualizes the average gradient saliency over 100 factual edits on GPT2-XL 1.5B (48 layers) [Radford et al., 2019] and Qwen2.5 7B (28 layers) [Qwen et al., 2025], excluding the final output token to highlight internal dynamics. The visualization reveals a striking Dual-Peak Pattern where gradients are significantly active at two distinct loci across the layers. Specifically, we observe high gradient norms at the Subject Pathway (pls), indicating the model leverages entity representations to adjust the prediction, alongside strong gradient flows at the Relation Pathway (plr), which confirms that the pre-trained weights initially attempt to utilize the relational context to mediate the prediction of the target object. This empirical evidence suggests that the optimization landscape naturally presents two competing pathways for minimizing the loss: modifying the subject representation or leveraging the relation representation. The existence of the relation pathway in the gradient map proves that the model can theoretically attend to the relation. However, as we unveil in the subsequent section, the standard optimization objective fails to sustain this balance, causing the optimization trajectory to collapse onto a shortcut.

3.3 The Shortcut Mechanism: Causal Path Collapse

While the gradient analysis in Section 3.2 reveals that the optimization landscape initially offers two viable pathways (subject and relation), it remains unclear how the edited model actually utilizes these features for inference. To investigate the functional shift in the model's internal mechanism, we employ Causal Tracing [Meng et al., 2022a], a technique that measures the contribution of specific hidden states to the final prediction by intervening on corrupted runs. Specifically, we calculate the Indirect Effect (IE) of an MLP output mlj i (at token position i and layer lj) by restoring its clean activation into a noise-corrupted forward pass. Formally, the IE for the target object o is defined as IE(o, θ,mlj i ) = Pθ(o | p′ , mlj i ) - Pθ(o | p′ ), where p′ is the prompt with noisecorrupted embeddings.

To quantify the structural change induced by editing, we introduce the Ratio of Indirect Effect (RIE), which measures

1 The token positions are categorized into relation prefix ("rp"), first subject ("fs"), middle subject ("ms"), last subject ("ls"), first relation ("fr"), middle relation ("mr") and last relation ("lr") according to their positions in the tokenized subject or relation of a factual association.

(Liu

ncies

d, we

rs k∗

s

ector

ctors

sub-

ls on

con-

ided.

ative

-like

miza-

h the

] (4)

t (t ∈

h the

is the

strain

miza-

cy of

at the

f hid-

uring

(5)

er af-

ayers

n po-

oken

refix

ms"),

rela-

their

of a

tions rp fs ms ls fr mr lr

5

10 15 20 25

0.04

0.03

0.02

0.01

0.00

(b) GPT-J 6B

s . The x-axis

the token positions. Each heat value represents the gradient magnitude at that position. the logarithmic shift in causal reliance between the original model θo and the edited model θedited. The RIE for a specific activation is calculated as:

(a) GPT2-XL Figure 2: The average gradient saliency map of the first epoch of the optimization process for v∗ represents the layer numbers and the y-axis represents Figure 2: The average gradient saliency map during the first epoch of optimization for v∗ s . The heatmap visualizes gradient magnitudes across layers (x-axis) and token positions (y-axis). A distinct DualPeak Pattern is observed at the last subject token (ls) and the last relation token (lr). Left: GPT2-XL (1.5B). Right: Qwen2.5 (7B).

1.5B (Radford et al., 2019) with 48 layers, GPT-J 6B (Wang, 2021) with 28 layers by ROME. Here we display the results of the first epoch in Fig. 2, while the results of initial several epochs are similar, as shown in Appendix A. We omit the gradients at hLn-1 since this is the output hidden representation and it is supposed to have the most salient gradient. Fig. 2 illustrates that the gradients are most salient across layers at the last-subject position and the last-relation position. Since the hidden representations at the last-subject position is found out to encode the subject-related attributes via projection onto the vocabulary space and are significantly most decisive for processing the subject tokens (Geva et al., 2023; Meng et al., 2022a), they can be viewed as the subject feature of factual prompts that models learn. The phenomena are similar for the hidden representations at the last-relation position for relation tokens (Liu et al., 2024), which can be viewed as the relation feature of factual prompts. Therefore, we conclude that hidden representations at the last subject position and the last relation position are two crucial features for auto-regressive language models to learn during optimization, namely the subject feature and the relation feature. RIE(mlj i ) = log IE(o∗ , θedited,mlj i ) IE(o, θo, mlj i ) (3) This metric isolates the relative gain or loss in causal significance for each component. We analyze the maximum RIE of MLP outputs across all layers at each token position over 100 factual edits. Figure 3 presents these results, with a detailed case study provided in the Appendix. The empirical results reveal a critical pathology: the causal contribution of the subject representation (hs) increases exponentially after editing, while the relation representation (hr) shows much less gain. This suggests that the optimization process has decoupled the prediction from the relational context. We formalize this phenomenon as follows: Definition 1 (Subject-Dominant Shortcut). Given a factual association ⟨s, r⟩ → o∗ , an editing process converges to a Subject-Dominant Shortcut if the learned conditional probability distribution Pθ∗ exhibits approximate conditional independence of the object o∗ from the relation r, given the subject s: Pθ∗(o∗ | s, r) ≈ Pθ∗(o∗ | s) (4) This degeneracy manifests structurally when the causal influence of the subject representation dominates the computation, effectively marginalizing the relational operator such that IE(hs) ≫ IE(hr). This definition implies a causal path collapse. In a healthy model, the prediction follows a compositional path f(s, r). However, existing optimization forces the subject vector vs to absorb the target information o∗ directly. Mathematically, the optimizer finds a solution where the mapping approximates vs → o∗ , bypassing the relational gate. Consequently, when queried with the same subject but a different relation (e.g., ⟨s, r′ ⟩), the dominant subject pathway triggers the retrieval of o∗ , resulting in the severe subject-dominant memory interference (i.e., poor relation specificity) observed

3.3 The Shortcut Learning Issue in existing methods.

The subject feature and the relation feature during optimization are revealed in the previous section. We further observe how well these two features are learned through ROME editing. 5886

log ratio of indirect effect

5.0

4.0

3.0

2.0

1.0

0.0

rp fs ms ls fr mr lr

4 Eliminating Shortcut Learning via Causal Path Alignment

295 296 297 298 299 300 301 302 Having identified the collapse of the causal path as the root cause of subject-dominant memory interference, we propose a principled optimization framework to rectify this pathology. The standard editing objective, which minimizes the negative log-likelihood -logP(o∗ |vs), is fundamentally underconstrained: it demands only that the target o∗ be predicted, without specifying how. Given the pre-existing dominance of subject features, the optimizer inevitably exploits the direct shortcut vs → o∗ . To dismantle this shortcut, we must explicitly regularize the optimization trajectory, forcing the information flow to traverse the relational computational path. We term this approach Causal Path Alignment (CPA), which effectively realigns the model's internal causal mechanism.

303 304 4.1 Theoretical Framework: Causal Path Alignment via Anchoring

305 306 307 308 309 310 311 Our strategy hinges on anchoring the prediction to the relation feature. Let hr denote the hidden state at the last relation token position in a layer la (la > l) subsequent to the edit layer. In the standard forward pass of a LLM, the subject injection vs propagates through intermediate layers to form hr, which then dictates the final prediction [Geva et al., 2023]. Formally, this dependency can be modeled as a Markov chain: vs → · · · → hr → · · · → o∗ . Ideally, to ensure the relation r mediates the prediction, we should maximize the joint likelihood P(o∗ , hr | vs). By applying the chain rule of probability, we decompose this objective into:

315 316 317 318 319 320 321 The first term, P(o∗ | hr, vs), represents the probability of the target object given both the relation feature and the subject injection. The structural flaw in current locate-then-edit editing is that the model learns to rely on vs directly to predict o∗ , rendering hr redundant (Sec. 3). To eliminate this shortcut, we introduce a crucial Conditional Independence Constraint that we enforce the prediction of the object o∗ should depend solely on the relation feature hr, making it independent of the specific subject injection vs given hr. Mathematically, this imposes o∗ ⊥ vs | hr, which simplifies the first Prediction Head term in Eq. 5 to P(o∗ | hr).

322

323

324

325

326

Consequently, our refined optimization objective maximizes the decomposed likelihood:

This decomposition provides the theoretical foundation for our two-stage optimization process. The first term, representing Relation Anchoring, establishes a valid mapping from the relation feature to the target object, creating a "virtual anchor" in the activation space that encodes relation semantics decoupled from the subject. The second term, representing Trajectory Alignment, ensures that the edited subject vector vs correctly reconstructs this pivotal relation feature. By optimizing these terms sequentially, we force the optimization path to be anchored to the intermediate state hr, thereby strictly prohibiting the bypass of relational semantics.

4.2 Implementation: Two-Phase Optimization

We implement CPA as a sequential optimization process that anchors the update trajectory to the causal structure derived in Eq. 6. As shown in Fig. 4, we sequentially optimize the decomposed likelihood terms to enforce relational mediation.

Phase 1: Relation Anchoring. We first optimize a relation anchor h∗r by maximizing logP(o∗ | hr) at the bottleneck layer la. This shifts the prediction to o∗ while operating solely in the activation space to avoid parameter overfitting. We solve h∗r = argminhr L1(hr) via:

The KL divergence against the pre-edit state hor confines the anchor to the local semantic manifold, ensuring the edited relation remains distinguishable from unrelated facts.

Phase 2: Trajectory Alignment. Fixing h∗r , we maximize logP(hr | vs) by optimizing the subject injection vector vs (the output of the MLP down-projection at the editing layer l) to accurately reconstruct the relation anchor. The updated vector v∗ s is obtained by minimizing the discrepancy in the feature space:

where F[vs, p] is the forward propagation result in hr given vs and the prompt p. By enforcing vs to reconstruct h∗r , this step compels the vector update to traverse the established causal path, effectively eliminating the shortcut learning pathology. All weight decays are omitted for simplicity.

5 Experiments

5.1 Experimental Setup

Datasets and Evaluation Metrics. We evaluate CPA under the standard single-edit protocol, where each run rewrites one factual association ⟨s, r, o⟩ → ⟨s, r, o∗ ⟩ and quantifies both the success of the rewrite and the unintended spillover to other memories.

Our primary benchmark is COUNTERFACT RS [Liu et al., 2025], a relation-sensitive evaluation suite derived from COUNTERFACT, on which we sample 2,000 edits and repeat evaluation five times, reporting mean and standard deviation. COUNTERFACT RS provides a controlled decomposition of editing quality into Efficacy (whether the rewritten object is preferred under rewriting prompts), Subject Specificity (SSPEC., whether the new object spuriously transfers to neighborhood subjects ⟨s′ , r⟩), and Relation Specificity (R-SPEC., whether the new object leaks to other relations of the same subject ⟨s, r′ ⟩). Because our goal is to eliminate SubjectDominant Memory Interference, we treat R-SPEC. as the key controllability metric, while also reporting a Fluency score that captures the quality of generated continuations under the benchmark protocol.

To complement the evaluation with a broader subjectdominant memory interference assessment, we also evaluate on the EVOKE [Zhang et al., 2025] benchmark and report corresponding metrics: DP (Deterioration Probability) to measure how often unrelated answers are disrupted after editing, CAP (Correct Answer Probability) to track overall correctness, and EOS (Editing Overfit Score) to capture over-confident collapse, serving as a primary indicator of the model's specificity and overall performance; lower DP and higher EOS indicate stronger resistance to interference.

Baselines and Model Architectures. We compare against representative locate-then-edit editors, including ROME [Meng et al., 2022a], MEMIT [Meng et al., 2022b], and AlphaEdit [Fang et al., 2024], as well as recent mitigation-oriented variants such as RETS [Liu et al., 2025] and ROME-LTI [Zhang et al., 2025]. Experiments span multiple LLM families and scales, including GPT2-XL (1.5B), GPT-J (6B) [Wang and Komatsuzaki, 2021], and Qwen2.5 (7B/14B). In particular, we report the main results for ROME-CPA because ROME is the representative and most widely adopted single-edit backbone, and it cleanly decomposes editing into (i) optimizing the target value representation and (ii) a closed-form rank-one weight update at the located MLP module. This decomposition makes the shortcut pathology and our trajectory-alignment intervention directly comparable. We nevertheless verify portability by transplanting CPA to other locate-then-edit methods in subsequent experiments.

Implementation Details. For each backbone, we follow the commonly adopted layer choices for locate-then-edit updates and select a later bottleneck layer la > l for relation anchoring. Concretely, for 48-layer models we edit the 18th and 19th MLP layers and anchor at la ∈{19, 20} (GPT2-XL) or la=28 (Qwen2.5 14B), while for 28-layer models we edit layer 6 and anchor at la=11 (GPT-J and Qwen2.5 7B). More details can be referred to the Appendix.

5.2 Main Results

Table 1 summarizes the results on COUNTERFACT RS and EVOKE across multiple LLM backbones.

CounterFact RS. Across all evaluated models, CPA consistently improves relation specificity (R-Spec.), indicating a substantial reduction of Subject-Dominant Memory Interference. Concretely, compared with ROME, CPA raises R-Spec. from 45.7% to 72.2% on GPT2-XL, from 52.8% to 83.5% on GPT-J, from 70.7% to 92.2% on Qwen2.5 7B, and from 56.9% to 84.5% on Qwen2.5 14B, while maintaining high editing success (Eff.) and strong subject specificity (S-Spec.). Notably, this gain does not come from overly restrictive constraints: fluency remains comparable to ROME/MEMIT, whereas methods with aggressive constraints (e.g., RETS)

can exhibit pronounced fluency degradation and less stable cross-model behavior.

EVOKE. EVOKE directly probes collateral corruption and shortcut-induced overfitting. CPA achieves the lowest (or second-lowest) damage proxy DP and the best EOS on most settings, suggesting that the edited behavior is less dominated by a subject-only shortcut and induces fewer unintended changes. While RETS can reach strong CAP/EOS on some settings, it often does so with a notable trade-off in fluency and/or robustness; in contrast, CPA provides a more balanced improvement profile across architectures.

Overall, the main results demonstrate that CPA provides a reliable remedy for Subject-Dominant Memory Interference across both benchmarks and model families. CPA consistently yields large gains in relation specificity and further reduces detailed damage indicators on EVOKE, while preserving (and often saturating) editing efficacy and maintaining competitive fluency. These findings support our central claim: anchoring the optimization to a relation-aware causal

60

40

20

0

Efficacy S-Spec.

60

40

20

0

Efficacy S-Spec.

R-Spec.

Figure 5: Ablation study of ROME-CPA on COUNTERFACT RS.

R-Spec. GPT-J

Left: GPT2-XL (1.5B). Right: Qwen2.5 (14B).

bottleneck and aligning the update trajectory to reconstruct this anchor prevents causal path collapse, enabling edits that are both effective and controllable.

5.3 Mechanistic Validation

Beyond aggregate metrics, we verify that CPA indeed restores the relational causal pathway rather than merely suppressing outputs. Following Section 3.3, we re-run causal tracing and compute the maximum Ratio of Indirect Effect (RIE) over layers for each token position (Fig. 6). The key signature of shortcut elimination is that the relative causal gain at the last relation position becomes closer to that at the last subject position, indicating that the edited prediction is mediated by relation-aware features instead of a subject-only bypass. This mechanistic evidence aligns with the consistent improvements in relation specificity and the reductions in EVOKE side-effect indicators.

5.4 Ablation Study

We conduct ablation experiments on 2,000 edits from COUNTERFACT RS to disentangle the contribution of each phase in Causal Path Alignment (CPA). We compare three configurations under identical prompts, stopping criteria, and hyperparameters: (1) Phase 1 only performs relation anchoring by optimizing the bottleneck activation hr at the last-relation token position to minimize - log P (o∗ | hr), but omits the subsequent trajectory alignment; (2) Phase 2 only reduces to the conventional single-phase target construction by directly optimizing the injected subject value vector vs at the last-subject token position with the standard objective -logP(o∗ | vs); and (3) Full CPA executes the complete two-phase procedure (relation anchoring followed by trajectory alignment).

Score (%)

Score (%)

Results in Fig. 5 confirm the synergistic necessity of both phases. We observe a distinct trade-off: Phase 1-only secures relation specificity but suffers from catastrophic subject over-generalization (S-Spec. drops to ∼20-35%), whereas Phase 2-only restores subject focus but reverts to the shortcut pathology, degrading relation specificity to ∼46%. Only the full CPA reconciles these conflicting objectives, achieving an optimal balance (e.g., Qwen2.5: 83.4% S-Spec., 84.5% RSpec.) while maintaining near-saturated efficacy.

5.5 Plug-and-Play Generalization to Other Editors

CPA is editor-agnostic: it constrains how the target representation is optimized, and thus can be transplanted to other locate-then-edit pipelines without altering their localization or weight-update solvers. To test this, we integrate CPA into MEMIT and AlphaEdit by replacing their original target-vector optimization with CPA while keeping their original layer selections and update rules. As shown in Table 2, CPA substantially improves R-Spec. for both editors on GPT2-XL and Qwen2.5 7B (e.g., MEMIT: 61.5%→79.7% and 64.7%→90.6%; AlphaEdit: 41.2%→68.4% and 65.0%→88.9%), while largely preserving efficacy and S-Spec. This indicates that the shortcut pathology stems from the unconstrained target-vector optimization, and CPA provides a general remedy by anchoring the optimization trajectory to relation-aware causal states.

6 Conclusion

In this work, we diagnose the Subject-Dominant Shortcut-a pathology in locate-then-edit methods where optimization decouples predictions from relational context, causing severe subject-dominant memory interference. To rectify this, we propose Causal Path Alignment (CPA), a framework that anchors parameter updates to valid internal causal pathways. By enforcing a conditional independence constraint via a two-stage process, CPA ensures updates are mediated by relational features rather than unconditional subject mappings. Experiments across diverse LLMs demonstrate that CPA significantly boosts relational specificity without compromising efficacy. As a model-agnostic plug-in, CPA effectively resolves the plasticity-stability trade-off, paving the way for trustworthy parametric memory updates in evolving AI agents.

Ethical Statement

Our research focuses on addressing the challenge of outdated or erroneous knowledge encoded in large language models by developing methods to detect and correct such inaccuracies. At the same time, we are acutely aware of the ethical risks this technology entails, particularly the possibility of misuse for disseminating harmful or misleading content. To mitigate these risks, we underscore the critical importance of two key principles: first, language models must be obtained from rigorously vetted and reliable sources; second, users must exercise utmost caution and critical judgment when utilizing outputs generated by these models.

In the preparation of this manuscript, AI-assisted tools (including large language models) are employed strictly for the purpose of polishing and textual refinement to enhance clarity and organization. The results have been further controlled and edited. We retain full responsibility for the core scientific questions, methodological design, theoretical innovation, experimental analysis, and all conclusions presented. AI plays no role in the creative scientific conceptualization process.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Nos. 62472419, 62472420) and the Enterprise Project (No. E4V06811F3).