System 1&2 Synergy via Dynamic Model Interpolation

1 Introduction

The cognitive evolution of Large Language Models (LLMs) has established a functional dichotomy: lightweight "System 1" models excel at rapid, intuitive responses, while "System 2" models such as OpenAI's o1 [OpenAI, 2024] and DeepSeek-R1 [DeepSeek-AI et al., 2025] achieve superior reasoning through extended Chain-of-Thought (CoT) [Li et al., 2025]. Ideally, a unified model would seamlessly adapt between these modes, responding swiftly to simple queries while engaging deep deliberation for complex problems [Team, 2025]. However, training such an adaptive model remains an open challenge due to the zero-sum nature of training data [Shukor et al., 2025]. The verbose, selfreflective style of System 2 reasoning conflicts with the concise responses of System 1, causing mutual interference during joint optimization. Consequently, state-of-the-art LLMs typically release separate Thinking and Instruct checkpoints rather than a single unified model [Bai et al., 2025].

Given this difficulty, the community has pursued a more tractable alternative: making System 2 models more efficient. This pursuit has spawned various approaches, from trainingfree methods like token budgeting [Han et al., 2025], early exit [Yang et al., 2025c], and CoT compression [Xia et al., 2025], to training-based methods employing reinforcement learning (RL) with length penalties [Arora and Zanette, 2025; Su and Cardie, 2025] or trajectory pruning [Dai et al., 2025;

Hou et al., 2025]. Despite their diversity, these approaches share a common paradigm: output control, which limits what models produce. We argue that this framing is misaligned: output length is merely a symptom of the model's cognitive configuration, not the root cause of inefficiency. As illustrated in Figure 1, constraining generation with token budgets risks truncating the reasoning process, resulting in incomplete chains of thought that severely degrade performance on challenging tasks.

In this work, we shift the focus from output control to capability control, which modulates how models think rather than what they produce. Our key observation is that by interpolating parameters between existing System 1 and System 2 checkpoints, we can smoothly adjust the model's cognitive intensity without additional training. As shown in Figure 1 (upper), under comparable efficiency constraints, capability control consistently outperforms output control by producing naturally concise yet complete solutions.

A natural question arises: does parameter interpolation yield predictable behavior? Our pilot experiments in Section 3.1 provide affirmative evidence. We find that increasing the interpolation coefficient λ (where λ=0 corresponds to Instruct and λ=1 to Thinking) produces a smooth, monotonic Pareto frontier between accuracy and efficiency, as shown in Figure 1 (lower), with continuous transitions in representation space. This predictability is grounded in the structural similarity between models: cosine similarity exceeds 0.99 across all layers, indicating that both models satisfy Linear Mode Connectivity [Frankle et al., 2020] and reside within the same optimization basin. These findings establish parameter interpolation as a principled mechanism for capability control.

Building on this foundation, we propose DAMI (DynAmic Model Interpolation), a framework that dynamically estimates a query-specific Reasoning Intensity λ(q) ∈ [0, 1] to configure the model's cognitive depth for each input. We explore two complementary methods tailored for different deployment scenarios: DAMI-Pref, a preference learning method that jointly encodes accuracy and efficiency criteria, is suited for accuracy-critical applications where in-domain training data is available; DAMI-Conf, a confidence-based method that leverages inter-model cognitive discrepancy, enables rapid deployment across diverse domains without requiring any training data. Experiments on five mathematical reasoning benchmarks show that DAMI improves accuracy by 1.6-3.4% while reducing token consumption by 29-40% compared to the Thinking model, outperforming static merging, early-exit, and routing baselines.

Our contributions are summarized as follows:

• We advocate a paradigm shift from output control to capability control for efficient reasoning, and identify dynamic model interpolation as a low-cost realization that leverages existing checkpoints, bypassing the difficulty of training System1&2 unified models
• We propose DAMI with two complementary λ(q) estimation methods: preference-based learning for dataavailable scenarios and confidence-based inference for zero-shot deployment.
• Extensive experiments demonstrate that DAMI consis-

tently establishes superior Pareto frontiers across diverse benchmarks and model families.

2 RelatedWork

2.1 Efficient Reasoning

The inherent trade-off between System 1's efficiency and System 2's accuracy has motivated a growing body of research on efficient reasoning, which can be categorized into single-model optimization and multi-model collaboration. Single-model approaches include early-exit methods that terminate generation upon sufficient confidence [Yang et al., 2025c; Dai et al., 2025], CoT compression that removes redundant reasoning steps [Xia et al., 2025], adaptive reasoning that adjusts output length based on query complexity [Luo et al., 2025; Shen et al., 2025], and other trajectory-revised methods [Yang et al., 2025b]. While adaptive methods share our motivation, they rely on RL with length penalties, incurring high training costs and potential performance degradation on difficult problems. Despite their diversity, these single-model approaches predominantly follow the output control paradigm, constraining what models produce. Multi-model approaches leverage complementary capabilities across models. Task decomposition methods delegate simple steps to lightweight models while reserving complex reasoning for stronger ones [Akhauri et al., 2025; Fan et al., 2025]. LLM Routing dynamically dispatches queries to suitable models from a pool [Ong et al., 2025; Zhang et al., 2025]. Model merging statically integrates parameters of different models to combine their strengths [Wu et al., 2025a; Yao et al., 2025]. Our work bridges routing and merging by enabling continuous, query-adaptive interpolation rather than discrete model selection or static merging.

2.2 Model Merging

Model merging integrates parameters from multiple specialized models into a unified model without requiring original training data. It has been applied to mitigate catastrophic forgetting in continual learning [Schumann et al., 2023], integrate knowledge in multi-task learning [Yang et al., 2024; Ilharco et al., 2023], and aggregate updates in federated learning [Wang et al., 2020]. Recent work has introduced merging into efficient reasoning, with leading LLM teams releasing merged models that balance reasoning depth and efficiency [Wu et al., 2025b; Wang et al., 2023; Yang et al., 2025a]. However, existing approaches apply fixed merging coefficients across all queries. To our knowledge, we are the first to propose query-aware dynamic merging between System 1 and System 2 models, enabling fine-grained integration of their complementary strengths for adaptive control over reasoning depth.

3 Methodology

3.1 Why Interpolation Enable Capability Control

Before formulating our dynamic framework, we first validate that parameter interpolation can serve as a continuous and predictable control mechanism for reasoning. We conduct a pilot study using static linear interpolation between the Instruct model Θ(I) and the Thinking model Θ(T) with a fixed coefficient λ ∈ [0, 1] across the test set.

Performance Monotonicity. As shown in Figure 1, increasing λ from 0 to 0.9 yields a smooth improvement in accuracy alongside a predictable increase in token consumption. Crucially, this trajectory forms a convex Pareto frontier where intermediate interpolations can outperform both endpoints: achieving higher accuracy than the Instruct model while requiring fewer tokens than the Thinking model. This confirms that λ serves as a reliable proxy for reasoning depth, enabling fine-grained efficiency-accuracy trade-offs.

Representation Continuity. What underlies this behavioral smoothness? We analyze the hidden states of ⟨think⟩ token using Principal Component Analysis (PCA) across 50 samples on the MATH-500. As shown in Figure 2(a), representations of interpolated models form a continuous trajectory connecting the Instruct and Thinking endpoints. The first principal component strongly correlates with λ (r=0.974), indicating that interpolation induces a gradual cognitive transition rather than discrete jumps in the representation space.

Structural Connectivity. This representational continuity is grounded in the geometric properties of the parameter space. Figure 2(b) shows that the Instruct and Thinking models maintain high cosine similarity (>0.99) across all transformer layers, satisfying the Linear Mode Connectivity (LMC) property [Frankle et al., 2020; Zhan et al., 2025]. This indicates that both models reside within the same optimization basin, ensuring that interpolation traverses a valid path in the loss landscape. Notably, the L2 distance peaks in middle-to-deep layers, suggesting that System 2 capabilities are encoded through targeted modifications in these regions rather than global parameter shifts.

Key Insight. These findings establish a causal chain: structural alignment in parameter space → continuous transitions in representation space → monotonic Pareto frontier in performance. This connectivity transforms reasoning resource allocation into a well-posed prediction problem, where estimating the optimal λ(q) directly determines model behavior. These findings jointly support interpolation validity, performance monotonicity, and representation continuity. Together, these properties ensure that model interpolation behaves predictably and monotonically, providing the theoretical foundation for fine-grained, query-level dynamic merging. This validates our core premise: λ can serve as a reliable control knob for reasoning intensity, enabling the adaptive framework we introduce next.

3.2 The Dynamic Interpolation Framework

While Section 3.1 validates that parameter interpolation provides a reliable control mechanism, a fixed λ is inherently suboptimal since different queries require different reasoning depths. DAMI addresses this by making λ query-dependent:

We term λ(q) ∈ [0, 1] the Reasoning Intensity, which modulates the cognitive depth for each input. The key technical challenge is estimating the optimal λ(q). We explore two complementary methods: DAMI-Pref, a preference learning approach that jointly encodes accuracy and efficiency criteria, suited for accuracy-critical applications where in-domain training data is available; and DAMI-Conf, a confidence-based approach that leverages inter-model cognitive discrepancy, enabling rapid deployment across diverse domains without requiring any training data.

3.3 Training-based Estimation of λ

Baseline: Regression-based Estimation

A straightforward solution is to train a lightweight router model Rϕ to regress a pre-computed target λ∗ (q). We profile each query q across J discrete coefficients Λ = {l1, . . . , lJ}, measuring accuracy via N-sample evaluation:

where Mlj n (q) is the response from a model merged with coefficient lj, and I(·, ·) is a binary evaluation function. The target λ∗ (q) is defined as the minimum coefficient achieving the highest accuracy, and the router is trained with MSE loss:

However, this approach suffers from two weaknesses: (1) point-wise labels λ∗ (q) are inherently noisy due to sampling variance, and (2) regression requires large-scale data for robust generalization. These limitations motivate our preference-based method.

DAMI-Pref: Preference-based Estimation

Instead of regressing noisy labels, we recast λ estimation as a preference learning problem. A lightweight reward model Rψ is trained to score the relative efficacy of different coefficients, offering three advantages: (1) pairwise comparisons are more robust to noise, (2) preference criteria can jointly encode accuracy and efficiency, and (3) the method requires substantially less training data.

Preference Data Construction. We construct policy performance tuples P(q, lj) = ⟨lj,Acc(q, lj), Cost(q, lj)⟩, where Cost is the average token count. Preference pairs are generated according to a hierarchical criterion:

where δacc is a small tolerance threshold to account for sampling noise. This criterion prioritizes accuracy while using efficiency as a tiebreaker, directly reflecting the practical goal of achieving high performance at minimal cost. Pairs without clear preference are discarded to ensure signal quality. This process is applied to all pairs to construct the final dataset:

Optimization and Inference. The reward model is optimized via binary cross-entropy loss:

3.4 Training-free Estimation of λ

Baseline: Prompt-based Estimation

A straightforward approach is to leverage the model's incontext learning for difficulty assessment. We design a zeroshot prompt that instructs the Instruct model to rate query difficulty on a scale from 1 to 9, then linearly scale to obtain λ(q). However, this method depends heavily on instructionfollowing capability and generalizes poorly across model families.

DAMI-Conf: Confidence-based Estimation

Considering the above drawbacks, we propose to derive λ(q) from intrinsic confidence signals, offering two advantages: model-agnostic signals that generalize across architectures, and continuous-valued output for fine-grained control.

Confidence Extraction. Following [Yang et al., 2025c], we define confidence as the geometric mean of maximum token probabilities over the answer sequence a = (a1, . . . , an):

where i is an answer-inducing prompt like "Final answer is: \boxed{". We compute C I (q) and C T (q) for the Instruct and Thinking models respectively.

Dual-Signal Fusion. We construct λ(q) from two complementary signals that capture different aspects of query difficulty. Holistic Ambiguity measures overall system uncertainty. When neither model is confident, the query is likely challenging and requires deeper reasoning:

Cognitive Discrepancy quantifies the disagreement between models by measuring the difference in their confidence. A high discrepancy suggests that the two models employ different strategies for addressing the query, indicating potential ambiguity that requires extensive reasoning to resolve:

The fused signal Sfinal = Sambi + Sdis is transformed via calibrated sigmoid to yield Reasoning Intensity:

where µ and τ control the decision boundary and sensitivity.

4 Experiments

4.1 Experimental Setup

Models and Datasets. We conduct experiments on the following two pairs of open-source models: (Qwen3-4B2507-Thinking, Qwen3-4B-2507-Instruct) and (DeepSeekR1-Distill-Qwen-7B, Qwen2.5-Math-7B). The evaluation benchmarks primarily consist of mathematical reasoning tasks, encompassing five specific datasets ranging from simple to difficult: GSM8K [Cobbe et al., 2021], MATH-500 [Hendrycks et al., 2021], AMC 2023, AIME 2024, and AIME 2025. For the training of the router model and reward model, we select training data from DeepMath [He et al., 2025], which is out-of-distribution (OOD) relative to the evaluation sets. Since this dataset inherently contains difficulty labels, we first conduct an initial screening based on these labels, selecting an average of 1,000 samples across various difficulty levels for constructing the training data described in Section 3.2.

Baselines. We compare our methods (DAMI-Pref, DAMIConf) against three categories of approaches: (1) output control methods, including the early-exit approach DEER [Yang et al., 2025c]; (2) static capability control methods, including Task Arithmetic (TA) [Ilharco et al., 2023], TIESMerging (TIES) [Yadav et al., 2023], and Model Interpolation (MI) [Wu et al., 2025b], which apply fixed merging coefficients across all queries; and (3) dynamic capability control methods, including prompt-based LLM Routing, as well as two strong dynamic merging baselines introduced in this work (DAMI-Pred, DAMI-Prompt). We also include the two original models (Instruct, Thinking) as reference endpoints. The hyperparameters for all baseline methods are set strictly according to the original papers.

Metrics and Implementations. We selected Accuracy (Acc n×), Token Number (Tok #N), and Thinking Ratio (Think #R) as our evaluation metrics. Acc n× denotes the final answer accuracy, and n × represents the number of sampling rounds conducted for evaluation. Tok #N indicates the average number of generated tokens per sample, serving as a measure of computational cost. Think #R is defined as the percentage of responses containing the ⟨/think⟩ token, quantifying the prevalence of explicit CoT reasoning. Given the limited sample sizes in the AMC 2023, AIME 2024, and AIME 2025 datasets, we perform 8 sampling rounds per instance and average the results across all metrics to ensure statistical stability and reliability. Across all models, we consistently apply the hyperparameters T = 0.6 and Top-p= 0.95. For confidence-based estimation, we set µ = 0.3 and τ = 0.3.

4.2 Experimental Results

Overall Performance. As shown in Table 1, DAMI-Pref and DAMI-Conf consistently achieve the best performance across both model families. On Qwen3-4B pair, DAMI-Pref attains 91.6% accuracy (+3.4% absolute) while reducing token consumption by 29% compared to vanilla Thinking. DAMI-Conf further improves efficiency with 40% token reduction while maintaining 90.7% accuracy (+2.5% absolute). Figure 4 visualizes the overall accuracy-efficiency trade-off: DAMI-Pref and DAMI-Conf consistently occupy the Pareto-optimal region, dominating both output control methods and static capability control methods across both model pairs.

Comparison of Strong Dynamic Merging Baselines. The DAMI-Pred consistently underestimates required reasoning intensity, achieving only 83.3% accuracy versus 91.6% for DAMI-Pref on Qwen3-4B. The DAMI-Prompt performs reasonably on Qwen3-4B but degrades severely on 7B, as Qwen2.5-Math-7B lacks robust instruction-following for difficulty assessment. In contrast, DAMI-Pref benefits from pairwise comparisons that are robust to label noise, while DAMI-Conf leverages model-agnostic confidence signals that generalize across architectures.

Figure 7: Generalization to multimodal tasks of DAMI.

Comparison with SoTA Methods. DEER achieves comparable accuracy to DAMI-Pref on Qwen3-4B but consumes more tokens and risks truncated reasoning. LLM Routing suffers from its discrete nature: on Qwen2.5-7B, it collapses to 42.0% accuracy due to the weak System 1 model, while our continuous interpolation maintains robust performance. Static merging methods (TA, TIES, MI-λ) cannot adapt to varying difficulty distributions. While MI-07 achieves strong results on Qwen3-4B, our dynamic approach consistently outperforms the best static configuration across benchmarks. Analysis of Thinking Ratio. The Thinking Ratio reveals adaptive resource allocation. On simpler benchmarks like GSM8K, DAMI-Pref and DAMI-Conf activate deep thinking for only 25-41% of queries on Qwen3-4B. As difficulty increases toward AIME24/25, the ratio rises to 79-95%. This adaptive behavior distinguishes our approach from static methods, which either overthink simple queries (Thinking, TA) or underthink difficult ones (TIES, MI-03). The smooth variation demonstrates that DAMI successfully calibrates reasoning depth to query complexity.

4.3 Analysis

Ablation study. Figure 5 presents ablation results on the Qwen3-4B model pair, examining the contribution of key design components. For DAMI-Pref, removing the cost-based tiebreaker (-Cost) slightly improves accuracy but increases token consumption by 18.0%. This confirms that the hierarchical preference criterion effectively balances accuracy and efficiency. For DAMI-Conf, both signals contribute to the final performance. Removing Cognitive Discrepancy (-Sdis) causes a 3.7% accuracy drop, while removing Holistic Ambiguity (-Sambi) results in a 1.9% drop. Using only the Thinking model's confidence (C T (q)-only) degrades accuracy to 84.5%, demonstrating that the inter-model comparison is essential for reliable difficulty estimation.

Training data efficiency of DAMI-Pref. Figure 6 compares DAMI-Pred and DAMI-Pref under varying training data sizes. DAMI-Pref achieves near-optimal performance with only 1K samples and shows minimal improvement beyond, demonstrating strong robustness to limited supervision. In contrast, DAMI-Pred requires significantly more data to stabilize: it does not surpass the Thinking baseline until 4K samples. This gap confirms our motivation for preference-based estimation: pairwise comparisons provide more robust learning signals than pointwise regression, particularly in low-data

regimes where label noise has greater impact.

Case study. Figure 8 compares responses on a geometry problem from MATH-500. The Thinking model produces 2,846 tokens with typical overthinking patterns. DEER reduces output to 1,188 tokens through early termination, but cannot skip the lengthy content before the exit marker. This reflects a fundamental limitation of output control methods. LLMRouting, constrained to discrete model selection, routes this query to Thinking and yields 3,017 tokens. In contrast, DAMI generates only 351 tokens with qualitatively different behavior: direct problem formulation, concise calculation, and no redundant verification. This illustrates the core advantage of capability control: DAMI produces inherently efficient reasoning by modulating cognitive configuration.

Generalization on Multimodal Tasks. Beyond text-only models, we evaluate DAMI on the multimodal model Qwen3VL-8B [Bai et al., 2025]. We curate a diverse evaluation suite comprising 5 video understanding benchmarks (VideoHolmes, MMVU, VideoMMMU, VideoMME, LVBench) and 4 image-text reasoning benchmarks (WeMath, MathVista, MMMU, MathVision). As shown in Figure 7, DAMI-Conf consistently outperforms all baselines across both modalities. On video understanding tasks, DAMI-Conf achieves 64.0% average accuracy, surpassing the Thinking model by 2.5% absolute. On image-text reasoning tasks, DAMI-Conf attains 66.5%, a 1.6% improvement over Thinking. These findings confirm that our DAMI framework generalizes effectively to widely multimodal tasks. Moreover, it reveals that DAMI not only enables efficient reasoning, but also effectively integrates the complementary knowledge and reasoning capabilities of both models, leading to consistent performance gains across diverse benchmarks.

Computational cost. Figure 9 presents end-to-end latency. Despite introducing additional steps for λ prediction and parameter merging, both methods achieve substantial speedups over vanilla Thinking. The overhead from λ prediction is modest: DAMI-Pref only requires scoring each candidate λ with a lightweight reward model, while DAMI-Conf extracts confidence from short answer-only generations rather than full reasoning chains. Parameter merging adds ∼5.3s when loading models on-demand, but this can be reduced to under 1s by pre-loading both checkpoints into GPU memory. These overheads are negligible compared to generation time, which dominates total latency. By reducing output length by 35% (DAMI-Pref) and 56% (DAMI-Conf), our methods achieve 1.46× and 1.86× end-to-end speedups respectively.

5 Conclusion

This paper addresses the challenge that System 1's efficiency and System 2's reasoning capability cannot be readily unified within a single model. Unlike existing methods that constrain what models produce (output control), we propose capability control, which modulates how models think through dynamic parameter interpolation. Our DAMI framework offers two complementary methods, training based DAMI-Pref and training free DAMI-Conf, to achieve query-adaptive reasoning intensity. Beyond the technical contributions, this work suggests a broader perspective: the dichotomy between fast and slow thinking need not be a binary choice resolved at training time. The monotonicity and continuity properties we establish indicate that reasoning depth exists on a smooth spectrum, accessible through simple linear operations in parameter space.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Nos. 62472419, 62472420) and the Enterprise Project (No. E4V06811F3).

Contribution Statement

Chenxu Yang and Qingyi Si contributed equally to this work. Zheng Lin is the corresponding author.