Sprint or Delve: A Distribution-Aware Approach to Efficient Reasoning

1 Introduction

Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, largely driven by Chain-ofThought (CoT) prompting mechanisms [Zhou et al., 2023; Chen et al., 2024; Nye et al., 2022; Wei et al., 2022]. However, this performance enhancement often comes with a relatively static computational paradigm: in practice, models tend to allocate similar generation budgets, measured in output token length, regardless of the underlying problem difficulty. For instance, current state-of-the-art reasoning models, such as DeepSeek-R1 [Guo et al., 2025], often generate extensive reasoning chains even for trivial arithmetic instances commonly found in reasoning benchmarks, resulting in significant redundancy and inference latency [Wan et al., 2024; Sprague et al., 2025]. This inefficiency highlights a critical misalignment between the model's computational expenditure and the actual difficulty of the task.

To mitigate this overhead, recent research has explored methods to incentivize concise reasoning [Sui et al., 2025]. While prompt engineering [Han et al., 2024; Ding et al., 2024] and supervised fine-tuning on variable-length data [Xia et al., 2025b; Ye et al., 2025; Ma et al., 2025] offer partial solutions, Reinforcement Learning (RL) with lengthbased reward shaping has emerged as a promising direction [Arora and Zanette, 2025; Luo et al., 2025; Li et al., 2025a; Xiang et al., 2025; Liu et al., 2025; Zhang et al., 2025; Li et al., ].

Despite their success, most existing approaches rely on linear or normalized length penalties. We argue that such formulations are fundamentally limited because they ignore the variation in problem difficulty. Blindly compressing reasoning chains may lead to more concise responses for simple problems; however, applying the same degree of compression to complex tasks often degrades the model's reasoning capability and can even suppress its ability to perform deep, multi-step reasoning. This leads to a critical dilemma: a penalty strong enough to compress simple tasks will inadvertently suppress the deep reasoning required for hard tasks, whereas a penalty weak enough to preserve accuracy on hard tasks fails to reduce redundancy on easy ones. This distributional mismatch raises an important research question:

Can we design a length penalty mechanism that naturally adapts to the variance inherent in reasoning chains of varying complexities?

Empirically, our analysis of reasoning chain lengths yields two primary observations.

First, we find that reasoning length serves as a quantifiable proxy for cognitive complexity. As illustrated in Figure 1, there is a clear positive correlation between the length of reference solutions and problem difficulty across varying datasets. This aligns with established findings in cognitive science, where complex problems necessitate deeper, more structured, and multi-step reasoning processes [Kahneman, 2011; Perez et al., 2020].

Second, we observe significant variability in the reasoning lengths generated for identical problems. Rather than clustering tightly, the length distribution for a single prompt exhibits a pronounced heavy-tailed behavior. Through statistical fitting, we demonstrate that this variability is wellapproximated by a Log-Normal distribution across diverse benchmarks.

Motivated by above findings, we treat reasoning length as an implicit indicator of task complexity. In contrast to prior approaches, we design a reinforcement learning framework that leverages the variance structure of the Log-Normal distribution to adaptively regulate reasoning length. Rather than imposing a fixed compression strength, our approach estimates task difficulty from the generated response length itself and dynamically adjusts the penalty applied to the model. This design aims to reduce redundant computation on simple problems while preserving, or even enhancing, performance on more complex reasoning tasks.

Our contributions can be summarized as follows:

• We empirically demonstrate that reasoning lengths across diverse models approximate a Log-Normal distribution, and we propose a tree-structured divide-andconquer framework to explain this phenomenon.
• We propose the Powered Length Penalty (PLP), a reinforcement learning objective motivated by the variance structure of the Log-Normal distribution. PLP scales penalty strength inversely with intrinsic task complexity, enabling models to internalize problem difficulty without relying on static heuristics or additional inference-time overhead.
• Extensive experiments against multiple baselines demonstrate that PLP achieves consistent improvements in reasoning efficiency on both mathematical and non-mathematical tasks, while maintaining or even enhancing accuracy.

2 RelatedWork

Large Language Models for Reasoning. In recent years, large language models have demonstrated remarkable potential in tackling complex reasoning tasks. Pioneering approaches like CoT and Tree-of-Thought [Yao et al., 2023]

have enabled these models to perform multi-step logical inference through step-by-step reasoning processes. Meanwhile, models such as ChatGPT [Achiam et al., 2023], DeepSeek-R1 QwQ-preview , and Kimi have further enhanced their reasoning abilities via large-scale reinforcement learning, equipping them with advanced capabilities like branching and validation, which substantially boost their overall reasoning performance.

Efficient reasoning. Although reasoning models and the CoT approach enhance model reasoning capabilities, they also entail increased computational costs. To address this, various methods have been proposed to improve reasoning efficiency. Token-Budget [Han et al., 2024] estimates the minimum number of tokens needed to answer a question and incorporates a prompt to guide the model to keep its reasoning within this token limit, thereby reducing computational overhead. However, this method incurs additional token usage when estimating the problem's difficulty. TokenSkip [Xia et al., 2025b] evaluates the importance of each token in the response, marks them accordingly, and fine-tunes the model on a dataset where a portion of less important tokens are removed. This often leads to answers that omit conjunctions, resulting in less coherent responses. Another approach, training language models for efficient reasoning [Arora and Zanette, 2025], modifies the reward function in RLOO by adding a length penalty based on the number of actions α, followed by reinforcement learning on a mixed dataset. However, it does not differentiate the length penalty between difficult and easy samples. Although the aforementioned methods effectively reduce the number of tokens used by the model, for complex problems the compressed reasoning chains are often insufficient to support correct answers, leading to a decline in accuracy. Motivated by this limitation, we propose the PLP, which aims to reduce the model's output only on simple problems while preserving its capacity for deep reasoning on complex tasks.

3 Methodology

3.1 Statistical Perspective on Reasoning Length

To motivate the design of our distribution-aware penalty, we first analyze the statistical properties of reasoning chain generation.

Empirical Observation of Length Variability. We first examine the distribution of reasoning lengths generated by LLMs when solving complex problems. By performing Monte Carlo sampling to generate 500 distinct responses for the same prompt using DeepSeek-R1-Distill-Qwen-7B on a single representative problem from the GSM8K dataset, we observe significant variability in the physical length of reasoning chains. Rather than clustering tightly around a mean, the lengths exhibit a pronounced heavy-tailed behavior.

As illustrated in Figure 2, this empirical distribution is robustly approximated by a Log-Normal distribution. The histogram (Figure 2a) displays the characteristic positive skew, while the Q-Q plot (Figure 2b) confirms the goodness-of-fit against the theoretical quantiles. The Kolmogorov-Smirnov (K-S) test further supports this fit with a statistic of 0.0597 and a p-value of 0.0544. We note a minor

(a) Histogram of Reasoning Lengths

deviation in the extreme upper tail (Figure 2b), which is an artifact of the maximum generation length cap (set at 32,694 tokens), causing a truncation effect. However, this does not undermine the goodness-of-fit in the bulk region.

Theoretical Mechanism: Recursive Task Decomposition. To provide a principled explanation for this observed LogNormal regularity, we model the reasoning process not merely as linear sequence generation, but as the traversal of a hierarchical solution tree. We view CoT reasoning as a recursive divide-and-conquer process. Complex problems are decomposed into intermediate steps, which further branch into finer-grained verification or correction substeps. Statistically, this can be modeled as a multiplicative branching process. Let Xk denote the stochastic branching factor (or expansion rate) at depth k. The number of active reasoning nodes at this depth expands as Nk ≈ Nk-1 · Xk. Although the total length L is physically the sum of tokens across all depths (L = ∑ Nk), for any non-trivial branching process, the total magnitude is dominated by the multiplicative expansion of the tree's width. Thus, the scale of the reasoning chain can be approximated by the cumulative product of these expansion factors:

Here, Xk represents an independent random variable capturing the local reasoning complexity at depth k. As shown in [Limpert et al., 2001], the cumulative effect of a multiplicative process converges to a Log-Normal distribution. Consequently, the length L itself follows a Log-Normal distribution:

This theoretical result explains why the multiplicative accumulation of uncertainty in a tree-structured reasoning process naturally gives rise to the heavy-tailed distributions observed in our experiments.

3.2 Difficulty-Aware Length Penalty

Assuming a log-normal distribution for generation lengths, the variance is determined by both the log-mean µ and the log-standard deviation σ:

In standard reinforcement learning settings, the sampling temperature is typically held constant to ensure training stability, implying that σ remains effectively invariant. Under this condition, Eq. (3) reveals that the variance scales quadratically with the expectation, i.e., Var(L) ∝ (E[L])2 .

This scaling law reveals a critical insight: uncertainty accumulates polynomially with reasoning depth. Consequently, standard linear penalty terms fail to capture the heavy-tailed nature of the variance distribution. This theoretical property dictates that the penalty term must follow a Power-Law Formulation, L-γ , to structurally align with the uncertainty growth.

While a strict inverse-variance normalization would theoretically imply fixing γ ≈ 2, directly applying L-2 creates a significant optimization bottleneck. In reasoning tasks where L often spans 102 to 104 , a quadratic decay suppresses the penalty magnitude to a negligible range (e.g., 10-6 ), resulting in vanishing gradients that provide little effective signal.

To resolve the limitation, we propose a Relaxed PowerLaw Formulation. We retain the polynomial decay profile mandated by the distribution theory but treat the curvature γ as a tunable hyperparameter. The relaxation offers a dual advantage, as visualized in Figure 3.

• Structurally (Shape Matching): Unlike linear penalties which impose uniform marginal costs, or exponential penalties that decay too aggressively (leading to premature saturation), the power-law curve aligns with the heavy-tailed distribution. It provides high resolution for short trajectories while naturally compressing distinctions in the long-length regime.
• Functionally (Signal-to-Noise Trade-off): By tuning γ, we can balance the signal magnitude against variance reduction. This mechanism mitigates the high uncertainty inherent in long reasoning chains without excessively diminishing the reward scale, thereby maintaining a robust learning signal during policy optimization.

Accordingly, we define the reward function r(·) as:

where α modulates the incentive magnitude, and γ calibrates the sensitivity decay. Maximizing this reward effectively imposes a soft constraint on verbosity, guiding the model toward the principle of parsimony via diminishing marginal returns.

3.3 Choice of RL Algorithm

Selecting an appropriate reinforcement learning algorithm is critical for effectively minimizing the proposed PLP. We analyze three representative approaches widely used in reasoning-oriented fine-tuning: PPO [Schulman et al., 2017], GRPO [Shao et al., 2024], and RLOO [Ahmadian et al., 2024], and justify our choice of RLOO.

PPO relies on a learned critic and incurs substantial computational overhead. In contrast, both GRPO and RLOO are critic-free and employ explicit reward functions, making them more suitable for efficiency-aware optimization.

In GRPO, the advantage is computed via group-wise normalization:

This normalization rescales rewards according to withingroup statistics, which can compress variance on simple problems while amplifying differences on complex ones. When combined with length-sensitive reward shaping, such adaptive rescaling may introduce additional optimization variance. To mitigate these effects, we adopt the RLOO algorithm:

Here, ARLOO denotes the leave-one-out advantage. This subtraction-based baseline centers rewards without rescaling their magnitude, thereby preserving the relative variance structure induced by PLP and making RLOO well-suited for length-aware optimization across varying difficulty levels. An overview of our pipeline is presented in Figure 4

3.4 Experimental Setup

In this section, we present experimental results to evaluate the effectiveness of our method.

Models. We conduct experiments using the DeepSeek-R1 framework [Guo et al., 2025], evaluating three models from this family: DeepSeek-R1-Distill-Qwen-1.5B, DeepSeekR1-Distill-Qwen-7B, and DeepSeek-R1-Distill-LLaMA-8B. The first two models are based on the Qwen architecture [Hui et al., 2024], while the third is based on LLaMA[Dubey et al., 2024]. These models vary in both scale and pretraining architecture, enabling us to assess the generality of our method.

Datasets. The primary training dataset is GSM8K [Cobbe et al., 2021]. We randomly select 3,200 examples from the GSM8K training set, ensuring a balanced distribution of difficulty levels. To evaluate generalization and robustness across diverse domains and difficulty levels, we test our models on six datasets: GSM8K, MATH500 [Lightman et al., 2023], AIME2024, GPQA [Rein et al., 2024], MMLU [Hendrycks et al., 2020], and HumanEval [Chen et al., 2021].

Baselines. In our experiments, we evaluate six baseline methods grouped into three categories: prompt-based approaches such as Prompt Engineering; supervised finetuning techniques including SFT and TokenSkip [Xia et al., 2025b]; and reinforcement learning-based methods like Efficient [Arora and Zanette, 2025], ShorterBetter [Yi et al., 2025], and ThinkPrune [Hou et al., 2025]. To ensure a fair comparison, all methods are trained on the same dataset of 3,200 GSM8K samples, using identical prompt formats and evaluation metrics. More details about the baseline settings can be found in the Appendix G.

Implementation details. Training is performed using the OpenRLHF framework [Hu et al., 2024], with efficient evaluation carried out via vLLM [Kwon et al., 2023]. We employ a learning rate of 5 × 10-6 for the 1.5B model, and a lower learning rate of 2 × 10-6 for both the 7B and 8B models. In this study, we perform experiments by fixing γ = 0.5 and adjusting the value of α. The maximum number of tokens is set to 32,768 for all experiments. To account for variability and ensure robust results, we perform three independent training runs for each configuration and report the average performance. More details can be found in the Appendix E.

Evaluation Setup. During the evaluation process, experiments are conducted using vLLM. The temperature is set to 0.6 and the seed is set to 42, while all other parameters followed the default settings of vLLM. The prompts employed for the different datasets are provided in the Appendix D.

Evaluation Metrics. To comprehensively assess model performance, we employ three evaluation metrics: Accuracy, Tokens, and AE (Accuracy-Efficiency). Accuracy measures the proportion of test cases in which the model produces the correct final answer, serving as the primary indicator of reasoning reliability. Tokens refers to the total number of generated tokens per response, including intermediate reasoning steps and the final answer, and is used to evaluate the efficiency of the model's reasoning process. Together, these metrics provide a balanced assessment of both correctness and computational efficiency. The Accuracy-Efficiency metric, directly adopted from the O1-Prune methodology [Luo et al., 2025], quantifies the trade-off between accuracy and conciseness. It offers a unified measure that captures both the effectiveness and brevity of model outputs, making it particularly well suited for evaluating reasoning models under efficiency constraints. The detailed information regarding the calculation formula for AE can be found in the Appendix H.

3.5 Results

As shown in Table 1, our method consistently outperforms baselines on in-domain datasets, achieving the highest AE scores across all cases. These results highlight the effectiveness of our approach in facilitating efficient reasoning. Furthermore, as demonstrated in Table 2, our method generalizes well to out-of-domain datasets, consistently obtaining the highest AE scores across various models and datasets. This underscores its robust generalization capability.

Furthermore, we observe that when the base model exhibits limited capability on an evaluation dataset, reflected by an accuracy below 50%, our method consistently outperforms the base model in accuracy while simultaneously reducing the number of output tokens. Conversely, for datasets where the base model performs strongly, our method prioritizes token length reduction, sometimes at the cost of a slight accuracy decline. Importantly, despite not being explicitly trained on these datasets, our method effectively adheres to the principle of "fast on easy, deep on hard", dynamically adjusting its behavior based on task difficulty to strike a balance between efficiency and accuracy.

3.6 Scalability Analysis

To evaluate the scalability of our approach across both model architecture and model size, we further conduct experiments on the large-scale Qwen3-32B model [Yang et al., 2025]. As shown in Table 3, our method consistently achieves the highest accuracy across all benchmarks while substantially reducing inference token usage compared to the baseline. Notably, although WHISPER [Xia et al., 2025a] aggressively compresses reasoning tokens, it suffers from a significant performance drop on the challenging AIME2024 benchmark. (this result is directly quoted from the WHISPER paper) In contrast, our method preserves deep reasoning on hard problems while effectively eliminating redundancy on simpler ones, resulting in the best overall efficiency-performance trade-off.

3.7 Empirical Analysis

We employed Gemini 2.5 Pro [Comanici et al., 2025] to assign difficulty scores, ranging from 2 to 9, to each problem in three mathematics datasets of varying difficulty levels. Based on these scores, we analyzed the models' performance across different difficulty tiers. As shown in Figure 5, the figure consolidates multiple aspects: Our fine-tuned model demonstrates adaptive efficiency: on simple tasks, it achieves drastic token reduction for rapid inference, with only a minor tradeoff in accuracy. Notably, as difficulty increases, the model recovers and even surpasses the baseline on intermediate tasks, while maintaining performance parity on the most challenging problems. This confirms the model's ability to accelerate simple queries without compromising its reasoning capabilities on complex ones. This observation demonstrates that our approach aligns with the "fast on easy, deep on hard" principle.

3.8 Impact on Reasoning Composition

To investigate how the PLP alters the internal structure of reasoning chains, we conducted a fine-grained component analysis comparing model outputs before and after training. We utilized Gemini-2.5-Pro as an impartial evaluator to classify reasoning steps into four distinct categories: Pivotal Reasoning, Productive Elaboration, Redundant Calculation, and Non-Substantive Statement.

As illustrated in Figure 6, the training process induces a substantial shift in the distribution of reasoning components:

• Densification of Valid Reasoning: The proportion of essential reasoning steps-comprising Pivotal Reasoning and Productive Elaboration-increased significantly from 65.4% to 83.9%. This indicates that the model learns to allocate its generation budget more efficiently towards effective problem-solving states.
• Suppression of Noise: Conversely, components associated with inefficiency were markedly suppressed. NonSubstantive Statements dropped by nearly half , and Redundant Calculations were virtually eliminated

Robustness Verification. To mitigate potential biases inherent to a single evaluator , we replicated this component analysis using other state-of-the-art models. We observed highly consistent distribution shifts across different evaluators, confirming that the densification of reasoning is a robust phenomenon rather than an artifact of a specific judge. Detailed comparative results from these auxiliary experiments are provided in Appendix M. These results suggest that PLP does not merely constrain length; rather, it implicitly optimizes the information density of the generated chain. By penalizing variance-induced redundancy while protecting deep thinking, our method encourages the model to prune irrelevant tokens while reinforcing the logical backbone required for complex reasoning.

3.9 Ablation Studies

Effect of the Length Penalty Parameters. We have conducted an ablation study on α and γ. For α, we test values from 1 to 30 and find optimal settings of 5, 3, and 5 for the 1.5B, 7B, and 8B models, respectively. Our results indicate that as α increases, the number of tokens generally decreases. We ablate γ over {0.05, 0.1, 0.25, 0.5, 1} and observe that γ = 0.5 yields the most stable performance across all models. Details can be found in Appendix O .

4 Discussion

While PLP outperforms baselines such as TokenSkip and ThinkPrune, it is instructive to examine its theoretical distinctions within adaptive reasoning. Constraint-based methods like LEASH [Li et al., 2025b] enforce predefined length targets, which can conflict with the long-tailed nature of complex reasoning and degrade performance on challenging tasks. Outcome-based methods such as ALP adjust penalties according to empirical success, but risk signal collapse as models saturate in accuracy. In contrast, PLP leverages the intrinsic statistics of the reasoning process, deriving penalties from log-normal length distributions. Its curvature-controlled power-law naturally decays with sequence length, penalizing redundancy on simple tasks while asymptotically preserving flexibility for complex reasoning, ensuring robust adaptation across difficulty levels.

5 Conclusion

In this work, we introduce a novel reinforcement learning framework that leverages a powered length penalty to adaptively control the reasoning depth of large language models based on input difficulty. Our approach promotes concise responses for simple problems while allowing more elaborate reasoning for complex ones, thus adhering to the principle of "fast on the easy, deep on the hard.". Remarkably, despite being trained solely on the simplest reasoning benchmark, GSM8K, our method demonstrates strong generalization to more challenging tasks and even out-of-domain datasets.

Contribution Statement

Zehui Ling, Deshu Chen, and Hongwei Zhang contributed equally to this work. Yuan Cheng is the corresponding author.