TPPMG: Temporal Planning-driven Progressive Motion Generation

1 Introduction

Human motion generation bridges high-level semantic understanding and low-level movement execution, with broad applications in virtual reality, game animation, and robotics [Yan et al., 2019; Zhao et al., 2020; Zhang et al., 2020; Raab et al., 2023]. In recent years, diffusion models [SohlDickstein et al., 2015; Song and Ermon, 2020] have become the dominant paradigm for text-to-motion (T2M) generation [Petrovich et al., 2022; Kim et al., 2023; Tevet et al., 2023], substantially improving realism and diversity [Kong et al., 2020; Ho et al., 2022a; Saharia et al., 2022; Ho et al., 2022b]. However, most existing methods adopt a fixed-length sampling paradigm: motion sequences are padded or cropped to a preset length, and the denoiser is trained and decoded on a fixed window.

Key issue: systematic temporal mismatch between text and generated motion. This paradigm induces a systematic temporal mismatch between text semantics and generated motion, consistently manifesting as two phenomena:

1. Duration trailing for short texts. Short instructions typically finish within 0.5-1 s, yet fixed-length sampling is forced to generate many more frames. Consequently, semantic content concentrates early, while the remaining frames become a low-energy tail with little new intent-often manifested as near-static jitter, tempo flattening, or small "action echoes"(Fig. 1, bottom).
2. Stage-wise semantic collapse for long texts. Under a fixed-length budget,multi-stage instructions (e.g.,"walk to the table, wave, then turn around and leave") often collapse into a motion stream with unclear boundaries, leading to missing stages, semantic entanglement, and blurred transitions(Fig. 1, top)-essentially a distortion in temporal budget allocation.

These two phenomena share the same root cause: duration is fixed as a training hyperparameter rather than inferred from semantics as a decision variable. Under fixed-window training/sampling, diffusion models optimize

where Tfix is set prior to training. The text condition c specifies what to generate, but the generative space has no variable for how long to generate or how to allocate stages. Consequently, the model cannot adapt motion length to semantics: when the required duration is less than Tfix, it exhibits "passive filling"; when longer, it exhibits "passive compression."

Our perspective: semantics → temporal structure → motion realization. We argue that temporal structure is semantic information that can be inferred from text and modeled explicitly. Ideally, generation should be factorized as

and, for long texts, further introduce a stage script S to instantiate p(T | c), yielding an explicit mapping from "semantics" to "(stages, durations)."

Motivated by this, we propose TPPMG (Temporal Planning-driven Progressive Motion Generation), which consists of two complementary components:

1. Temporal Planner. It uses an LLM[Raffel et al., 2020] with a duration calibrator to infer stages and durations from text, establishing an explicit mapping from semantics to temporal structure.
2. Progressive Diffusion Generator. Given a variablelength stage budget from the planner, the generator faces a key mismatch: the denoiser is trained on a fixed window length F, yet must produce variable-length sequences at inference. We therefore adopt a sliding-window procedure (denoise→emit→shift→append) that turns variable-length generation into budget-driven chunk emission, and introduce heterogeneous noise scheduling to improve cross-window stability and boundary quality. This design bridges fixed-length training and variable-length inference, resulting in motion sequences that match the planned temporal structure.

To evaluate our method, we construct three derived evaluation benchmarks: HumanML3D-Short, filtered by text length; HumanML3D-Concat and BABEL-Concat, multistage benchmarks built by concatenating single-stage samples with external ground-truth boundaries and segment lengths. We develop a comprehensive evaluation suite measuring effective-frame proportion, stage-structure recovery, boundary alignment, and segment-level semantic consistency.

Contributions

1. We systematically characterize duration trailing on short texts and stage-wise semantic collapse on long texts under the fixed-length paradigm, and identify the root cause as treating duration as a hyperparameter instead of a semantic variable.
2. We propose an LLM-based Temporal Planner with a duration calibrator to explicitly model the mapping semantics → (stages, durations).
3. We introduce heterogeneous noise scheduling and a sliding-window mechanism to resolve the structural mismatch between fixed-length training and variable-length sampling, enabling stage-aware progressive generation.
4. We construct three derived evaluation benchmarks and validate that TPPMG suppresses trailing (UR: 0.52 → 0.85), recovers temporal structure (PhaseRecall: 0.56 → 0.90, BoundaryF1: 0.28 → 0.56).

2 RelatedWork

2.1 Diffusion-based T2Mand Fixed-Length Bias

Diffusion models have become dominant for T2M [Zhang et al., 2024a], but jointly denoising the full sequence requires a fixed window length at training time, leading to semantic trailing for short prompts and phase blurring for long prompts [Tevet et al., 2023]. Existing works incorporate keyframe/trajectory constraints [Cohan et al., 2024; Karunratanakul et al., 2023] or masked completion [Chen et al., 2023a; Zhang et al., 2024b], but lack mechanisms to infer phased duration structures and drive generation budgets.

2.2 DurationModeling and Variable-Length Generation

Autoregressive methods support variable-length generation but cannot infer stage durations from text. Pose-level continuous AR [Guo et al., 2020] enables flexible stopping but suffers from error accumulation; action-level AR (TEACH [Athanasiou et al., 2022]) reduces accumulation via clip-wise generation, yet stage durations remain externally specified. Discrete token-based methods (T2M-GPT, MoMask [Zhang et al., 2023; Guo et al., 2024]) apply languagemodel-style generation but rely on heuristic token budgets without stage-level planning. VAE-based methods (TEMOS, ACTOR [Petrovich et al., 2022; Petrovich et al., 2021]) condition on duration as external input. Latent diffusion [Chen et al., 2023b; Dai et al., 2024], Flow Matching [Hu et al., 2023], and INR [Cervantes et al., 2022] offer efficient sampling or continuous-time representation, but none decompose total duration into text-inferred stage-level structures.

2.3 Summary

Existing methods lack unified mechanisms to infer phased duration structures from text, achieve stable variablelength inference, and ensure inter-segment continuity-gaps TPPMG addresses via two components.

3 Method

Problem Setup Given text c, our goal is to generate a motion sequence x with an explicit temporal structure. We model generation via a latent stage script S = {(texti, di)}Ni=1 , where texti is the i-th sub-instruction and di > 0 its duration (s); total length is induced by S via fps. Accordingly,

which decomposes the problem into (i) semantics → temporal structure (p(S | c)) and (ii) temporal structure →motion realization (p(x | S)).

Method Overview As illustrated in Fig. 2, TPPMG is a plan-then-generate pipeline: (i) Temporal Planner outputs {(texti, ˆ di)} with a lightweight calibrator for duration bias; (ii) Progressive Diffusion Generator bridges fixedwindow diffusion and variable-length budgets via slidingwindow sampling, with heterogeneous noise and overlapped hard-conditioning to stabilize boundaries.

3.1 Temporal Planner

Schema-Constrained Script Generation

Given instruction c, the planner outputs a stage script S = {(texti, di)}Ni=1 in a fixed JSON schema. We enforce schema validation (and log decoding hyperparameters) to ensure reproducible and parseable plans. The script is constrained by

These bounds prevent degenerate estimates and cap total length.

Duration Calibration

Raw LLM durations can be systematically biased across action types. We thus apply a lightweight calibrator hψ to obtain the final budget

where ϕ(·) is a frozen text encoder, and softplus with projection enforces positivity and feasibility. We train hψ with stage-level supervision:

where d⋆i comes from single-stage real samples or Concat benchmarks with external ground-truth segment lengths, avoiding circular targets.

3.2 Progressive Diffusion Generator

Goal and Challenges

Given a stage script produced by the planner, S = {(texti, Ti)}Ni=1 , our goal is budget-driven variable-length generation using a fixed-window denoiser. Naively generating each stage independently and stitching can match durations, but introduces tempo distortion, boundary discontinuities, and error accumulation.

To adapt the planner's variable-length temporal structure without changing the fixed-length training paradigm, we introduce three core designs:

1. Budget-Driven Sliding-Window Sampling, which converts variable-length generation into a controllable chunk emission process;
2. Heterogeneous Noise Scheduling within a fixed window, which assigns lower noise to previously generated frames to suppress repeated perturbations, thereby improving cross-window stability and boundary quality;
3. Cross-Stage Continuity Interfaces, which combine overlapped hard-conditioning and end-of-stage cropping to ensure boundary continuity and exact length alignment.

Setup: Fixed-Window EDM Training

We train a text-conditioned denoiser on a fixed window length F, using EDM-style continuous noise parameterization and the standard v-parameterization (v-pred). To minimize traintest mismatch, we assign a per-frame noise scale {σn}Fn=1 and apply the forward perturbation

The network vθ takes as input the noisy motion x, a noise embedding γ(σn) (e.g., a log-σ MLP), and the text condition cond(c), and minimizes a frame-wise MSE under vprediction:

where v follows the standard EDM v-parameterization (details in Appendix to avoid notation ambiguity). If a training sequence is shorter than F, we use a length mask and compute the loss only on valid frames.

Budget-Driven Sliding-Window Sampling

The planner outputs a duration script in seconds. To interface it with a fixed-window denoiser, we first convert the calibrated duration of each stage ˆ di into a target number of frames:

We then discretize the stage length into the number of chunks to emit:

where C is the chunk length (in frames) and the denoiser window length is fixed to F = KC. This yields a key equivalence: variable-length inference reduces to budget control over the emission count Mi. The denoiser always operates on an input of fixed length F; by emitting exactly Mi chunks we obtain a sequence that matches the planned stage length, without any post-hoc truncation/interpolation that retimes the motion.

To make chunk-by-chunk emission stable, we structure sampling into two phases: a warm-up phase that establishes a stable conditional prefix, followed by a main loop that emits one chunk per iteration while sliding the window, until Mi chunks are produced.

1. Warm-up. We initialize W ∼ N(0, I). If the previous stage provides a tail chunk Tail, we copy it into the first chunk of the window as an optional hard condition and keep it frozen throughout the reverse process; the remaining positions are filled with noise placeholders. We then run a fixed number of reverse updates Lwarm, allowing the prefix to converge into a stable context while preserving the suffix as noise placeholders for subsequent expansion.
2. Main loop. After warm-up, we repeatedly perform denoise-emit-shift-append. In each iteration, we run Lstep reverse updates within the fixed-length window W (denoise), output the first chunk (emit), shift the window left by one chunk (shift), and append an all-noise chunk at the end as a placeholder for future generation (append). We repeat this loop until Mi chunks have been emitted.

Heterogeneous Noise Scheduling

Under sliding-window sampling, after each shift the alreadygenerated prefix remains inside the window and is updated again in subsequent reverse steps. If this prefix is repeatedly perturbed at a comparable noise level, drift accumulates and stage boundaries blur. To mitigate this, we introduce heterogeneous noise scheduling within a fixed window.

We partition a window into K chunks of length C, and assign a monotonically increasing noise scale to chunk k:

Here σn is the per-frame noise scale and {σk} is derived from a standard EDM/Karras schedule (approximately uniform in log-space) and broadcast within each chunk. Intuitively, the prefix chunk is placed in a low-noise region (σ0), so it is only mildly updated across iterations, while the suffix stays at higher noise for new content. We use the same organization during training to reduce train-test shift.

To improve robustness to noise-scale allocations, we apply a log-space offset augmentation:

where ˜σk is the augmented noise scale, [σmin, σmax] is the EDM noise range, and ∆ controls the offset magnitude. This log-space perturbation is equivalent to multiplicative noisescale jitter and improves generalization to different schedules.

Cross-Stage Continuity

Budgeted sliding-window sampling resolves within-stage variable-length generation, but between-stage discontinuities may still occur at boundaries. We thus introduce an overlapped hard-conditioning interface.

Let the last chunk of stage i be

When generating stage i+1, we place Tail(i) into the first chunk of the new window and hard-freeze it throughout the entire reverse process:

The remaining K-1 chunks are initialized as noise and denoised under the heterogeneous schedule. This provides an immutable overlapped prefix for the new stage, substantially reducing boundary jumps and improving continuity.

Finally, if Ti is not divisible by C, we apply end-ofstage cropping only to achieve exact length alignment: let r = Ti - (Mi - 1)C, and keep only the first r frames of the last emitted chunk (discard the rest). This cropping is performed after stage generation completes and does not introduce interpolation or re-timing, avoiding tempo distortion.

4 Experiments

4.1 Core Questions and Evaluation Setup

TPPMG targets the semantic-temporal mismatch induced by fixed-length sampling. Accordingly, our experiments are organized around two core questions, each paired with a dedicated benchmark and evaluation suite.

• Q1 (Short texts): Duration trailing. Under fixed-length generation, short instructions finish their semantic content early and exhibit passive filling in remaining frames. To evaluate this, we construct HumanML3D-Short:
• HumanML3D-Short: a short-text subset filtered from HumanML3D test split by word count (≤ 6 or ≤ 10 words). Note that word count is used solely as a filtering proxy for dataset construction. A pilot study confirmed 98% of ≤ 6-word samples correspond to short-duration motions under HumanML3D's distribution.

On HumanML3D-Short, we measure the Utilization Ratio (UR) (↑), defined as the proportion of effective frames from velocity/acceleration energy; higher UR indicates fewer trailing frames and more compact motion.

• Q2 (Long texts): Structure, semantics, and continuity. The challenge is not merely generating more frames, but allocating time by semantics and producing stable boundaries. Since natural long-text datasets lack verifiable ground-truth boundaries, using planner output as supervision creates circular evaluation. We thus build two oracle Concat benchmarks with external ground-truth boundaries:
• HumanML3D-Concat (oracle): we sample m ∈ {2, 3, 4} single-stage samples from the HumanML3D[Guo et al., 2022] test split and concatenate both motions and texts. Concatenation points provide ground-truth boundaries B⋆ and ground-truth segment lengths {T ⋆ j }.
• BABEL-Concat (oracle): we concatenate m ∈ {2, 3, 4, 5} atomic segments from BABEL and describe the sequence via templated text (e.g., 'do ... then

...') to reduce linguistic noise. Motions are converted to HumanML3D's representation with fps = 20.

On the oracle Concat benchmarks, we answer Q2 using three complementary metrics: (i) structure recovery via PhaseRecallcap (↑), measuring whether the predicted number of stages reaches a sufficient level; Boundary-F1 (↑), measuring how well predicted boundaries align with B⋆ ; (ii) segment-level semantic consistency via PSC@1 (↑), measuring whether the generated i-th segment best matches the ground-truth i-th segment; and (iii) boundary continuity via RootJump/JerkSpike (↓) computed at the oracle boundaries B⋆ , measuring abrupt root-position jumps and jerk spikes at stage transitions.

4.2 Baselines

TPPMG is designed to improve temporal control without changing the diffusion backbone. We therefore compare against representative diffusion-based T2Mmethods under a unified setting. We also report a small number of variablelength (token-based / autoregressive) approaches as reference lines to contextualize performance under natively variablelength paradigms.

4.3 Protocol and Implementation Details

We report mean±std over the evaluation set.

All methods use the HumanML3D motion representation with fps = 20. Following common practice, diffusion baselines are sampled at the community-standard fixed length Tfix = 196. TPPMG inference uses a fixed window F = 64 and chunk size C = 8 (K = 8), with one-chunk overlap freezing for cross-stage continuity.

On the Concat benchmarks, oracle boundaries B⋆ and segment lengths {T ⋆ j } are available. To avoid ambiguity caused by different generated total lengths, we first resample each generated sequence to the oracle total length T ⋆ = ∑j T ⋆ j , and then segment it by B⋆ before computing structure, semantic-consistency, and boundary-continuity metrics.

4.4 Main Results

Q1: Trailing Suppression on Short Texts

We first evaluate whether TPPMG suppresses duration trailing on short instructions. We test on HumanML3D-Short and report the Utilization Ratio (UR) (↑).

HumanML3D-Short As shown in Tab. 1, fixedlength diffusion baselines (MDM[Tevet et al., 2023], MotionDiffuse[Zhang et al., 2024a], SALAD[Hong et al., 2025]) exhibit substantial trailing on short texts. In the ≤ 6-word bucket, their UR remains low (0.52-0.55), indicating that only a small portion of frames are effective motion while the rest form low-energy tails. Token/AR baselines (T2M-GPT, MotionGPT[Jiang et al., 2024]) alleviate trailing (≈ 0.68-0.69), yet TPPMGachieves the highest UR (0.85 ± 0.12). The ≤ 10-word bucket shows the same pattern: MotionGPT reaches 0.77 ± 0.15, while TPPMGattains 0.90±0.11, indicating consistently fewer trailing frames under budget-driven generation.

Qualitative evidence. Fig. 1 and Fig. 3 visualize the same effect: for short prompts, SALAD finishes the core action early but exhibits trailing artifacts (near-static jitter, tempo flattening, or action echo), whereas TPPMGterminates promptly, indicating effective trailing suppression.

Q2: Structure, Semantics, and Continuity on Oracle Concat

We evaluate multi-stage long-text generation on two oracle Concat benchmarks (Tab. 2). Across both datasets, TPPMGconsistently achieves the best structure-semanticscontinuity trade-off: higher PhaseRecallcap, Boundary-F1

(robust under Detector-A/B), and PSC@1, with lower RootJump/JerkSpike.

HumanML3D-Concat. Diffusion baselines (MDM, MotionDiffuse, SALAD) remain limited by fixed-length sampling, while token/AR models are stronger but still lag behind TPPMG. Against the strongest baseline MotionGPT, TPPMGimproves PhaseRecallcap from 0.78±0.06 to 0.90± 0.05, BoundaryF1A/B from 0.44 ± 0.06/0.41 ± 0.06 to 0.56 ± 0.06/0.53 ± 0.06, and PSC@1 from 0.54 ± 0.08 to 0.72 ± 0.07, while reducing RootJump/JerkSpike from 0.10 ± 0.04/0.46 ± 0.09 to 0.06 ± 0.03/0.35 ± 0.08.

BABEL-SimpleConcat. The same advantage holds under cleaner concatenation: TPPMGoutperforms MotionGPT in PhaseRecallcap/PSC@1 (0.94 ± 0.04/0.80 ± 0.06 vs. 0.86 ± 0.06/0.62 ± 0.08), raises BoundaryF1A/B (0.62 ± 0.06/0.58 ± 0.06 vs. 0.49 ± 0.06/0.46 ± 0.06), and lowers RootJump/JerkSpike (0.05±0.03/0.32±0.07 vs. 0.09± 0.04/0.44 ± 0.09).

Qualitative evidence. Fig. 1 and Fig. 4 reflects the same trend: under a fixed-length budget, baselines often collapse multi-stage texts into a stream with blurred oracle boundaries, causing stage omission/repetition and semantic entanglement, with visible discontinuities near splice points; in contrast, TPPMGyields clearer stage progression and smoother boundary transitions, consistent with the gains in boundary alignment, continuity, and segment correspondence.

Generation Quality. Beyond temporal metrics, TPPMG also achieves competitive generation quality on our benchmarks, attaining the lowest FID and highest R-Precision@1 among all compared methods, confirming that temporal planning improves rather than trades off against overall motion quality.

4.5 Ablation Studies

We ablate each component under identical denoiser weights, sampling budgets, and evaluation protocol (Sec. 4.3). Tab. 3 reports UR on HumanML3D-Short and Concat metrics.

w/o Heterogeneous Noise Scheduling. Replacing per-chunk monotonic noise scales with uniform ones (warm-up and overlap freezing unchanged) causes the largest drops in structure/semantics: on HumanML3DConcat, PhaseRecallcap 0.90→0.79, PSC@1 0.72→0.58, BoundaryF1A/B 0.56/0.53→0.46/0.43, with minor continuity changes (RootJump/JerkSpike 0.06/0.35→0.07/0.39); BABEL-SimpleConcat follows the same trend. This confirms heterogeneous scheduling as key to stage separability and segment correspondence.

w/o Overlap Freezing. Removing overlap freezing selectively hurts continuity: RootJump/JerkSpike rises 0.06/0.35→0.11/0.49 on HumanML3D-Concat, while structure/semantics change marginally (≤ 0.02); BABELSimpleConcat shows the same pattern. This supports overlap freezing as the primary mechanism for smooth cross-stage transitions.

w/o Duration Calibration. Removing calibration degrades alignment on both Concat datasets: HumanML3D-Concat PSC@1 0.72→0.67, BoundaryF1A/B 0.56/0.53→0.52/0.49; BABEL-SimpleConcat PSC@1 0.80→0.74, BoundaryF1A/B 0.62/0.58→0.58/0.55; continuity is near-unchanged. UR decreases on HumanML3DShort, indicating weaker duration matching.

4.6 Inference Overhead

Despite stage-wise sampling, TPPMG achieves comparable wall-clock time and lower peak GPU memory than fixedlength baselines. The smaller window (F=64 vs. F=196) reduces self-attention cost quadratically, partially offsetting multi-stage overhead; heterogeneous scheduling limits perslide denoising to a few steps. Stage-wise generation retains only conditioning frames between stages, reducing peak memory.

5 Conclusion

This paper addresses temporal misalignment in text-driven human motion generation by proposing TPPMG. We reveal that treating duration as a hyperparameter rather than a semantic variable causes duration trailing on short texts and stage-wise semantic collapse on long texts. TPPMG follows "semantics → temporal structure → motion realization": the Temporal Planner models semantics-tostructure mapping, and the Progressive Diffusion Generator resolves the fixed-length training / variable-length inference mismatch. Experiments on three derived evaluation benchmarks (HumanML3D-Short, HumanML3DConcat, BABEL-Concat) show that TPPMG improves effective duration utilization, stage recovery, boundary alignment, and semantic consistency.

Limitations and Future Work. On HumanML3D-Concat, the Temporal Planner achieves 88.2% stage segmentation accuracy and 0.48s mean duration error, confirming overall reliability. However, two failure modes remain: on ambiguous prompts (e.g., "do some warm-up exercises"), underspecified semantics lead to unstable stage decomposition; on prompts with many unevenly paced stages, brief actions tend to be over-extended while longer stages are compressed, causing systematic boundary shifts. End-to-end optimization of the planner with boundary alignment as a reward signal is a promising direction for addressing these limitations.

Acknowledgements

The authors would like to thank Prof. Qianchuan Zhao for his valuable guidance and insightful contributions throughout this work.

This work was supported in part by the National Natural Science Foundation of China under Grant 62192751, in part by the Key R&D Project of China under Grant 2017YFC0704100, in part by the 111 International Collaboration Program of China under Grant B25027, in part by BNRist Program under Grant BNR2019TD01009, in part by the National Innovation Center of High Speed Train R&D Project under Grant CX/KJ-2020-0006, in part by the InnoHK Initiative, The Government of HKSAR, and in part by the Laboratory for AI-Powered Financial Technologies.

Contribution Statement

Ruoyu Wang and Can Deng contributed equally to this work. Ruoyu Wang is the corresponding author.