DToMA: Training-free Dynamic Token MAnipulation for Long Video Understanding

1 Introduction

Long video understanding requires models to process and reason over video content spanning several minutes to hours. Large language models (LLMs) [Yang et al., 2024] have shown exceptional comprehension of textual information. Multimodal LLMs (MLLMs) [Li et al., 2024; Wang et al.,

∗Corresponding author

2024] further equip LLMs with multimodal perception ability, greatly extending their comprehension of images and short videos. Those advances motivate researchers to investigate methods for effectively understanding long videos.

Existing VideoLLMs typically process videos by converting each frame into hundreds of visual tokens, leading to a rapid token increase as video length grows. For instance, LLaVA-Video [Zhang et al., 2024d] generates over 11k tokens for merely 64 frames. However, these visual tokens often have heavy redundancy [Chen et al., 2025], exacerbating the computational challenges. To address this, prior works focus on token reduction and alternative video representation strategies. The former includes pruning visual content outside LLMs [Shang et al., 2024; Shen et al., 2024] and compression mechanisms within LLMs [Chen et al., 2025; Ye et al., 2024]. The latter methods, such as image grids [Kim et al., 2024], slow-fast processing [Xu et al., 2024], or querybased methods [Li et al., 2025], design alternative ways to effectively represent videos. While improving efficiency, they are often at the expense of visual details or temporal dependencies, limiting their comprehension ability. Effective visual token processing remains to be explored for simultaneously improving efficiency and video comprehension.

To enhance both efficiency and comprehension abilities, we explore the internal reasoning mechanisms of VideoLLMs, inspired by human reasoning processes. Human reasoning consists of two key aspects: a general reasoning process and intuitive adjustment mechanisms [Dundas and Chik, 2011]. Humans generally approach video understanding tasks by first perceiving video content, then analyzing based on questions, and finally inferring based on prior knowledge. For quicker and better comprehension, they employ strategies such as conducting an initial coarse scan to adjust focus, rewatching video to clarify uncertainties or refine their understanding, and ceasing examining visual content once confident in their conclusions. We delve into the internal reasoning mechanisms of VideoLLMs to analyze whether they exhibit reasoning processes similar to humans. Inspired by recent studies [Fang et al., 2024] and [Chen et al., 2025], we analyze reasoning patterns in multi-choice VideoQA tasks through entropy dynamics and cross-attention. Specifically, we identify three distinct reasoning stages in VideoLLMs: 1) ShallowLayers: Text strongly attends to visual tokens, facilitating rapid multimodal information exchange. Entropy remains high, suggesting an information-gathering phase [Chen et al., 2025], akin to human initial perception of video content. 2) Intermediate Layers: Attention declines and narrows its focus, and entropy fluctuates as the model explores and analyzes visual content, similar to humans refining and analyzing visual content given questions. 3) Deep Layers: The model greatly reduces attention on visual tokens and entropy steadily declines, indicating a shift to knowledgedriven reasoning, resembling how humans draw conclusions after gathering sufficient information. Our analysis reveals an important insight: while VideoLLMs align with the general reasoning process observed in humans, they overlook the intuitive adjustment mechanisms crucial for human cognition. Unlike humans, VideoLLMs do not prioritize questionrelevant content but instead attempt to memorize all video details. They cannot re-examine the visual content when uncertain and cease processing visual content once sufficiently confident in their conclusions. This limits leads to suboptimal efficiency and comprehension abilities in VideoLLMs.

Based on these observations, we propose DToMA, a training-free Dynamic Token MAipulation method to improve efficiency and comprehension abilities of VideoLLMs in three aspects: text-guided keyframe-aware reorganization, uncertainty-based visual injection, and early-exit pruning. Firstly, text-guided keyframe-aware reorganization imitates humans to selectively focus after an initial scan. It uses crossattention scores from shallow layers to identify keyframes, allocating more tokens to keyframes while reducing nonkeyframe tokens. Secondly, uncertainty-based visual injection emulates humans to revisit related content when uncertainty arises. For samples with high entropy, this process dynamically reintroduces visual tokens in the model's feedforward layers, enhancing alignment with visual content and reducing uncertainty. Thirdly, early-exit pruning mirrors humans to stop examining visual content once they are confident about their conclusions. This process prunes visual tokens before the model reaches its deep layers, reducing redundant computations without sacrificing reasoning accuracy. As shown in Fig.1, by systematically emulating these human-inspired cognitive strategies, DToMA achieves gains in both efficiency and comprehension capability.

Our contributions are summarized as follows:

• We analyze VideoLLMs' reasoning processes through entropy and attention dynamics, revealing that while these models partially emulate human general reasoning processes, they lack key intuitive adjustments critical for efficiency and comprehension capability.
• We propose DToMA, a training-free dynamic visual token manipulation method that improves efficiency and reasoning capability by imitating human intuitive adjustment mechanisms, including text-guided keyframeaware reorganization, uncertainty-based visual injection, and early-exit pruning.
• Experiments on 6 long video understanding benchmarks demonstrate that DToMA outperforms state-of-the-art methods and generalizes across diverse LLM architectures and sizes.

2 RelatedWorks

2.1 Video Large Language Models

The demand for enhanced understanding capabilities in MLLMs has led to a substantial increase in visual tokens. Early works like LLaVA [Liu et al., 2024a] encode a 336px image into 576 tokens, while further efforts [Li et al., 2024; Liu et al., 2024b] splitting higher resolution images into multiple sub-images, leading to more extended tokens, e.g., 2,306 tokens for a 672px image. Video understanding tasks exacerbate this challenge due to sequential frames, e.g., up to 4k tokens for a 16-frame video, straining LLM context limits [Liu et al., 2024b]. To process longer videos, efforts like Gemini1.5 [Team et al., 2024] and LongVA [Zhang et al., 2024b] extend context windows but suffer from prohibitive computational costs, with Gemini-1.5 encoding one-hour videos into 920k tokens. Compression strategies, such as LLaMA-VID [Li et al., 2025] and Vid-Compress [Lan et al., 2024], reduce the token numbers but often sacrifice visual detail. Alternative approaches like image grids [Kim et al., 2024] and slow-fast processing [Xu et al., 2024] improve efficiency but struggle with long videos.

The key challenge remains in the excessive number of tokens to represent video content. To address this, we propose DToMA, a training-free dynamic token manipulation method that enhances efficiency and understanding of VideoLLMs.

2.2 Visual Token Reduction

Extensive research has focused on reducing visual tokens to enhance efficiency without obviously compromising visual understanding. Methods like LLaVA-PruMerge [Shang et al., 2024], FasterVLM [Zhang et al., 2024c], and FoPru [Jiang et al., 2024a] drop redundant tokens using attention guidance in visual encoder but lack textual adaptability. Methods like TRIM [Song et al., 2024] and [Chen et al., 2024b] employ paired text encoders, i.e., CLIP, to assist token filtering but are incompatible with fine-tuned image encoders. Recent works consider compression within LLMs, such as FastV [Chen et al., 2025] and Feather [Endo et al., 2024], retaining tokens with high visual-text attention scores. FitPrune [Ye et al., 2024] and G-Search [Zhao et al., 2024] further progressively prune tokens within LLM layers, thereby enhancing computational efficiency.

Existing methods do not adequately balance efficiency with performance, often compromising video understanding. To address this gap, DToMA 1) mitigates inefficiency by pruning non-keyframe tokens and exiting all visual tokens in LLM deep layers, and 2) improves comprehension by exerting emphasis on keyframes and reintroducing visual information when uncertainty arises. Thus, DToMA improves both efficiency and comprehension without additional training.

2.3 InterpretingMLLM Internal Mechanism

Understanding the inner workings of MLLMs can gain insights into their reasoning processes, identify inefficiencies, and develop targeted strategies for improvement. Recent interpretability studies using attention scores [Chen et al., 2025; Li et al., 2022], intermediate representations [Jiang et al., 2024b], and activation patterns [Chen et al., 2024a] mitigate token redundancy and image hallucination issues. For instance, analyzing attention scores provides insight into the model's focus at each layer. Recent studies leverage this method to reveal that LLMs often emphasize textual information over visual content [Liu et al., 2025; Fu et al., 2024b]. By better balancing the emphasis on visual tokens, these methods effectively mitigate image hallucinations.

Building on these insights, we analyze VideoLLMs through the lenses of attention and entropy dynamics to compare their reasoning processes to human cognition. We find that while VideoLLMs can imitate general human reasoning phases, they lack key intuitive strategies for efficient and accurate reasoning. DToMA bridges this gap by incorporating human-inspired strategies to enhance reasoning efficiency and comprehension.

3 Method

3.1 Preliminary

VideoLLMs generate text responses based on input video and text queries. The core components include a visual encoder E(·), a projector for modality alignment, an LLM with L layers, and an LM head Vocab(·) which predicts the vocabulary distribution of the next token. Given a video with N frames, the visual encoder extracts N frame embeddings. The projector maps frame embeddings as visual tokens Tv ∈ RN×P×d to enable processing by the LLM, where P, d are the number of patches and model dimension. Then, visual tokens Tv and text query tokens Tt ∈ RNt×d are concatenated as the input sequence, i.e., {xi}t1 , t = N × P + Nt, and are processed through L LLM layers. The LLM final layer output h(L) xt is used by Vocab(·) to predict the next token xt+1 as:

With greedy decoding, the most probable word for output would be selected.

3.2 Observations of LLM Internal Behaviors

In this section, we aim to explore the inefficiencies in the reasoning processes of VideoLLMs by analyzing their internal behaviors. Through this analysis, we identify key areas where the models struggle and propose potential improvements.

Specifically, to gain insights into these behaviors, we employ two critical metrics: entropy and cross-attention dynamics. These metrics are chosen because they provide a comprehensive view of the model's confidence levels and its prioritization of multimodal information, respectively. Entropy helps track uncertainty and confidence across layers [Fang et al., 2024], while cross-attention reveals how the model integrates information from different modalities [Chen et al., 2025]. Detailed observation implementations are as follows:

• Entropy dynamics are observed by tracking the evolution of answer probabilities across layers, inspired by recent interpretability research [nostalgebraist, 2020] which finds intermediate outputs h(l) xt in each l-th layer can be interpreted by Vocab(·). By prompting the model to answer directly with the letter option from given choices, we can decode the first token output at each layer l to track answer probabilities P(x = o|x1:t) following Eq. (1), denoted as Po, where o ∈ {A, B, C, D}. Then, the normalized entropy is computed as H = ∑ -PologPo/logm, where m = 4 is the number of options. H is the model's uncertainty at each layer.

• Cross-attention dynamics are analyzed through attention maps from each layer. Specifically, for attention maps A(l) xt ∈ Rn×t×t in each layer, where n is the number of heads, we extract the cross-attention scores between text query Tt and visual tokens Tv, i.e., C (l) xt ∈ Rn×Nt×NP , from A(l) xt . Aggregating C (l) xt across all attention heads yields: C (l) xt = ∑n C (l) xt . Measuring C (l) xt allows us to analyze how text token aggregates visual information.

Our analysis reveals distinct stages in VideoLLMs' reasoning process, as in Fig.2, 3. In shallow layers, entropy is high and Po is nearly random, yet text tokens strongly attend to visual tokens, indicating an initial information-gathering phase [Chen et al., 2025]. In intermediate layers, entropy exhibits dynamic fluctuations. For simple questions, it decreases steadily as the model converges on the correct answer. However, for complex ones, prolonged fluctuations suggest ongoing uncertainty. Attention declines and narrows its focus, reflecting information refinement. In deep layers, entropy rapidly converges for simple questions but remains higher and fluctuates for complex ones. Attention to visual tokens diminishes, indicating internal textual reasoning.

The above observations highlight several inefficiencies: in shallow layers, models attempt to perceive all visual content. Methods like FastV [Chen et al., 2025] reduce the influence of redundancy but do not adequately emphasize key information, leading to unnecessary computational overhead and suboptimal performance. Though uncertainty occurs in intermediate layers, the diminishing cross-attention from text tokens makes it challenging for the model to independently and adequately reassess necessary visual evidence. In deep layers, reduced attention to visual tokens suggests inefficiency in continuously computing visual context for final reasoning.

3.3 DToMA

To this end, we propose DToMA, a training-free Dynamic visual Token MAnipulation method to address those inefficiencies and improve reasoning capabilities, as in Fig. 4. DToMA has 3 token processing strategies: text-guided keyframeaware reorganization, uncertainty-based visual injection, and early-exit pruning, detailed in the following sections.

Text-Guided Keyframe-Aware Reorganization (TKR). To solve the inefficiency that the model attempts to memorize all video details in shallow layers, we propose TKR that prioritizes keyframes while reducing non-keyframe tokens for both efficiency and understanding ability improvement. The method involves a two-pass process. The first pass is executed only in LLM shallow layers to obtain question-related guidance for keyframe selection and non-keyframe tokens reduction. Then, the second pass is executed formally through all layers with our reorganized visual tokens. Details of TKR are shown in Algorithm 1. Specifically, the LLM first performs an r1-layer forward pass using coarse representations ˆ Tv to get cross-attention C (r) ˆ xt , which is then averaged to obtain frame importance F (r) for keyframe selection. Note that F (r) would be divided into S non-overlapping segments, i.e., F (r) 1 , ..., F (r) S , and the topk frames are selected from each segment to avoid concentrating keyframes in a local video clip. For non-keyframes, we drop the lowest attended tokens in ˆ Tv, ensuring the pruning ratio respects the token budget. Finally, keyframes retain their fine representations and non-keyframes are with reduced coarse representations, both are reorganized in chronological order for full propagation through the LLM as: LLM1:L(Concat(Tfinal,Tt)). This method balances computational efficiency with representational fidelity, highly adaptable to various computational budgets and tasks.

Uncertainty-Based Visual Injection (V-Inj). Intermediate layers often exhibit uncertainty for complex questions. As attention to visual tokens diminishes over layers, it is challenging for the model to incorporate necessary visual evidence independently, leading to propagated confusion. To address this, we propose uncertainty-based visual injection, which identifies uncertain layers and reintroduces visual evidence to support more accurate reasoning. Inspired by studies showing that feed-forward networks (FFNs) can act as key-value memories for factual knowledge retrieval [Geva et al., 2021] and that visual information can be effectively integrated into FFNs for hallucination mitigation [Zou et al., 2025], we inject visual tokens into the FFN during uncertain scenarios.

The standard FFN is defined as: FFN(x) = σ(x·W1)W2, where W1 ∈ Rd×D , W2 ∈ RD×d , σ is activation function, D is the intermediate dimension. Furthermore, by represent weights as W1 = (k1, k2, ..., kD) and W2 = (v1, v2, ..., vD)T , ki, vi ∈ Rd , the FFN can be rewritten in

Algorithm 1: Token Reorganization Strategy

Input: visual tokens Tv ∈ {T 1 v , ..., T N v }, T i v ∈ RP×d . Text tokens Tt, selected shallow layer r1, number of keyframes m, number of segments S, token budget B.

Output: Reorganized tokens Tfinal

## First Pass: Early Layer Execution

1. Obtain coarse frame representations (e.g., pooling):

1. Run r1-layer forward pass to get C(r) ˆ xt :

1. Aggregate C(r1) ˆ xt to get frame importance:

1. Identify top-k frames in each segment:

• ## Second Pass: Token Reorganization
• Compute pruning ratio:

1. Reorganize visual tokens Tv:

key-value form:

where FFN can be interpreted as using input x as queries to compute similarity with keys ki and gather values vi.

To inject visual evidence, we convert our reorganized visual tokens Tfinal ∈ RB×d into keys and values and perform similar retrieval from the visual key-value memory as:

where K(Ti), V(Ti) ∈ Rd are key and value corresponding to visual token Ti ∈ Tfinal. We employ identical mappings for K(·) and V(·) to maintain its training-free nature. This visual memory is integrated into FFN as:

where β = Norm(FFN(x))/Norm(VisInj(x)) is a balance factor and α is injection strength. For modern Gated Linear Units (GLU) instead of FFN in recent LLMs, the visual injection bypasses the gating mechanism for simplicity.

We dynamically activate visual injection based on the entropy H of output probabilities in intermediate layers rs -re. If entropy in layer r2 ∈ [rs, re] exceeds a threshold G, visual evidence is injected at the next layer r2 + 1 by replacing FFN(r2+1) as FFN(r2+1) visual . This injection is performed only once in uncertain scenarios to preserve feature stability. This method enriches the visual cues tied to the question and enhances reasoning consistency with visual content, thus improving the model's capability to integrate multimodal information for accurate predictions.

Early-Exit Pruning (EP). To reduce inefficiency in deep layers, we introduce early-exit pruning by omitting overall visual token processing. Initially, we explored a dynamic early exit based on entropy convergence, enabling the model to bypass unnecessary computations for high-confidence samples. However, experiments showed that samples with prolonged uncertainty paid minimal attention to image tokens in deep layers, indicating that delayed exits do not enhance understanding and instead increase computing costs. Therefore, we adopted a fixed-layer exit strategy, which simplifies implementation and ensures consistent efficiency.

Specifically, for layers r > r3, all visual token computations are omitted. The attention mechanism is simplified to only focus on text tokens Tt and the FFN computations for visual tokens are bypassed. This pruning strategy significantly reduces computational costs while ensuring the model maintains reasoning accuracy.

4 Experiments

4.1 Benchmarks andMetrics

We conducted evaluations of our method on 6 long video understanding benchmarks, including VideoMME [Fu et al., 2024a], LongVideoBench [Wu et al., 2024], EgoSchema [Mangalam et al., 2023], MLVU [Zhou et al., 2024], NExTQA [Xiao et al., 2021], and PerceptionTest [Patraucean et al., 2024]. Following evaluation tool LMMs-Eval [Zhang et al., 2024a], we perform standardized evaluation settings and metrics, i.e., accuracy, on each benchmark. We evaluate the efficiency by computing FLOPs and prefill time of LLMs using the library from LLM-Viewer [Yuan et al., 2024], and assume 1000 text tokens for LLaVA-Video due to their time prompt, while 100 for the others.

4.2 Implementation Details

Our implementation mainly follows LLaVA-Video-7B [Zhang et al., 2024d]. We adopt SigLIP [Zhai et al., 2023] as the vision encoder and Qwen2 [Yang et al., 2024] as the LLM. For DToMA, the selected layer r1 = 3, r2 ∈ [12, 18], r3 = 21. For TKR, following optimal design [Du et al., 2024] for SigLIP, we use 2×2 pooling for keyframes, while 3×3 for coarse non-keyframes. Token budget B is pre-defined according to experimental requirements, and token compression ratio is self-adaptive according to B. With no specifically stated, we set m = S = N/4. For V-Inj, we set threshold G = 0.75, factor α = 0.25. We also adapt DToMA in LLaVA-OV-0.5B [Li et al., 2024], Qwen2-VL-2B [Wang et al., 2024] to evaluate generalizing across VideoLLM architectures and sizes.

4.3 Comparative Evaluation

Comparison with SOTA Methods. We compare DToMA with state-of-the-art (SOTA) methods, focusing on long video comprehension under identical token budgets. Specifically, we evaluate DToMA across three different architectures and sizes of video models, i.e., LLaVA-OV-0.5B, Qwen2-VL-2B, and LLaVA-Video-7B. By compressing visual tokens and using coarse representations for non-keyframes, DToMA enables more frame inputs within the same token budget. We set a double frame input for DToMA, with an 80.5% nonkeyframes compression ratio. Table 1 summarizes the results on 6 video understanding benchmarks. DToMA consistently improves performance across all 3 tested models, with the averaged improvement of 1.48%, 1.35%, and 1.43% on the 6 benchmarks, respectively. Specifically, when applied to LLaVA-Video-7B, it achieves SOTA results on 5 out of 6 benchmarks. Moreover, under the same token budget, the efficiency of DToMA also shows an improvement, with 83.6% FLOPs and 79.0% prefill time, which will be discussed in the ablation. This demonstrates the effectiveness of DToMA in enhancing both efficiency and understanding capabilities, and the generalization ability across diverse architectures and sizes of VideoLLMs.

Comparison with token reduction and efficient video representation Methods. To assess performance and efficiency (FLOPs and prefill time) of DToMA, we compared it with existing token reduction techniques, i.e., ToMe [Bolya et al., 2023], FastV [Chen et al., 2025], FasterVLM [Zhang et al., 2024c], and efficient video representation strategies, i.e., coarse representation ˆ Tv, SlowFast [Zhang et al., 2024d], SF-LLaVA [Xu et al., 2024], on VideoMME using LLaVAVideo with 64 and 128 frame inputs. The results are shown in Table 2. It reveals that DToMA achieves superior performance in terms of FLOPs and prefill time while improving accuracy on VideoMME. Specifically, With 64-frame videos, DToMA at a 50% compression ratio surpasses baseline (+0.4%) whereas other methods perform below baseline. For 128-frame videos, DToMA's advantages are more highlighted. This is especially notable as 128-frame inputs increase redundancy challenges for LLaVA-Video which is trained on 64-frame inputs. DToMA consistently outperforms the baseline (+1.0%) and other methods (+0.7%) with fewer FLOPs and prefill time, proving the effectiveness of DToMA in improving efficiency and understanding abilities.

4.4 Ablation Study

Effectiveness of Each Component. We ablate the three key components of DToMA: text-guided keyframe-aware reorganization (TKR), uncertainty-based visual injection (V-Inj), and early-exit pruning (EP), in VideoMME dataset based on LLaVA-Video. The results are shown in Table 4. TKR reduces the token count by 50% and thus achieves marked efficiency gains, using only 49.6% of baseline FLOPs and 43.9% of prefill time while improving performance. It demonstrates that TKR effectively emphasizes the related visual context and reduces redundancy, thereby enhancing both efficiency and accuracy. V-Inj reintroduces visual information when uncertainty is detected, adding less than 0.5% FLOPs but boosting accuracy by 0.4% with baseline tokens and 0.7% with reorganized tokens in 128 frames. The results prove its effectiveness in mitigating uncertainty and aligning reasoning with visual context with only a small computing increase. EP uses 72.9% FLOPs and 71.7% prefill time of baseline to pro-

63.3

63.4

63.7

63.1

63.8

63.6

64.0

64.5

64.4

63.8

65.2

65.0

Table 4: Deployment layer analysis of each component in DToMA based on LLaVA-Video in VideoMME dataset.

cess the same visual tokens by pruning all visual tokens in deep layers, improving model efficiency without compromising performance. Overall, these strategies enable DToMA to consistently improve both performance and computing efficiency. Notably, with 128-frame inputs, DToMA achieves a 1.7% improvement over the 64-frame baseline, using just 83.5% of the FLOPs and 81.4% of the prefill time, proving its excellent accuracy-efficiency trade-off.

Layer selection analysis. In DToMA, the methods TKR, V-Inj, and EP target inefficiencies in shallow, intermediate, and deep LLM layers, respectively. We select target layers based on attention and entropy dynamics. TKR is applied at the first layer when cross-attention shifts from inclined to vertical patterns. V-Inj starts when entropy diverges between easy and hard questions and ends when entropy rises sharply again, with visual attention drops. EP is applied when attention nearly vanishes. We evaluated various layer indices to find the best balance between accuracy and efficiency. Results in Table 4 show the optimal setup is r1 = 3, r2 ∈ [13 - 18], r3 = 21, maximizing accuracy while minimizing resource usage. We also tested dynamic r3 based on model uncertainty, but found no clear performance benefits and delayed the early exit due to diminished attention focus. Thus, we opt for a fixed early exit layer.

Uncertainty Decline Analysis. To investigate the impact of V-Inj on uncertainty, we visualize the uncertainty H across LLM layers in two datasets: VideoMME and LongVideoBench. The visualization results in Fig. 5 reveal that injecting visual content at intermediate layers greatly reduces model uncertainty. This reduction is not only immediate but also propagates to deeper layers, thus ensuring a sustained decrease in uncertainty throughout VideoLLM. VInj also lowers the uncertainty peaks in intermediate and deep layers, enabling the model to make more informed responses based on visual context in complex scenarios. The improved consistency and confidence underscore the value of V-Inj in enhancing the performance and reliability of VideoLLMs.

Hyper-parameter Analysis. To investigate the impact of hyper-parameters on DToMA, we conduct experiments by varying segment S in TKR, injection ratio α, and threshold G in V-Inj. More specifically, we consider the following settings: S ∈ {1, 4, 16, 32}, α ∈ {0.1, 0.15, 0.2, 0.25, 0.3, 0.4}, and G ∈ {0.6, 0.65, 0.7, 0.75, 0.8, 0.85}. The results in Fig. 6 show that the optimal performance is achieved with S = 16, α = 0.25, and G = 0.75.

5 Conclusion

This paper addresses the challenge of efficiently processing long videos in VideoLLMs while enhancing comprehension. By analyzing reasoning patterns of VideoLLMs in multi-choice VideoQA task through entropy and crossattention dynamics, we identify three distinct reasoning stages-shallow, intermediate, and deep stages-and the inefficiencies existing in these stages. To address these, we proposed DToMA, a training-free Dynamic Token Manipulation method in three aspects: text-guided keyframe-aware reorganization, uncertainty-based visual injection, and earlyexit pruning. Experiments on 6 long video benchmarks show that DToMA boosts both performance and efficiency, outperforms SOTA methods, and generalizes well across various VideoLLMs. In sum, DToMA offers a promising alternative to directly using the baseline, enabling superior long video understanding within reduced computational resources.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grants 62325206 and 62301276, the Key Research and Development Program of Jiangsu Province under Grant BE2023016-4, and the Opening Foundation of the State Key Laboratory of Tibetan Intelligence, Key Laboratory of Tibetan Information Processing, Ministry of Education (2024-2-003).