S²-VLA: State-Space Guided Vision-Language-Action Models for Long-Horizon Manipulation

1 Introduction

In recent years, significant advances in vision-language models (VLMs) [Bai et al., 2025a; Chen et al., 2024; Karamcheti et al., 2024] have propelled the development of generalist robotic systems capable of perception, reasoning, and action under open-ended instructions. Within this paradigm, Vision-Language-Action (VLA) [Zhang et al., 2025a] models have emerged as a promising framework that bridges highlevel understanding to low-level control, enabling instruction driven robotic manipulation. The core of VLA research lies in effectively aligning multimodal representations comprising visual observations and language instructions with the action space to generate precise and executable policies.

The mainstream method aligns multimodal representations vision and language with the action space to generate executable policies by performing end-to-end finetuning of Vision-Language Models on large-scale robotic datasets [Collaboration et al., 2025; Khazatsky et al., 2025]. However, when applied to long-horizon manipulation tasks that require multi-step reasoning, these models are fundamentally limited by their static representation fusion mechanism. Considering a task such as put both the cream cheese box and the butter in the basket. The model employs fixed weights to fuse information from different modalities like vision and language throughout its execution. Consequently, during the grasping phase which demands precise localization, the model cannot sufficiently focus on spatial details of the object. Meanwhile, in the planning phase which requires comprehension of the overall goal, it fails to allocate adequate attention to semantic intent. This lack of phase-adaptive capability in its rigid design leads to decision biases at critical steps. More critically, in long-horizon tasks, the lack of a context-aware correction mechanism allows these early biases to propagate and amplify along the action chain, ultimately leading to task failure.

To overcome this limitation, we propose the S 2 -VLA framework. Its core is a novel State-Space Guided Adaptive Attention (SSGAA) mechanism. As illustrated in Figure 1, unlike static fusion, S 2 -VLA maintains a compact belief state that tracks task progression and dynamically gates three complementary information streams spatial visual features, semantic task intent, and temporal action history enabling phase-aware adaptation. Extensive experiments on benchmarks such as LIBERO [Liu et al., 2023], SimplerEnv [Li et al., 2024b] and ALOHA [Zhao et al., 2023] platform demonstrate that S 2 -VLA, despite with compact 2B parameters, consistently outperforms larger 7B-scale models and sets a new state-of-the-art, effectively mitigating error propagation and significantly improving long-horizon task success.

The main contributions of our work are as follows:

• We propose S 2 -VLA, which leverages pretrained VLMs to directly generate robot action policies through a belief-state-space interface, eliminating the need for large-scale action-sequence pretraining.
• We propose a novel State-Space Guided Adaptive Attention mechanism, which models task progress as a dynamic belief state. This belief state is used to guide a gating network that dynamically modulates the fusion of visual, linguistic, and action representations for longhorizon manipulation tasks.
• We show that S 2 -VLA achieves robust long-horizon robotic manipulation with a lightweight model architecture, outperforming prior VLA methods on multiple standard benchmarks.

2 RelatedWork

2.1 Vision-Language-ActionModels

In the ongoing development of generalist robotic policies, the field is observing a distinct trajectory that moves from imitation-learning-based expert policies toward generalpurpose foundation models. Early pioneers in this domain, such as ACT [Zhao et al., 2023], proved that it is possible to achieve strong dexterity by directly fitting the action distribution from demonstration data. Following this breakthrough, researchers have further explored generalist policies pre-trained on large-scale robot trajectories, resulting in representative systems like Octo [Team et al., 2024], RDT [Liu et al., 2025b], Seer [Tian et al., 2024], RT-1 [Brohan et al., 2023], RT-1-X [Collaboration et al., 2025], and UnifiedVLA [Li et al., 2025b]. However, it is worth noting that these approaches often struggle to handle open-world semantic instructions effectively. They tend to face challenges with longtail concepts and compositional generalization, especially in scenarios where semantic understanding is not grounded in strong pre-trained VLMs. Consequently, to address this bottleneck, contemporary research has started to leverage pretrained VLMs as semantic backbones, a strategy that has successfully given rise to VLA models [Zhang et al., 2025a; Gao et al., 2025].

Current VLA architectures can be broadly summarized into two paradigms for action decoding. Autoregressive Token Generation Models discretize actions into tokens and generate them sequentially, such as OpenVLA [Kim et al., 2024], RT-2 [Zitkovich et al., 2023], π0-FAST [Pertsch et al., 2025], CoT-VLA [Zhao et al., 2025], FlowVLA [Zhong et al., 2025], UniVLA [Bu et al., 2025], TraceVLA [Zheng et al., 2025], WorldVLA [Cen et al., 2025], SpatialVLA [Qu et al., 2025], Magma [Yang et al., 2025], MolmoAct [Lee et al., 2025], etc. While semantically more unified, this serial generation typically incurs higher inference latency. In contrast, Parallel Decoding with Decoupled Policy Heads decouples representation from execution to simultaneously improve speed and precision, such as OpenVLA-OFT [Kim et al., 2025], π0 [Black et al., 2024], GR00T N1 [NVIDIA et al., 2025], SmolVLA [Shukor et al., 2025], CogACT [Li et al., 2024a], 4D-VLA [Zhang et al., 2025b], ThinkAct [Huang et al., 2025], CronusVLA [Li et al., 2025a], MemoryVLA [Shi et al., 2025], RoboVLMs [Liu et al., 2025a], and others. Under this paradigm, the VLMmainly provides semantic representations or high-level planning signals, while concrete action generation is handled by a decoupled policy head (e.g., diffusion or flow-matching models or ACT-style chunking). We adopt the latter paradigm to achieve efficient continuous control, with a focus on optimizing the feature fusion mechanism, aiming to fully leverage the perceptual capabilities of VLMs and appropriately bridge them with action generation for effective long-horizon task execution.

2.2 Policy Generation and Feature Integration

The performance of decoupled VLAs depends on what information is utilized from the backbone and how it is integrated. Existing methods typically exhibit statically configured feature integration, which limits their adaptability.

Static Focus on Compressed Representations. Most mainstream methods [Liu et al., 2025b; Kim et al., 2025] restrict their attention to the final output representations. Specifically, OpenVLA-OFT [Kim et al., 2025] utilizes Learnable Action Queries to perform parallel decoding of future action chunks. However, the policy head attends exclusively to embeddings at these specific query token positions. While these embeddings encapsulate rich semantic intent, they constitute an information bottleneck, abstracting away the fine-grained spatial details present in the VLM's image token embeddings. This rigid focus makes it difficult for the model to recover the precise geometric information required for high-dexterity tasks.

Static Multi-level Fusion. To retrieve lost details, SmolVLA [Shukor et al., 2025] and VLA-Adapter [Wang et al., 2025] introduce mechanisms to aggregate multilevel backbone features (e.g., intermediate hidden states). Although this expands the information scope, the fusion strategy remains static. The models learn a fixed set of weights to balance these information sources. Consequently, they fail to adaptively shift their focus as applying the same fusion ratio regardless of whether the robot is in a visually critical phase (e.g., aligning for a grasp) or an intent-critical phase (e.g., executing motion primitives).

We argue that effective manipulation necessitates phase aware trade-offs visual precision for localization versus semantic intent for planning a critical temporal dynamic ignored by current methods.

3 Methodology

3.1 ProblemFormulation

We formalize the language-conditioned long-horizon manipulation tasks in VLA model. At each time step t, an agent receives a multi-view visual observation Vt, a natural language instruction Lt, and proprioceptive feedback Pt. A standard VLA policy is typically formulated as a direct mapping from these inputs to a future action sequence

where At:t+K-1 = [at, at+1, . . . , at+K-1] denotes K the predicted actions.

Although effective for short-horizon tasks, this formulation fundamentally lacks an explicit mechanism to maintain temporal coherence. Each action is generated from a static sensory snapshot, disregarding the execution history and how past actions have led to the current state. This makes the model susceptible to error accumulation in long-horizon scenarios.

3.2 S 2 -VLA Belief-Driven VLA Paradigm

To overcome the lack of temporal coherence in static VLA models for long-horizon tasks, we formulate an internal belief state bt ∈ Rdb . This state serves as a compact temporal context, summarizing where we are in task execution, and is recursively updated to encode task progression and historical information. The belief state update integrates the recent action sequence and its sensory feedback, formulated as

where fϕ is a learned update function, K is the lookback horizon, and t and l denote the time step and the layer index of the action head, respectively. h(l) t represents the hidden state updated via the l layer at time step t, initialized by the low-level features h(0) t . This formulation explicitly models the causal relationship between the agent's actions At-K:t-1 and environmental feedback (proprioception), enabling the model to track task progress and detect execution deviations. For initial steps where the number of past actions is less than K, zeropadding is applied to the sequence. We implement fϕ using a lightweight Gated Recurrent Unit (GRU) [Cho et al., 2014]. Its recurrent nature maintains temporally coherent representations, compressing the history of action-perception pairs into a dynamic representation of task phase and execution quality. The complete policy is then conditioned on this belief state to generate actions as

The belief state bt is learned end-to-end from the action prediction loss, without any explicit supervision on task phases or execution errors. By minimizing prediction error, the model is forced to encode predictive information about future successful actions from the history of action-perception sequences. Consequently, distinct patterns arising from different task phases (e.g., approaching, grasping, placing) as well as discrepancies between expected and actual outcomes due to errors or perturbations are naturally encoded into bt. This makes bt an latent representation that emergently encodes both task-phase information and execution fidelity, distilled directly from action-perception dynamics through endto-end optimization.

Framework Overview

The belief state bt provides crucial guidance for adaptive fusion of multimodal information. Building upon this, we propose the S 2 -VLA framework. Its core is a novel State-Space Guided Adaptive Attention mechanism. SSGAA consist of three functionally complementary and parallel attention pathways. Instead of static fusion, these pathways are adaptively integrated via a dynamic gating network regulated by the belief state bt, enabling stage-aware and adaptive feature aggregation. This design enables the policy to dynamically balance spatial details an d semantic intent based on action history and proprioception, thereby enhancing execution consistency in long-horizon tasks. The overall architecture of S 2 -VLA are illustrated in Figure 2.

Unified Input Construction

Visual features Ft = ViT(Vt) ∈ RNv×d are extracted from multi-view images Vt. Language instructions L are tokenized into {l1, l2, . . . , lNl } ∈ RNl×d , and learnable intent tokens Atokens are embedded as {˜a1, ˜a2, . . . , ˜aNa } ∈ RNa×d for task-specific context. The combined input sequence is:

The sequence is processed through H Transformer layers to produce hierarchical representations.

State-Space Guided Adaptive Attention

The SSGAA mechanism consists of belief-guided dynamic gating and three complementary attention pathways, all operating on contextualized representations derived from the unified input sequence X.

Action Sequence Self-Attention. Ensures temporal coherence in the predicted motion trajectories. This branch operates on a dedicated action-sequence embedding A(0) ∈ RT×K×d where T denotes the action dimension and K the number of predicted steps, which is initialized as zero vectors and refined layer-wise through self-attention [Vaswani et al., 2017] within the SSGAA module. This enables modeling of the temporal dependencies across the predicted K steps.

Low-Level Visual Cross-Attention. Provides precise spatial grounding by attending to fine-grained visual features. We construct a visual vector Cvis ∈ RNv×d by stacking the hidden states of visual tokens from one transformer layer. The operation is defined as

where Q ∈ RK×d is the query projection derived from the learnable action sequence representation and Wvis k ,Wvis v ∈ Rd×d are learnable key and value projections that preserve fine-grained spatial details crucial for object manipulation.

High-Level Intent Cross-Attention. Facilitates semantic intent understanding by processing aggregated multimodal context. We form a semantic context vector Cite ∈ RNa×d from the hidden states of the Na learnable intent tokens across layers. The attention is computed as

where Wite k ,Wite v ∈ Rd×d project the action-token representations, which encode the distilled task semantics, into keys and values respectively.

Belief-Guided Gating. The belief state bt dynamically modulates these three branches through a gating mechanism. At layer l, soft gating weights are computed as

Branch outputs are fused as

where O(l) vis , O(l) ite , and O(l) act denote the outputs of the visual, semantic, and action self-attention branches at layer l, respectively. The fused representation H(l) is processed by a feed-forward network. The processed output then serves a dual function. First, it provides the query input Q(l+1) for the next layer's visual and context cross-attention branches. Second, it is fed into the same layer's action sequence selfattention branch. This design enables iterative refinement through multi-layer processing.

Parallel Action Decoding

The final action prediction is generated by applying layer normalization followed by a linear projection to the output of the last SSGAA layer are computed as

where LN(·) denotes layer normalization. The learnable parameters are the weight matrix Wout ∈ RT×d and the bias vector βout ∈ RT . The bias βout is broadcasted across the temporal dimension K during the addition.

3.3 Training and Inference Strategies

Unified Loss Formulation

The training objective is defined by the action prediction loss, which ensures precise trajectory generation by minimizing the discrepancy between predicted and ground-truth actions

where at+k denotes the ground-truth action at future step t+k and ˆat+k is the corresponding prediction. This formulation directly optimizes the model for action prediction accuracy without additional regularization terms.

End-to-End Training and Inference

The S 2 -VLA model is trained end-to-end, jointly optimizing all components including the pre-trained Qwen3-VL [Bai et al., 2025a] backbone, belief update module, and SSGAA attention mechanism to allow gradient flow and coordinated adaptation for robotic manipulation. During inference, the model operates in a temporally recurrent loop. At each step, it first updates its state through a GRU [Cho et al., 2014]. The model then computes adaptive gating weights to fuse visual, textual, and action features into a unified representation. Based on this representation, it predicts and executes the next action before proceeding to the following time step. This design ensures temporal consistency and robustness via continuous state estimation. Simultaneously, the dynamic gating mechanism enables automatic attention shifting across different modalities according to the current task phase, thereby facilitating adaptive and reliable task execution. Details are provided in the supplementary material.

4 Experiments

4.1 Experimental Setup

To evaluate the robustness of policies in long-horizon manipulation tasks, we evaluate S 2 -VLA on three main benchmarks. During training, the model is optimized using four H100 GPUs. For inference, the system requires only 7GB of VRAM to perform deployment. In simulation, we use the LIBERO [Liu et al., 2023] benchmark (Spatial, Object, Goal, and Long subsets) with fixed trajectories per task, language instructions, and third-person views, following official protocols with scripted initial scene sampling. Additionally, we evaluate within the SIMPLER [Li et al., 2024b] beachmark which is a high-fidelity simulation platform designed to bridge the real-to-sim control and visual gap by faithfully replicating real-world conditions for representative robotic arms such as WidowX. Extensive testing has demonstrated a strong correlation between SIMPLER performance and realworld results. In the real world, we use the ALOHA bimanual system [Zhao et al., 2023] (30 Hz control) with multi-view overhead and wrist cameras and proprioceptive inputs. Tasks include pick-and-place, stacking, desktop organization, and utensil distribution. Experimental setup details are provided in the supplementary materials.

4.2 Main Results Analysis

Performance on Simulation Benchmark

As shown in Table 1, we conduct comparative experiments with multiple methods on the LIBERO benchmark. Bold is the best performance and Bold is the suboptimal performance. The results indicate that S 2 -VLA demonstrates SOTA while maintaining efficiency advantages. It achieves success rates of 98.4%, 99.6%, 98.4%, and 96.4% on the Spatial, Object, Goal, and Long-Horizon subsets, respectively, with an average success rate of 98.2%, significantly outperforming the compared methods. Particularly in long-horizon tasks, the model effectively mitigates error accumulation issues through the State-Space Guided Adaptive Attention mechanism.

Table 2 presents the performance on four manipulation tasks using the WidowX robot platform. Our method still achieves state-of-the-art results even in the highly realistic simulation environment, SimplerEnv-Bridge.

Real-World Experimental Evaluation

We evaluated S 2 -VLA on a real-world ALOHA dual-arm mobile manipulation platform through a series of complex manipulation tasks. The evaluation comprised four distinct tasks. In the pick-and-place task, the positions of a blue and a yellow block were randomly initialized, requiring the system to pick up the blue block and place it into a bowl. In the stacking task, after random initialization of the block positions, the system was required to stack the yellow block on top of the blue block. The desktop organization task involved randomly initializing block positions and then first placing the yellow block into a bowl, followed by placing the blue block into the same bowl. Finally, the utensil distribution task required the system, after random initialization of utensil positions, to use the left arm to pick up a pair of chopsticks, pass them to the right arm, and then place the chopsticks into a bowl with the right arm. Details are provided in supplementary materials.

As illustrated in Figure 3, the key steps in the execution process of our S 2 -VLA are presented. It achieved superior success rates compared to all baseline methods, demonstrating that its SSGAA mechanism enables effective real-world transfer capability.

4.3 Ablation Studies

This section ablates the core design of S 2 -VLA, the action head constructed from a 24-layer SSGAA mechanism. We systematically address four key questions (i) Is dynamic gating necessary? (ii) Is gating performance depth-sensitive? (iii) Does belief-state guidance provide additional benefit? (iv) What is the optimal SSGAA configuration? All experiments follow the same training and evaluation protocols as the main experiments, with only the gating strategy modified.

The study begins by evaluating whether state-driven gating improves performance. As shown in Table 3, disabling gating (w/o Gate) results in 95.0% performance, while enabling gating in all layers (Gate@All) slightly degrades performance to 94.4%. This suggests that indiscriminate gating across all layers is not beneficial. To investigate depth sensitivity, we conduct single-layer gating experiments, enabling gating only at specific layers while keeping others static. Results reveal a clear pattern that gating provides significant gains at middle layers (6, 12, 18), with layer 12 achieving the best performance (96.4%), while shallow and deep layers showminimal or negative impact. This reflects the hierarchical structure of feature extraction. Intermediate levels capture semantically rich features ideal for adaptive resource allocation, while early and final stages are better by static fusion.

Given the success of single-layer gating at layer 12, we investigate whether combining multiple high-performing layers (6, 12, 18) yields further improvements. Surprisingly, this multi-layer configuration performs worse (94.4%) than both the best single-layer setting and the no-gating baseline. This indicates that multi-point gating introduces instability. To isolate the contribution of belief-state guidance, we replace the dynamic state input bt with a learnable static vector in the Gate@12 configuration. This w/o State variant achieves only 95.8%, significantly lower than the full SSGAA (96.4%). This confirms that performance gains arise not merely from introducing gating, but specifically from the dynamic modulation of gating weights by the evolving belief state, enabling phase-aware adaptation.

Ablation on State Guided Gating

To further validate the core role of belief-state guidance, we conduct a controlled experiment. On the best-performing gating layer (Gate@12), we remove the input from the belief state bt and replace it with a learnable constant vector to generate static gating weights (denoted as w/o Belief State). The results in table 3 show that the static gating setup performs worse than the full belief-state-guided gating. This demonstrates that the performance improvement is not simply due to the introduction of a gating mechanism, but rather depends critically on having the gating weights be dynamically modulated by the current belief state bt. This dynamic mechanism enables the model to adaptively adjust the fusion of multisource information (visual, semantic, and action-sequence) according to the task phase. Our ablation study demonstrates that Gating is more effective at semantically rich middle layers and belief-state guidance is essential for optimal performance. Consequently, S 2 -VLA adopts single-layer gating at a critical middle layer (layer 12, The experimentally determined optimal SSGAA configuration layer) with belief-state guidance as its core configuration.

4.4 Evaluation of QwenVL-BasedModels

To isolate the performance gains attributable to architectural design, we systematically transplant several mainstream SOTA VLA Policy Head methods onto the QwenVL family of backbones including Qwen2.5-VL [Bai et al., 2025b] and Qwen3-VL [Bai et al., 2025a] for a fair comparison. As shown in Table 4, although all methods are built upon the same pre-trained QwenVL weights, S 2 -VLA achieves the highest success rate of 96.4% with its compact 2B parameter architecture. This performance not only surpasses the similarly sized Qwen3-VL+Adapter-Pro [Wang et al., 2025] with the success rate of 94.4% but also exceeds all methods using larger 3B and 4B backbones, which Qwen3-VL+OFT [Kim et al., 2025] only gets success rate of 93.8%. The results demonstrate that the SSGAA mechanism is more effective than merely scaling up parameters for long-horizon tasks. S 2 - VLA successfully breaks through the performance plateau observed in static fusion methods, which typically saturates in the 88%-94% range as task complexity increases. By integrating the semantic representations from the foundation model with real-time state predictions, it achieves dynamic shifting of perceptual focus across task phases, thereby enhancing robustness against long-horizon error accumulation.

4.5 Gating Interpretability Analysis

The primary cause of failure in long-horizon manipulation tasks lies in the propagation and amplification of perception or decision-making errors from early key steps during subsequent execution, ultimately leading to overall task failure. To elucidate how S 2 -VLA addresses this challenge at a mechanistic level, we performed step-by-step visualization of the gating weights in its SSGAA module. A representative longhorizon task put both the cream cheese box and the butter in the basket is selected for analysis. During a successful execution trajectory, the model's dynamic allocation of gating weights across its three attention branches is recorded at each reasoning step, which include low-level visual crossattention (preserving fine-grained spatial details), high-level semantic cross-attention (goal-oriented planning), and action sequence self-attention (temporal coherence in motion generation). The resulting dynamic trajectories of the weights, aligned with stage-wise keyframes, are shown in Figure 4. The main observations are summarized as follows.

During phases that require precise object grounding and fine spatial alignment (e.g., when the end-effector approaches the target object, corresponding to the "Aim target" and "Locate and move to basket" keyframes marked by a green triangle in Figure 4), the weight of the low-level visual crossattention (green curve, gvisual) rises significantly. This indicates that the model relies more heavily on early-layer visual tokens to support high-precision positioning.

At key decision points involving subtask switching or phase transitions (e.g., triggering the grasp action, transporting the object to the basket, and executing the release action, corresponding to stages such as "Grip item" and "Place item" in keyframes marked by a red triangle Figure 4), the weight of the high-level Intent cross-attention (red curve, gintent) exhibits clear peaks. This demonstrates that the model leverages aggregated multimodal task representations to perform explicit stage transitions and behavioral planning.

During long-range, steady-state motion segments, the action sequence self-attention (blue curve, gaction) remains dominant. This allows the model to maintain smooth and temporally consistent trajectories through coherent step-by-step motion modeling, thereby reducing unnecessary perceptual and planning overhead.

This SSGAA mechanism enables the model to adaptively enhance the most pertinent information-processing pathway according to different task phases, thereby improving decision robustness in error-prone stages such as fine manipulation and phase switching. Experimental results confirm that this mechanism effectively reduces the likelihood of errors in critical steps, suppresses error propagation and accumulation from the source, and consequently enhances both the overall success rate and execution stability of long-horizon manipulation tasks.

4.6 Performance EfficiencyMetrics

To verify inference efficiency, we compare the 7B model with our method, with results shown in Table 5.

5 Conclusion

S 2 -VLA effectively mitigates error accumulation in longhorizon tasks by employing a state-space-guided adaptive attention mechanism, which dynamically fuses visual, linguistic, and action representations. Despite its compact size of 2B parameters, the model outperforms conventional 7B-scale models and achieves state-of-the-art success rates on longhorizon manipulation benchmarks, including LIBERO [Liu et al., 2023] and SimplerEnv [Li et al., 2024b]. Future work will explore the architectural compatibility of the SSGAA mechanism and extend its core idea of belief state modeling to different generative frameworks, such as Diffusion models or FlowMatching, to enhance the modeling capability for complex multimodal action distributions.

Acknowledgments

This work is supported by the National Natural Science Foundation of China under Projects 62476089, 62576206 and 62572194, and by the Shanghai Science and Technology Committee under Grant No. 24511103202.

Contribution Statement

Zhipeng Xie and Zongyi Han contributed equally to this work. Jing Zhao is the corresponding author.