Spatially Generalizable Mobile Manipulation via Adaptive Experience Selection and Dynamic Imagination

1 Introduction

Mobile Manipulation (MM) is a cornerstone of embodied AI, enabling agents to perceive, reason, and act in unstructured environments. Unlike isolated navigation or fixed-base manipulation, MM requires agents to coordinate locomotion and dexterous interaction over extended spatial and temporal horizons. For example, MM allows embodied agents to complete complex goal-directed tasks such as opening doors and fetching objects, as shown in Fig. 1 (c). MM is quite challenging, as it typically involves long-horizon decision-making over multi-stage compositions of heterogeneous skills [Huang et al., 2023; Lin et al., 2025b]. To address the complexity of MM, early methods decompose the problem into navigation [Chen et al., 2024; Kang et al., 2024] and stationary manipulation [Nguyen and La, 2019]. This factorized design simplifies modeling and learning, and has shown promise in structured environments. However, such approaches [Jauhri et al., 2024; Arduengo et al., 2021; Feng et al., 2025] often optimize only short-term objectives and overlook the dynamic coupling between mobility and manipulation, making them vulnerable to error accumulation over long horizons.

∗Corresponding Author.

Recent work [Honerkamp et al., 2023a; Huang et al., 2023] models the mobile base and manipulator as a unified system using whole-body control or joint policy learning. Despite encouraging progress, these solutions, especially model-free Reinforcement Learning (RL)-based methods [Honerkamp et al., 2023a; Huang et al., 2023], usually suffer from inefficient exploration and low sample efficiency. The reason is that MM is essentially a long-horizon task consisting of skills with strict sequential dependencies, which introduces unique data distribution challenges that are different from those of standard RL benchmarking: (1) Failures in the early stages (navigation) can hinder the experience collection of all downstream tasks (picking and placing), thus hindering the learning of complete skill chains. (2) As the trajectory lengthens, the experience buffer becomes dominated by navigationintensive fragments. This results in sparse but critical interaction moments in the trajectory being overwhelmed by redundant navigation experiences. (3) Such methods lack explicit mechanisms for learning reusable interaction patterns, which can easily lead to skill forgetting. Although recent data-driven or model-based methods [Chen et al., 2025; Cen et al., 2025; Lin et al., 2025a] can help mitigate these issues, they are often sensitive to environmental variations and difficult to spatially generalize. The above limitations raise a fundamental question: How can a MM policy efficiently exploit long-horizon experience to achieve robust decision-making and generalization across different spatial layouts?

Recent advances [Sun et al., 2023; Tafazoli et al., 2025] in system neuroscience suggest that higher animals solve novel tasks by recalling task-relevant episodic fragments and composing them with previously acquired skills or habitual behaviors. Inspired by this, we aim to investigate how to memorize key and informative episodic fragments from rich experience trajectories, and use them to improve RL-learned MM policies so that they can adapt to new spatial layouts. To this end, we address the above limitations by recalling episodic memories and composing them with MM skills to produce robust and spatially generalizable MM behaviors.

As shown in Fig. 1 (a), we propose Adaptive Experience Selection (AES) to make MM agents pay more attention to key experience fragments in long trajectories that affect task success, improving skill chain learning and mitigating skill forgetting. Based on the informative state transitions wellselected by AES, a Recurrent State-Space Model (RSSM) is introduced for Model-Predictive Forward Planning (MPFP) by capturing the coupled dynamics between the mobile base and the manipulator and imagining the dynamics of future manipulations. RSSM is trained together with model-free RL-based MM skill learning, and they share a single experience buffer. The difference is that AES helps RSSM memorize critical episodic fragments in a sample-efficient manner, thereby identifying weaknesses of the current MM skills and further consolidating them. As shown in Fig. 1 (b), RSSMbased MPFP can prospectively imagine future state transitions and evaluate state goodness to support RL-based MM skill learning. As system neuroscience suggests, RSSM here acts as a function of memorizing and recalling episodic fragments, while RL-learned skills act as habitual behaviors specific to MM. Their combination can improve MM policies on the current task and generalize to new spatial layouts without performance loss and additional training.

We implement our above ideas based on the popular EndEffector (EE)-centric MM framework [Honerkamp et al., 2023a]. Our approach is evaluated through sufficient comparative studies in diverse experimental configurations, demonstrating significant performance gains over the State-of-TheArt methods. Ablation studies further confirm the individual contributions of AES and RSSM-based MPFP to sample efficiency and spatial generalization. Finally, real-world robotic experiments highlight the practicality of our methods in realistic MM scenarios. In summary, our contributions are threefold: (1) We propose a system neuroscience-inspired method that leverages sample-efficient episodic recall to consolidate RL-based skill learning for robust and spatially generalizable MM. (2) An AES mechanism is introduced to make MM agents pay more attention to key experience fragments in long trajectories that affect task success, improving sample efficiency. (3) RSSM-based dynamic imagination is proposed for MPFP by capturing the coupled dynamics between the mobile base and the manipulator and imagining the dynamics of future manipulations.

2 RelatedWork

Model-Free RL-based Mobile Manipulation. Due to the difficulty of planning in the joint space of the mobile base and the manipulator, early work solves an MM task by decomposing it into sequential navigation and static manipulation tasks. Such a decomposition has been widely adopted in methods based on planning [Jauhri et al., 2024], reachability analysis [Feng et al., 2025], and RL [Gu et al., 2022; Fang et al., 2022]. Planning-based methods [Jauhri et al., 2024] plan trajectories for robots in joint space to ensure the kinematic feasibility of MM tasks. Given explicit task constraints, inverse reachability maps [Vahrenkamp et al., 2013] can also be exploited to determine favorable placements for the mobile base that facilitate subsequent manipulation. However, these classical methods are usually unable to instantly respond to diverse unstructured scenes and dynamic changes, and are thus less adaptable. Recently, RL-based approaches [Gu et al., 2022; Huang et al., 2023] mitigate these issues by using hierarchical policy learning and curriculum learning techniques. In addition, there are methods [Jauhri et al., 2022] that use classical methods as behavioral priors to guide sample-efficient exploration. Most recently, some methods [He et al., 2025; Honerkamp et al., 2023a] investigate the EE-centric MM framework, which achieve strong autonomy and robustness by explicitly solving for base motion that cooperates with the manipulator during grasping.

Despite recent progress, existing MM policies still struggle with low sample efficiency, as they fail to exploit the sparse but informative transitions embedded in long-horizon trajectories. Moreover, policies learned for specific tasks generalize poorly to new spatial layouts due to the absence of mechanisms for memorizing and reusing task-relevant experience. In this work, we address these issues by proposing AES and RSSM-based dynamic imagination.

Model-based Mobile Manipulation. Vision-LanguageAction (VLA) models [Liu et al., 2025; Zhang et al., 2025] have recently emerged as a powerful paradigm for grounding multimodal instruction and perception into direct robot control. HybridVLA [Liu et al., 2025] jointly integrates autoregressive and diffusion-based policy heads to improve robustness across diverse manipulation tasks. EchoVLA [Lin et al., 2025a] augments VLA with declarative and episodic memory to better handle long-horizon MM and spatial context reasoning. Diffusion-based action models [Wen et al., 2025] introduce masked diffusion mechanisms tailored for structured action generation, setting new standards over autoregressive VLAs in multimodal control. In parallel, worldmodel-augmented frameworks [Cen et al., 2025; Hafner et al., 2025] seek to unify dynamics prediction with action planning, enabling joint learning of environment dynamics and policy, which can improve long-horizon foresight. Despite these advances, current model-based methods often require large multimodal datasets and still struggle with sample efficiency and reliable generalization across diverse spatial layouts. In thiswork, we employRSSM-basedMPFP to improve and generalize RL-based MM skills to address these issues.

3 Preliminaries

Problem Definition. The MM problem we consider in this work involves a robot that includes a mobile base and a manipulator. The MM tasks described in this paper involve skills such as navigation, collision avoidance, picking, placing, and opening. Such a problem is modeled as a goalconditional Partially Observable Markov Decision Process (POMDP) defined as a tuple X = (S,O,A,P,R, G, ρ, γ) with underlying state space S, observation space O, action space A, reward function R : S ×A → R, initial state distribution ρ and discount factor γ. The state transition function P : S ×A× S → R+ represents the probability density of the next state st+1 ∈ S given the current state st ∈ S and action at ∈ A. For tasks like object rearrangement [Szot et al., 2022], the goal space g ∈ G is specified using the object's initial pickup pose or the goal pose where the object has to be placed. Our objective is to learn a policy πθ(at |ot , g) mapping an observation ot ∈ O and goal g to an action at that maximizes the expected return [Chen et al., 2023]:

where V ∗ (st ; g) = Eπθ,P [ ∑Tk=t γ k-tR(sk , π∗ θ (st ; g))] is the optimal value function. In this setting, P(st+1 | st , at ) represents the state transition over time, which is usually unknown to the MM agent. In this work, our MM policy recall key experience fragments and combine them with habitual skills to produce robust and spatially generalizable MM behaviors. Therefore, the state transition P and the optimal value function V ∗ are learned together through dynamic imagination and model-free RL, respectively.

EE-Centric MM. To instantiate the MM tasks, we use the Robot Operating System (ROS) to control an robotic system consisting of a Husky mobile base and a 6-DoF UR5 manipulator. Following the EE-centric MM framework [He et al., 2025; Honerkamp et al., 2023a], given a sequence of 6D poses in SE(3), the robot needs to find appropriate base placements in SE(2) to allow the manipulator's EE to reach these 6D goal poses sequentially along dynamically feasible paths. Such a solution is natural, just as humans reach for objects through leg movements in conjunction with their arms.

In practice, we employ an Inverse Kinematics (IK) solver to solve the manipulator's motion in joint space. On this basis, our objective is to learn the navigation of the mobile base to cooperate with the long-horizon manipulation. This is challenging because the mobile base must move in a collision-free manner while ensuring that the next 6D goal pose of the manipulator is reachable. Otherwise, the MM task is terminated due to collision of the mobile base or IK solving failure of the manipulator. To ensure the robot's kinematic feasibility, the mobile base's 2D action space A ∈ R2 is continuous. The observation space O includes a robot-centric local occupancy map mt and a state vector vt consisting of the previous actions at-1 , the joint states of robot, the EE's pose and velocity, and the desired pose of EE. Such an EE-centric MM framework inherits decades of experience in the field of robotics and incorporates the advantages of robot learning while ensuring robustness and practicality.

4 Methodology

In the following section, we propose leveraging sampleefficient episodic recall to reinforce and spatially generalize RL-based MM skills. We first present how to employ AESenhanced RSSM to memorize informative experience fragments for dynamic imagination (§4.1). Then, we present how to achieve MPFP based on RSSM to boost RL-based MM skill learning and promote spatial generalization (§4.2).

4.1 AES-Enhanced RSSM for Dynamic Imagination

Recurrent State-Space Model. RSSM is responsible for modeling state transitions in a latent space to capture the coupled dynamics between mobile base and manipulator. We adopt a structured latent representation consisting of two components: the stochastic representation zt and the recurrent state ht , maintaining consistent MM progress while incorporating new observations. An encoder eϕ is used to encode the observation ot-1 = {mt-1 , vt-1 } as a stochastic representation zt-1 , which is integrated into the recurrent state ht-1 to form a comprehensive state representation for MM decision-making. A recurrent module fϕ is utilized as the fundamental component of RSSM for latent state modeling, ensuring consistent temporal dynamics across state inference and future prediction. A decoder pϕ is used to predict observations and rewards from the recurrent state. As shown in Fig. 2 (a), the components of our dynamic imagination model are outlined below:

According to the problem definition in Sec. 3, the dynamic model fϕ approximates P by modeling state transitions. The training of eϕ, fϕ, and qϕ is parallel to RL-based MM skill learning, and they share the same experience buffer. The difference is that RSSM's training data is well-selected by AES as follows. The loss functions used to train RSSM are consistent with Dreamer V3 [Hafner et al., 2025].

Adaptive Experience Selection. MM skills learned in sample-inefficient manner are difficult to generalize to new spatial layouts. On the one hand, standard uniform sampling is insufficient, as it tends to overfit the abundant navigation data while neglecting the critical manipulation experience strongly associated with task success. On the other hand, focusing solely on hard failures risks catastrophic forgetting of foundational navigation capabilities. To address this challenge, we introduce an AES mechanism that pays more attention to critical state transitions in MM trajectories, as shown in Fig. 2 (b). We aim to maintain a balance between common navigation and critical failure experience, thus fostering a dynamic model that is robust across all skills and diverse spatial layouts to enhance generalization and prevent skill forgetting. Technically, the RSSM is trained on fixedlength experience fragments that are extracted from episodes collected by the RL-based MM agent. For each MM episode that is put into the experience buffer, the experience segments are sliding experience windows that contain consecutive state transitions that have the highest total prioritization. Prioritization is achieved by assigning the i-th state transition a composite priority score P i total , which integrates three complementary signals:

(1) Informative Experience Selection: From a data distribution perspective, informative state transitions that exhibit significant changes usually account for a small percentage and can thus be considered outliers. These state transitions usually correspond to moments such as opening a door, picking up or placing the target object, and avoiding collisions, as shown in Fig. 2 (b). Therefore, we propose to transform the task of selecting critical experience into the detection of outliers in the feature space of observations. Specifically, given a sequence of observations {o1 , o2 , ..., oT }, where ot = {mt , vt }, we use d(oi , oj ) to denote the distance in feature space:

where λ is a weight used to balance magnitude. Based on Eq. (3), we define the k-distance neighborhood: Nk(oi ) = {oj |d(oi , oj ) ≤ dk(oi )}, where dk(oi ) is the distance to the k-th nearest neighbor. For each neighbor oj ∈ Nk(oi ), the reachability distance is defined as reach-dk(oi , oj ) = max{dk(oj ), d(oi , oj )}. Using this, the Local Outlier Factor (LOF) [Breunig et al., 2000] of oi is the inverse of its mean reachability distance:

Then, the LOF is calculated as the relative density of oi compared to its neighbors:

Based on this formulation, the observation is selected when LOFk(oi ) > 1. A high LOFk(oi ) indicates a novel and unusual state, receiving proportional priority.

1. Task Criticality: This signal marks state transitions containing IK solving failures of the manipulator. These informative failure modes receive significant prioritization Ptc, ensuring that RSSM consistently focuses on learning to navigate to coordinate with successful IK solving, thereby preventing future failures.

(3) Prediction Error: We follow the idea of prioritized experience replay [Schaul et al., 2015], employing the RSSM's overall training loss as a dynamic indicator of learning difficulty. The state transitions that are modeled incorrectly by RSSM are given higher priorities Pper, which the RSSM should focus on in the following training.

Overall, the final priority P i total for i-th state transition is computed as a weighted sum of three signals:

where w1, w2, w3 are respective weights and I indicates whether an IK solving failure occurs. In practice, all highpriority experience fragments are structured as a SumTree [Wang et al., 2024] to efficiently manage and sample them based on normalized prioritization.

4.2 Dynamic Imagination-based ForesightedMM

Since RSSM and MM skills are trained together, AESenhanced RSSM can provide foresighted guidance for RLbased MM skill learning through dynamic imagination. In addition, we emphasize using the AES-enhanced RSSM to generalize fully trained MM agents to new spatial layouts without additional learning. Following common practice [Jauhri et al., 2022; Honerkamp et al., 2023b], we model a maximumentropy MM policy ˆπθ to explore the state space of a given task using the Soft Actor-Critic (SAC) algorithm [Haarnoja et al., 2018]. The following RSSM-based MPFP is introduced to reinforce and spatially generalize MM skills during training and evaluation, respectively.

Model-Predictive Forward Planning. During MM decision-making in both training and evaluation, the agent does not directly execute the action predicted by the policy ˆπθ learned by RL. Instead, it leverages RSSM as an internal simulator for dynamic imagination. As shown in Fig. 2 (c), this process integrates the concurrently trained actorcritic networks (ˆπθ,Q), which are used for the action rollout in dynamic imagination and the value estimation of the last imagined step, respectively. At each timestep, the MM agent samples candidate action sequences from a Gaussian distribution N(µ, σ), where the mean µ is initialized by the actor policy ˆπθ(ot ; g) based on the current observation ot and goal g. The sampled actions are then iteratively refined through CEM for model-predictive planning. During each iteration of Cross-Entropy Method (CEM) [Rubinstein, 1999], L candidate action sequences of horizon H are drawn and evaluated in parallel by unrolling the RSSM:

where i = 0, ..., L - 1 and k ∈ [t, t + H - 1]. As shown in Fig. 2 (c), each H-horizon dynamic imagination yields a cumulative return:

which is an estimate of

in Eq. (1). Notably, the critic Q provides a terminal value estimate based on the last imagined observation ˆ ot+H i , which is decoded from the belief state ht+H i . The top-K sequences with the highest returns are selected as elites, and the distribution parameters (µ, σ) of ˆπθ are refitted to their empirical statistics. This sampling-evaluation-refitting loop repeats for a fixed number of iterations. After convergence, only the first action of the final mean sequence is executed by the MM agent, and the overall process repeats at the next timestep, forming a standard model-predictive cycle. The combination of RSSM rollouts, MM decision-making, and CEM optimization enables robust and foresighted planning under complex dynamics and uncertainty.

5 Experiments

Based on the EE-centric MM framework introduced in Sec. 3, we answer the following research questions through comparative and ablative studies: (1) How does our method perform compared to strong baselines? (2) Does RSSM-based MPFP help to generalize the model-free RL-based MM skills to new spatial layouts? (3) Does our proposed AES mechanism help improve sample efficiency? (4) Does RSSM-based MPFP help to reinforce the MM policy? (5) How well does our MM policy perform on the real-world robot?

5.1 Experimental Setup

Experimental Configurations. As shown in Fig. 3, we follow the experimental configurations in N2M2 [Honerkamp et al., 2023b] and consider four different experimental configurations: cross-room, warehouse-oriented, home-sceneoriented, and dynamic-scene-oriented mobile manipulations. We first train and evaluate the proposed method and baselines individually in each of the four scenes. Since cross-room MM involves more diverse skills, including navigation, collision avoidance, picking up, opening doors, and placing, we evaluate the performance of generalizing the MM agent fully trained in such a configuration to the other three configurations. This refers to generalizing MM agents to new spatial layouts without additional training. Please see the supplementary material for more details.

Baselines and Evaluation Metrics. We compare our proposed method with several model-free RL-based MM policies, including the SoTA EE-centric MM method N2M2 [Honerkamp et al., 2023b], an end-to-end whole-body control method adapted from N2M2 , the award-winning hierarchical method BHyRL [Jauhri et al., 2022], and the SoTA approach based on auxiliary task distillation [Harish et al., 2024]. In addition, we compare our method with two world modelbased MM policies: (1) Dreamer V3-based MM [Hafner et al., 2025]: This baseline adapts the Dreamer V3 architecture to the MM setting by learning a latent world model and performing latent imagination rollout for policy optimization. We modify the open-source implementation of Dreamer V3 to make it suitable for EE-centric MM to obtain experimental results. (2) TD-MPC2-based MM [Hansen et al., 2024]:

This baseline applies TD-MPC2 to the EE-centric MM setting as a model-based MPC-style planner that uses online trajectory optimization in the latent state space. Similar to our approach, only the first action from the optimized sequence is executed in the environment, resulting in a receding-horizon control loop suitable for complex MM dynamics.

We report the Average IK solution Failures (AIKF), Average Base Collisions (ABC), Task Completion Rate (TCR), and Perfect Success Rate (PSR) as evaluation metrics. A lower AIKF means a better cooperation between the mobile base and the manipulator. TCR implies that the MM robotic system completes the task while allowing for IK solving failures and base collisions. We set the threshold for allowing both kinds of failures to 20. PSR means completing the MM tasks with zero failures. For implementation details, please see Section 1 of the supplementary material.

5.2 Comparative Studies

To answer question (1), we first conduct comparative studies of the proposed method with all baselines in the challenging cross-room experimental configuration, and the experimental results are shown in Tab. 1. Our approach significantly outperforms the SoTA EE-centric MM policy (N2M2 ), the RLbased maximum entropy policy (BHyRL), the world modelbased methods (Dreamer V3 and TD-MPC2), and the auxiliary task distillation-based method (AuxDistill) in terms of reducing IK solving failures and increasing task success. As stated in Section 2 of the supplementary material, our method converges faster and better compared with strong baselines during training, reflecting that our method is more sampleefficient. The results of End-to-End MM indicate that it is difficult to learn MM policies based on whole-body control in an end-to-end manner using model-free RL. The results in Tab. 1 reflect that our method sometimes achieved suboptimal ABC metrics. This is mainly because our approach reduces early task termination due to IK solving failures, and thus the extended episodes increase the likelihood of collisions. Furthermore, we train and evaluate our method and strong baselines in three other experimental configurations, with results shown in Tab. 2. Either in static or dynamic scenes, our approach significantly reduces the number of IK solving failures (AIKF). In addition, our approach achieves competitive ABC metrics while significantly improving MM success rates (i.e., TCR and PSR). These results reflect the significant superiority of our method over the strong baselines. Training curves reflecting the sample efficiency can be found in the supplementary material. These advances are due to the fact that (1) AES enables our method paying more attention to those key state transitions that determine the success of the MM task; (2) RSSM-based MPFP helps to generate foresighted and collision-free mobile base motions to cooperate with the IK solving of the manipulator.

To answer question (2), both our method and three strong baselines trained in the cross-room experimental configuration are directly evaluated in the other three configurations without additional training. The results in Tab. 3 reflect that our approach can better generalize the RL-based MM agent to new spatial layouts, while N2M2 generalizes poorly. Compared to world model-based baselines, our approach is effective in reducing IK solving failures and collisions while improving task success. In addition, we find that RSSM-based MPFP is necessary because the spatial generalization of our method is significantly weaker when it is absent. The results in Tab. 2 and Tab. 3 also reflect that our method sometimes achieved sub-optimal ABC metrics. Such a phenomenon is especially prominent in dynamic scenes. Nevertheless, significant gains in the TCR metrics reflect the robustness and spatial generalization of our approach.

5.3 Ablation Studies

We answer questions (3) and (4) by respectively ablating different signals in AES and MPFP in the challenging crossroom experimental configuration. Based on the experimental results in Tab. 4, we find that the absence of all AES signals significantly degrades the MM performance, especially by increasing IK solving failures and decreasing the TCR and PSR metrics. In addition, the absence of both informative experience selection (AES-pies) and task criticality (AES-ptc) impairs MM performance, with AES-pies having a greater impact. These results demonstrate the importance of AES for memorizing key experience fragments for robust MM. In addition, training curves in the supplementary material demonstrate that our AES-based MM policy is sampleefficient. By ablating the RSSM-based MPFP in training and evaluation, respectively, we find that it always helps to facilitate the collaboration between the mobile base and the manipulator through farsighted dynamic imagination, which improves the MM performance. More studies on parameter H and computational efficiency can be found in Section 3 of the supplementary material.

5.4 Real-World Experiments

We deploy our method to a real robotic platform consisting of a Husky A200 mobile base and a UR5 manipulator to verify its practicality. The robot's upper computer is a laptop with an Intel Core i9-13900HX CPU and GeForce RTX 4060 GPU. The robot uses a RealSense D435 RGB-D camera and YOLO v7 [Wang et al., 2023] to identify target objects. We use ROS to organize the hardware and software resources of the robotic system. Considering the security, we set the maximum speed and decision frequency of the mobile base to 0.3 m/s and 1 Hz, respectively. Please see Section 4 of the supplementary material and videos for sim-to-real experiments.

6 Conclusion and Limitations

This paper investigates the problems of low sample efficiency and poor spatial generalization faced by MM in the context of embodied AI. Correspondingly, we propose the AES mechanism and RSSM-based MPFP to address these challenges. We find that AES, especially informative experience selection, can help RSSM memorize key experience fragments to significantly reinforce MM agents. RSSM-based prospective dynamic imagination can better generalize MM policies to new spatial layouts without additional training. Our problem modeling and experiments are carried out in an EE-centric MM framework, which fully combines the advantages of traditional robotic motion planning and robot learning. The practicality of our approach is verified by migrating it to a real-world robotic systemwithout additional training.

Limitations. To focus on the contribution of the AESenhanced RSSM to MM, we assumed an ideal observation space, especially using ground truth local occupancy maps directly. In practice, the construction of low-noise local occupancy maps from sensors is essential.

Acknowledgments

This work was supported in part by the Key Project of Xiangjiang Laboratory under 25XJ02003 and in part by the National Natural Science Foundation of China under 62272489, 62332020, and 62350004. This work was carried out in part using computing resources at the High-Performance Computing Center of Central South University.