Walking on Ice: Adaptive Gait Control of Humanoid Robots Based on Visual Prediction and Proprioceptive Estimation for Variable Friction Environments

1 Introduction

Humanoid robots hold great promise for operating in humancentric environments, from household assistance to industrial inspection[Tang et al., 2025; Chen et al., 2025; Kojima et al., 2025]. A fundamental capability for such deployment is robust locomotion control across diverse scenarios. The context of advanced control technologies such as reinforcement learning[Li et al., 2024a; Li et al., 2024b]

* Corresponding authors.

have enabled impressive humanoid walking behaviors, including traversing stairs [Long et al., 2025; Duan et al., 2024], rough terrains [Cui et al., 2024; Yu et al., 2024], and sparse footholds [Radosavovic et al., 2024b; Gu et al., 2024; Wang et al., 2025]. However, these methods predominantly assume that the ground provides sufficient friction for stable foot-ground contact, or handle friction variations implicitly through domain randomization [Tan et al., 2018; Tobin et al., 2017] to stabilize gait control. Therefore, perceiving ground features such as friction variations is crucial for the gait control of humanoid robots.

Existing locomotion methods can be broadly categorized by their treatment of friction. Methods relying solely on domain randomization do not explicitly address friction, instead focusing on other aspects such as taking adaptive gaits by terrain geometry[Duan et al., 2024; Hoeller et al., 2024; Loquercio et al., 2023; Agarwal et al., 2023; Miki et al., 2022; He et al., 2025]. Yet, without explicit awareness of surface friction, these policies typically employ conservative gaits, or fail catastrophically when encountering slippery surfaces. While some recent works attempt to estimate terrain properties from proprioceptive perceptions[Kumar et al., 2021; Ji et al., 2022; Gu et al., 2024], or by leveraging visual information[Brandao et al., 2016; Peng et al., 2025]. Nevertheless, policies estimating friction from proprioception must first step onto the slippery surface before recognizing danger. Visual approaches also have limitations: CNN-based methods[Brandao et al., 2016] lack generalization to novel surfaces and are sensitive to lighting variations, while using Vision-Language Models (VLMs) with Retrieval-Augmented Generation (RAG) introduces additional retrieval latency that limits real-time applicability[Arslan et al., 2024].

In summary, enabling humanoid robots to walk robustly on surfaces with varying friction coefficients faces three key challenges: (1) Anticipating surface friction before contact. Unlike geometric features, friction is not directly observable from conventional visual representations and requires semantic understanding of material properties. (2) Adapting gait in real-time under imperfect friction prediction. Latency and uncertainty require the policy to handle prediction errors gracefully. (3) Learning a unified policy that generalizes across a wide friction spectrum. The optimal walking strategy differs between high and low-friction regimes.

To address these challenges, we propose Walking on Ice

(WoI), a friction-aware locomotion framework for humanoid robots. Our approach consists of three main components: (1) a VLM friction predictor which is fine-tuned by Low-Rank Adaption (LoRA)[Hu et al., 2022] that anticipates upcoming terrain friction from egocentric images, (2) a friction estimator that provides real-time friction perception, and (3) a friction-aware locomotion policy that combines proactive visual prediction with reactive proprioceptive estimation. Our main contributions are summarized as follows:

• We propose WoI, the first humanoid locomotion framework that jointly leverages both current and anticipated friction information, enabling proactive gait adaptation on slippery surfaces.
• We design a friction-awareness mechanism that combines proactive VLM-based prediction with reactive proprioceptive estimation, allowing the robot to adapt gait before contact and in real-time, thereby avoiding the overly conservative gaits caused by domain randomization or catastrophic failures from delayed reaction.
• We introduce a VLM friction predictor, which leverages the world knowledge of VLMs to achieve robust generalization across novel surfaces and lighting variations.

2 RelatedWork

2.1 Handling Friction by Domain Randomization

Recent years have witnessed remarkable progress in legged locomotion using reinforcement learning. However, most existing methods focus on traversing diverse terrain geometries and achieving agile skills such as running, jumping, and parkour and overlook terrain friction properties, instead relying on domain randomization[Tobin et al., 2017] to achieve robustness, both in humanoid robots [Radosavovic et al., 2024b; Radosavovic et al., 2024a; Singh et al., 2024; Cui et al., 2024; Yu et al., 2024; Long et al., 2025; Duan et al., 2024; Bang et al., 2024; Wang et al., 2025; Siekmann et al., 2021] and quadruped robots[Rudin et al., 2022; Miki et al., 2022; Tan et al., 2018; Choi et al., 2023; Kumar et al., 2021; Hoeller et al., 2024; Agarwal et al., 2023; Jenelten et al., 2024; He et al., 2025].

While domain randomization partly enables policies to handle friction variations and enhance robustness, it leads to inherently conservative gaits that sacrifice efficiency for stability, as the policy cannot distinguish between high and low friction conditions and must adopt a one-size-fits-all strategy. [He et al., 2025] achieves impressive performance on sparse foothold scenarios by introducing attention modules to process height maps, but focuses primarily on terrain geometry and assumes non-slippery surfaces. Similarly, [Radosavovic et al., 2024b] demonstrates real-world humanoid locomotion with causal transformer architecture, yet the training and deployment environments implicitly assume sufficient ground friction. [Long et al., 2025] proposes a perceptive internal model for humanoid locomotion over challenging terrain, but focusing on terrain elevation rather than friction properties.

2.2 Friction-Aware Locomotion

To address the limitations of domain randomization, some approaches estimate current friction from contact interaction[Lee et al., 2020; Gu et al., 2024; Ji et al., 2022; Jenelten et al., 2019], while other approaches tried visionbased methods to predict friction[Brandao et al., 2016; Chen et al., 2024; Peng et al., 2025].

Proprioceptive-based methods estimate friction through force and motion analysis during ground contact. [Gu et al., 2024] proposes a denoising world model that learns to predict terrain properties from proprioceptive history, enabling humanoid walking on snow and ice. [Ji et al., 2022] concurrently trains a state estimator with the control policy to infer ground properties. However, these methods can only estimate friction after contact occurs, which may result in falls when encountering sudden friction transitions.

Vision-based methods attempt to predict friction before contact by using visual cues. [Brandao et al., 2016] employs CNNs for material classification followed by friction database lookup, but this approach is constrained to predefined material categories and cannot generalize to novel surfaces. [Chen et al., 2024] proposes a self-supervised framework to predict terrain physical parameters from vision, yet requires extensive real-world interaction data for training. Recently, [Peng et al., 2025] explores VLMs with RAG for friction estimation in wheeled robots, but the RAG-based approach introduces retrieval latency [Arslan et al., 2024] that may limit real-time applicability in dynamic locomotion.

3 Method

As illustrated in Fig. 1, we present WoI, a friction-aware locomotion framework for humanoid robots. In contrast to approaches mentioned in Sec. 2, our key insight is to decompose friction awareness into two complementary components: anticipated visual prediction and current proprioceptive estimation, enabling proactive gait adaptation while maintaining real-time reactivity. (1) For proactive prediction, we fine-tune Qwen-2.5-VL-7B-Instruct by LoRA to directly regress friction coefficients from egocentric images. VLMs possess rich world knowledge about material properties, making our predictor inherently robust to lighting variations and capable of zero-shot generalization to novel surfaces without requiring extensive data collection across diverse environmental conditions. (2) For reactive estimation, we design a proprioceptionbased friction estimator within the policy network that infers current contact friction from joint states and forces, providing real-time correction when predictions are inaccurate. (3)We represent joint state history as a 2D "past-state-picture" and employ convolutional networks for temporal feature extraction, enabling the policy to capture motion patterns indicative of surface interactions. The locomotion policy takes the predicted and estimated friction as inputs along with proprioceptive observations. We first formulate the problem as a Markov Decision Process, then describe each component in detail.

3.1 ProblemFormulation

We formulate friction-aware humanoid locomotion as a Partially Observable Markov Decision Process (POMDP) defined by the tuple (S,O,A,P, r, γ), where S is the state space, O is the observation space, A is the action space, P : S ×A → S is the transition dynamics, r : S ×A → R is the reward function, and γ ∈ (0, 1) is the discount factor. The goal is to learn a policy πθ(at|ot) that maximizes the expected cumulative reward:

We train an actor-critic policy using Proximal Policy Optimization (PPO) [Schulman et al., 2017] with an asymmetric architecture [Pinto et al., 2018], where the actor receives only observations available at deployment, while the critic has access to privileged information during training.

The actor observation at time t is defined as:

where ct ∈ R3 denotes velocity commands, ωt ∈ R3 is base angular velocity, gt ∈ R3 is projected gravity, qt, ˙ qt ∈ RNj are joint positions and velocities, at-1 is the previous action, τt represents joint torques, ϕt = [sin(φt), cos(φt)] encodes gait phase, Ht ∈ RC×Nj×L is the joint state history, and µpred t ∈ [0, 1] is the VLM-predicted friction.

The critic additionally observes privileged information:

where vt is base linear velocity, ˙pfeet t and F contact t are feet velocities and contact forces, hfeet t is feet height, and µcontact t is the ground-truth friction at current contact. Notably, the actor sees predicted future friction µpred t while the critic observes actual contact friction µcontact t , enabling the policy to learn anticipatory gait adjustment.

The action at ∈ RNj specifies target joint positions, controlled by a PD controller. Reward function combines velocity tracking, style regularization, and friction-adaptive rewards (details provided in supplementary material).

3.2 VLM-based Friction Predictor

We leverage the world knowledge embedded in VisionLanguage Models, which have learned associations between visual appearances and physical properties (e.g., "rubber provides grip") from large-scale image-text pretraining.

Dataset Construction. We construct a terrain friction dataset containing 33261 images of 10 surface types by spliting online videos and label with the corresponding friction coefficient, including dry and rainy street, wooden and ceramic floors, icy, grassy, sandy, snowy, unpaved and marshy surfaces (details see supplementary material).

Model Architecture. We adopt Qwen2.5-VL-7BInstruct [Bai et al., 2025] as our base VLM and fine-tune it using Low-Rank Adaptation (LoRA) [Hu et al., 2022]. Given an egocentric RGB image It capturing the terrain ahead of the robot, the model predicts the friction coefficient µpred t :

where θbase represents the frozen pretrained weights and ∆θLoRA denotes the trainable low-rank adaptation matrices. LoRA decomposes the weight update as ∆W = BA, where B ∈ Rd×r and A ∈ Rr×k with rank r ≪ min(d, k), significantly reducing the number of trainable parameters while preserving the model's pretrained knowledge.

3.3 Friction-Aware Locomotion Policy

As illustrated Fig. 2, the locomotion policy receives the friction coefficient as an external input, allowing the policy to adapt its gait based on anticipated terrain properties.

Joint History as Past-State-Picture. To capture temporal patterns indicative of terrain interactions and slips, we maintain a history of joint states over the past L timesteps. We reshape it into a 2D image-like tensor:

where the channels C = 3 correspond to joint positions, velocities, and torques. This "past-state-picture" representation treats the degrees of freedom as the height dimension and time as the width dimension, enabling a 2D CNN to simultaneously capture temporal patterns hidden in history and cross-joint correlations through spatial convolutions.

Latent Feature Extraction. The past-state-picture is processed by a 2D CNN encoder to extract a compact latent representation:

The CNN consists of 2 convolutional layers with kernel size [3×3], followed by adaptive average pooling and ELU activations. The latent vector zH t encodes motion patterns that reflect terrain properties through foot-ground interactions.

Friction Estimator. In addition to the proactive VLMbased prediction µpred t , we incorporate a reactive friction estimator that infers the current contact friction from proprioceptive observations. The estimator takes proprioceptive inputs excluding velocity commands, as friction estimation should be independent of the commanded motion. The estimator uses an LSTM to capture temporal dynamics, followed by an MLP to regress the friction coefficient. During training, the estimator is supervised by the ground-truth contact friction µcontact t using MSE loss.

where oest t is the estimator input, µest t is the estimated friction coefficient, and Lest is the supervision loss.

To prevent gradient interference between the estimator and the policy, we detach µest t before feeding it to the action head.

4 Experiments

4.1 Experimental Setup

We conduct both simulation experiments and real-world deplyment both with the 29-DoF Unitree G1 humanoid robot. Simulation is performed in Genesis [Authors, 2024] and locomotion policy training is completed on NVIDIA A100, with 40GB of memory. During deployment, the policy is zero-shot transferred on NVIDIA A5000.

Evaluation Metrics

We evaluate the performance using the following metrics, where fi indicates whether the robot fell during trial i:

• Survival Rate. The percentage of trials where the robot does not fall, regardless of whether it reaches the target distance in time (20s):

• Success Rate. The percentage of trials where the robot traverses a target distance dtarget in time, where N is the total number of trials and di is the distance traveled in trial i:

.

• Stuck Rate. The percentage of trials where the robot fails to reach the target distance but does not fall:

.

• Slip Rate. The ratio of slip events to total contact steps during locomotion, where F is the set of feet, vcontact j,t is the tangential velocity of foot j at time t, vslip is the slip velocity threshold, and cj,t indicates foot-ground contact:

Baselines

To assess the effectiveness of WoI, we compare WoI against (1) PPE[Chen et al., 2024]; (2) FSL[Peng et al., 2025]; (3) DLSG[Jenelten et al., 2019]; (4) MRCNN[Brandao et al., 2016]; (5) LeggedGym [Rudin et al., 2022]; (6) DWL [Gu et al., 2024].

Ablations

To validate the contribution of each component, we evaluate the following ablation variants: (1) w/o VLM: eemoving the VLM-based friction predictor µpred t , relying solely on proprioceptive friction estimation; (2) w/o Estimator: removing the proprioceptive friction estimator µest t , using only VLM-predicted friction; (3) w/o Both: removing both friction inputs, equivalent to a friction-unaware baseline; (4) WoI1D CNN: replacing the 2D CNN history encoder with a 1D temporal convolution; (5) WoILSTM: replacing the 2D CNN history encoder with an LSTM network.

4.2 Simulation Results

Comparison with Baselines

Methods in Different Environments. Tab. 1 presents a comparison of methods in different environments. Unlike approaches that rely on proprioceptive estimation only (DLSG, DWL), vision-based prediction only (PPE, FSL, MRCNN), or domain randomization without explicit friction awareness (LeggedGym), WoI uniquely combines both reactive proprioceptive estimation and proactive VLM-based prediction, achieving both anticipatory gait adjustment and real-time correction. WoI covers a more extensive low-friction range (0.1) compared to LeggedGym (0.5) and DWL (0.2). Moreover, our method completes trials in 6.1 seconds on average, reducing trial time by approximately 90% compared to DLSG (69.8s) and MRCNN (60.0s).

Uniform Friction Surfaces. Tab. 2 presents that WoI achieves 100.0% survival rate and 100.0% success rate across all friction conditions. On extremely low friction (µ = 0.1), WoI improves survival rate by 29.4% and success rate by 100% compared to DWL, while reducing slip rate by 1.24%, demonstrating strong effectiveness and robustness on lowfriction surfaces. As shown in Fig. 3, WoI shows lower contact tangential velocity than DWL, indicating better slip control in extreme scenarios. At µ = 0.2, DWL achieves 93.3% survival rate, but has a slip rate of 1.89%, compared to 0.74% for WoI, indicating that our method avoids overly conservative behavior while maintaining robustness. At moderate-low friction (µ = 0.3), all methods achieve 100.0% survival rate, but WoI improves success rate by 7.6% over DWL's 92.4% and reduces slip rate by 4.91% from DWL's 5.55%. On moderate to high friction surfaces (µ = 0.5, 0.8, 1.0), DWL achieves 100.0% survival rate, but WoI further improves success rate by 1.2%, 0.6%, and 1.7% respectively, while reducing slip rate by 5.47%, 5.79%, and 5.35%.

For LeggedGym, which relies solely on domain randomization without explicit friction awareness, it achieves 100% survival rate (see Fig. 4(a)) and the lowest slip rate across all surfaces. However, even on high friction surfaces (µ = 0.8, 1.0), it exhibits 10.6% and 24.1% stuck rates respectively, while WoI has 0.0% stuck rate. On extremely low friction (µ = 0.1), LeggedGym reaches 100.0% stuck rate, while WoI still has a lower max tangential velocity of contact foot and similar min tangential velocity of contact foot, Ours DWL

LeggedGym

Ours DWL

LeggedGym

Figure 5: Foot Velocity Comparison on friction transition scenarios. The robot is tasked to walk with a ahead command of 1.0m/s with a initial area of 0.5m long with friction coefficient of 1.0, and data is collected over 1000 episodes with per episode of 1000 steps. The X axis stands for step count, Y axis stands for the tangential velocity of contact foot, which suggest the slip level.

indicating better control of slipping and efficiency.

These results demonstrate that WoI effectively handles the full spectrum of friction conditions, while baseline methods either adopt overly conservative gaits (LeggedGym) or fail to adequately handle low-friction surfaces (DWL).

Friction Transition Scenarios. Tab. 2 presents results on friction transition scenarios, including high-to-moderate (1.0 → 0.3) and high-to-extreme (1.0 → 0.1) transitions. WoI achieves the highest success rate across all methods and scenarios. In the extreme transition (1.0 → 0.1), although LeggedGym achieves 100.0% survival rate (See Fig. 4(b)), it exhibits a stuck rate of 23.9% and show very low tangential velocity of contact foot (See Fig. 5), indicating conservative gaits. DWL only achieves 47.3% survival rate, and obviously high tangential contact foot velocity, demonstrating that methods relying solely on reactive friction estimation have limited adaptability to friction transitions. In contrast, WoI improves success rate by 22.2% over LeggedGym and 97.5% over DWL, and shows low but suitable tangential velocity of contact foot, demonstrating strong robustness and traversability under varying friction conditions. In the 1.0 → 0.3 transition, WoI achieves 100.0% survival rate and success rate, improving success rate by 23.3% over LeggedGym and 8.5% over DWL, while reducing slip rate by 5.15% compared to DWL, demonstrating that WoI maintains excellent robustness, efficiency, and gait stability when transitioning from high to low friction surfaces. These results demonstrate that LeggedGym survives through conservative gaits but fails to make forward progress, DWL falls due to the lack of proactive friction anticipation, while only WoI achieves both high survival and traversability.

Ablation Studies

Friction Awareness Components. Tab. 3(a) presents the ablation on friction awareness components. All methods achieve 100.0% survival rate and success rate on uniform friction surfaces (µ = 0.3 and µ = 0.8) and moderate friction transitions (1.0 → 0.3), with w/o VLM and w/o Both achieving the lowest slip rates at µ = 0.3 and µ = 0.8. However, in extreme friction transitions (1.0 → 0.1), WoI achieves the highest success rate of 98.3% and the lowest slip rate of 1.44%. Compared to w/o VLM, WoI improves success rate by 4.8%, demonstrating the effectiveness and necessity of proactive friction prediction for friction transition scenarios. Compared to w/o Estimator, WoI improves success rate by 0.6% and reduces slip rate by 0.43%, confirming that the proprioceptive estimator contributes to improved performance in friction transition scenarios. The w/o Both variant achieves the highest survival rate of 99.9%, but also exhibits the highest stuck rate of 4.0%, resulting in 2.4% lower success rate than WoI, indicating relatively conservative gaits due to the complete lack of friction awareness. In summary, the VLM predictor enables proactive adaptation to friction changes, the proprioceptive estimator provides realtime refinement, and their combination achieves optimal performance across all scenarios.

History Encoder Architecture. Tab. 3(b) compares different history encoder architectures. All methods achieve 100.0% survival rate and success rate on uniform friction surfaces (µ = 0.3 and µ = 0.8) and moderate friction transitions (1.0 → 0.3). Since all variants retain the dual friction awareness mechanism, they exhibit 0.0% stuck rate across all scenarios. Additionally, WoI with LSTM encoder and WoI with

1D CNN encoder achieve the lowest slip rates at µ = 0.3 and µ = 0.8, with LSTM achieving the lowest slip rate in the 1.0 → 0.3 transition. However, in extreme friction transitions (1.0 → 0.1), WoI with 2D CNN encoder achieves the highest survival rate and success rate of 98.3% and the lowest slip rate of 1.44%, improving success rate by 3.7% over 1D CNN and 2.9% over LSTM. These results demonstrate that treating joint history as a 2D "past-state-picture" better captures the motion dynamics and contact information embedded in cross-joint correlations. In summary, while all encoder architectures perform comparably on moderate friction conditions, the 2D CNN encoder provides superior performance in challenging friction transitions by exploiting spatial correlations across joints.

4.3 Real-World Experiments

We deploy WoI on a real Unitree G1 humanoid robot without any fine-tuning. The VLM runs at 1 Hz, and the control loop runs at 50 Hz, consistent with simulation.

Results. As Fig. 6 illustrated, WoI successfully performed on surfaces varying from indoor to outdoor and with friction transition, demonstrating adaptive control across various scenarios. In the indoor wooden floor test (Fig. 6(a)), the right half of wood is watered to decrease friction and to create friction transition situation. WoI successfully traverse both the dry area and the watered area, validating the ability to handle varying friction surfaces in real-world. In normal indoor tile floor test (Fig. 6(b)), WoI walks with a natural,efficient gait. On icy surfaces, as Fig. 6(c) shown, robot encounters a slippage at initialization and recovers immediately and successfully navigates surfaces covered with snow and ice.

5 Conclusion

We presented WoI, a friction-aware locomotion framework that enables humanoid robots to walk robustly on surfaces with varying friction coefficients. Our approach combines three key components: a VLM-based friction predictor that leverages world knowledge to anticipate surface friction from egocentric images, a proprioceptive friction estimator that provides reactive correction during contact, and a 2D CNNbased history encoder that processes joint state history as a "past-state-picture" for effective temporal feature extraction. Experiments in simulation and real-world deployment demonstrate that WoI achieves higher success rate both on fixed low-friction surfaces and friction transition scenarios compared to different methods, while maintaining robust performance and effectiveness across diverse terrain conditions. We will explore policy work both in environments with changes both in physical properties and geometric shape in the future.

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant No. 62350710797).