Disturbance-Aware Hybrid Learning for Robust and Adaptive UAV Flight in Extreme Winds

1 Introduction

Quadrotor unmanned aerial vehicles (UAVs) have been increasingly deployed in a wide range of applications, including logistics delivery [Shao et al., 2025], surveillance [Gu et al., 2018], and search-and-rescue operations [Abdellatif et al., 2025]. These applications place stringent requirements on flight stability and control robustness, particularly under challenging environmental conditions [Gu et al., 2025]. Among various external disturbances, wind remains one of the most critical factors degrading autonomous flight performance in outdoor environments, where wind fields are inherently stochastic, time-varying, and difficult to model [Wang et al., 2025]. Due to their lightweight structure and underactuated dynamics, quadrotors are especially vulnerable to aerodynamic disturbances, where even moderate wind gusts can lead to significant tracking errors or loss of stability [O'Connell et al., 2022; Sun et al., 2025; Cioffi et al., 2025].

∗Corresponding author

Conventional approaches to wind disturbance rejection primarily rely on classical feedback control techniques, such as PID controllers with feedforward compensation [Liu et al., 2025b] or model predictive control (MPC) [Hanover et al., 2021; Pravitra et al., 2020]. While effective under nominal conditions, these methods exhibit inherent limitations when confronted with the nonlinear, stochastic, and spatially distributed nature of real-world wind fields. Accurate modeling of wind dynamics remains challenging, as wind speed, direction, and turbulence exhibit significant uncertainty and variability. Consequently, controllers based on simplified disturbance models often suffer from degraded performance in out-of-distribution scenarios [Huang et al., 2023; Kong et al., 2025]. Furthermore, uncertainty in aerodynamic parameters can necessitate frequent manual retuning, increasing computational overhead and limiting deployment on resource-constrained UAV platforms [Lee et al., 2025].

Recent advances in machine learning, particularly reinforcement learning (RL), have provided promising alternatives for UAV control under uncertainty [Liu et al., 2025a; Vaidya and Keshavan, 2025]. Deep RL methods can learn control policies directly from interaction data, enabling them to capture complex nonlinear dynamics without relying on explicit system models [Zhang et al., 2025a]. However, most existing RL-based UAV controllers implicitly treat wind disturbances as unstructured noise [Yang et al., 2025; Huang et al., 2023], without explicitly reasoning about their physical effects on flight dynamics. As a result, learned policies often exhibit limited generalization capability and may fail under wind conditions that differ from those encountered during training.

To address these limitations, we propose WA-TD3 ( WindAware Twin Delayed Deep Deterministic Policy Gradient), a novel control method that explicitly accounts for wind disturbances by inferring wind field characteristics from dynamics residuals and integrating this information into RL-based control, as illustrated in Fig. 1. The key idea is to leverage temporal deviations between predicted and observed UAV dynamics to construct a residual-driven wind perception model. By exploiting both current and historical state information, the proposed model captures latent wind-induced forces without requiring dedicated wind sensors. The inferred wind disturbance is then incorporated into a perception-augmented actor-critic architecture, where wind-aware state representations enable the policy to proactively compensate for wind effects rather than merely reacting to disturbances. Furthermore, we adopt a joint training paradigm that simultaneously optimizes wind perception and control performance within a closed-loop system. This end-to-end learning strategy improves robustness and generalization to unseen and complex wind conditions. Unlike prior approaches that treat wind as an unstructured disturbance, WA-TD3 implicitly learns wind dynamics and integrates them into decision-making, resulting in improved adaptability and control stability under severe wind disturbances.

Our main contributions are as follows:

• We propose a dynamics residual-driven wind perception mechanism that extracts wind field characteristics from temporal UAV state deviations, enabling real-time wind estimation without external sensors.
• We design a perception-augmented reinforcement learning framework that integrates estimated wind disturbances into the control policy, allowing proactive and wind-aware decision making.
• We demonstrate through extensive experiments that WA-TD3 consistently outperforms state-of-the-art methods across diverse wind conditions and trajectory tracking tasks, achieving over 62% improvement in tracking accuracy under strong wind disturbances.

2 RelatedWork

2.1 Traditional Wind Disturbance Control Approaches

Wind disturbance rejection for UAVs has traditionally relied on model-based and classical control. Early methods used PID controllers with feedforward compensation, modeling wind as external disturbances mitigated via observer-based estimation [Rosales et al., 2018]. These approaches work under mild conditions but assume simplified disturbances and require accurate system models and extensive manual tuning.

Adaptive control methods, such as L1 adaptive control combined with MPC [Pereida and Schoellig, 2018] or MPPI [Pravitra et al., 2020], improve robustness under model uncertainty and enable fast online adaptation. However, they often rely on restrictive assumptions, such as constant or slowly varying wind disturbances, and require continuous parameter adaptation, resulting in increased computational complexity and limited scalability to resource-constrained platforms.

Wind estimation techniques provide explicit disturbance awareness [Hollenbeck et al., 2018]. Direct sensors, such as pitot tubes or ultrasonic anemometers, measure wind but increase payload and system complexity. Sensor-free methods infer wind from discrepancies between commanded and observed motion using EKF/UKF [Ahmed and Woolsey, 2025]. More recently, learning-based estimators have been proposed to infer wind characteristics from flight data. While effective in structured settings, these approaches remain sensitive to modeling assumptions and often degrade under highly nonlinear or turbulent wind conditions [Sikkel et al., 2016; Ilyas et al., 2025].

Overall, traditional control and estimation methods exhibit limited robustness in complex, unstructured wind environments. Their reliance on explicit disturbance models, restrictive assumptions, and manual tuning motivates the exploration of more flexible learning-based approaches.

2.2 Learning-Based Control and Perception Integration

Learning-based control, particularly reinforcement learning (RL), has emerged as a promising approach for quadrotor flight under uncertainty. Early model-free RL methods demonstrate agility but treat wind as unstructured noise, relying on domain randomization and showing limited out-ofdistribution (OoD) generalization [Ma et al., 2023].

Hybrid learning-control frameworks aim to improve robustness. DATT [Huang et al., 2023] integrates deep RL with feedforward, feedback, and L1 adaptive control [Hovakimyan and Cao, 2010] for fast trajectory tracking, but is sensitive to hyperparameters and computationally heavy. NeuralIMC [Gao et al., 2025] combines a simplified dynamics predictor with RL using prediction error as an auxiliary signal, but this is only available during training. ProxFly [Zhang et al., 2025b] uses residual RL for aerodynamic compensation in proximity flight, yet residual corrections may induce highfrequency oscillations that reduce stability.

Recent methods focus on OoD generalization. OoDControl [Ye et al., 2024] uses noise-induced strategies to bound control errors but does not capture temporal wind structure. Domain randomization [Andres et al., 2025; Xue et al., 2023] improves robustness across conditions but incurs high computational cost and neglects structured, timevarying aerodynamic disturbances.

In contrast, WA-TD3 explicitly incorporates physically grounded acceleration residuals as real-time policy inputs, enabling sensor-free perception of wind-induced dynamics mismatches. By exploiting the temporal structure of these residuals within a perception-augmented RL framework, WA-TD3 allows policies to leverage structured disturbance information for robust performance and rapid adaptation under extreme, previously unseen wind conditions.

3 ProblemFormulation

We consider trajectory tracking for a quadrotor UAV under stochastic wind disturbances. The objective is to follow a reference trajectory while satisfying system and safety constraints. The problem is first posed as a constrained optimal control task and then reformulated as a Markov Decision Process (MDP) for reinforcement learning.

The quadrotor state is defined as: xt = [p⊤t , q⊤t , v⊤ t ,ω⊤t ]⊤ ∈ R13 , where pt, qt, vt, and ωt denote position, attitude quaternion, linear velocity, and angular velocity, respectively. The desired state xd,t = [p⊤d,t , q⊤d,t , v⊤ d,t ,ω⊤d,t ]⊤ ∈ R13 is generated from the reference trajectory.

Wind-disturbed dynamics are modeled as:

where f(·) represents nominal Newton-Euler dynamics, ut denotes motor inputs, and dt captures stochastic windinduced disturbances.

The tracking objective over horizon T is

We reformulate the task as an MDP (S, A, P, R, γ), where the state includes quadrotor observations and reference information, the action space is A = [-1, 1]4 , transitions follow wind-disturbed dynamics, and rewards penalize tracking errors and control effort. The learning objective is thus to find an optimal policy π∗ that maximizes the expected cumulative discounted reward:

where τ = (s0, a0, r0, s1, . . . ) denotes a trajectory generated under policy π, and the expectation is taken over stochastic transitions arising from wind disturbances.

4 ProposedMethod: WA-TD3

Robust quadrotor control in spatially varying winds is difficult for standard RL due to dynamics mismatch. Instead of treating wind as noise, WA-TD3 exploits inconsistencies between nominal dynamics and onboard measurements as disturbance cues. It integrates acceleration residuals into policy learning without requiring wind sensors. Wind-induced forces appear as systematic deviations from thrust-driven dynamics; explicitly modeling these residuals enables proactive compensation and improved generalization. As shown in Fig. 2, WA-TD3 comprises: (i) residual computation from sensing and actuation, (ii) a residual perception network using short histories, and (iii) a residual-conditioned TD3 policy.

4.1 External Force Residual as a Wind-Sensor-Free Disturbance Proxy

WA-TD3 is based on the observation that wind appears as an unmodeled external force, causing systematic deviations from nominal thrust-driven dynamics. By comparing the force inferred from onboard inertial measurements with that predicted by the nominal model, we obtain a physically meaningful residual. Given known thrust commands and vehicle parameters, this discrepancy serves as a proxy without explicit wind sensing for wind-induced disturbances.

The translational dynamics of a quadrotor are given by

where m is the vehicle mass, R(qt) maps body to world frame, ft ∈ R3 is the total rotor thrust, and Fwind,t ∈ R3 is the unknown wind force.

Rather than explicitly modeling Fwind,t, we recover its effect indirectly from inertial information. Specifically, let awt denote the quadrotor linear acceleration in the world frame, estimated from bias-corrected inertial measurements. The corresponding equivalent non-gravitational total force is given by

In parallel, a nominal force is predicted from the thrust model with known inputs and parameters:

where ˜ωi is rotor speed and CT the thrust coefficient. The total thrust Tt = ∑4i=1 fi yields the predicted world-frame force:

We define the external force residual as the difference between measured and nominal predicted forces:

This residual captures wind-induced and other unmodeled aerodynamic effects using only onboard inertial data and known control inputs, removing the need for wind sensors or explicit aerodynamic models. The residual thus serves as a physically meaningful disturbance proxy for the perception module described next.

4.2 Residual Dynamics Learning via Online Supervision

The external force residual rf,t provides a physically grounded representation of unmodeled aerodynamic disturbances. To capture temporally correlated wind effects, we employ a residual dynamics network that predicts rf,t from short-horizon onboard state histories. The network maps a history of k system states

to a 3D residual force estimate. This temporal input allows the network to learn structured disturbance patterns without explicit wind sensing or aerodynamic models.

The analytically computed residual rf from Eq. (8) serves as a self-supervised target. Network parameters are optimized via

Training samples are collected online in a dedicated residual replay buffer, decoupled from the policy buffer, improving data efficiency and stabilizing learning.

To handle non-stationary residuals and sparse wind excitation, we adopt a two-phase strategy that separates exploration, residual learning, and deployment. In phase I (Exploration & Bootstrapping): TD3 is trained under randomized winds, while the residual network is updated online from the buffer every 5 steps; In phase II (Residual Refinement): the actor and critics are frozen; the residual network is refined offline for 200 epochs to improve estimation accuracy. The trained network predicts residuals online, which are incorporated into translational acceleration:

Conditioning on short-horizon histories, the network encodes a compact latent representation of external disturbances-magnitude, direction, and temporal evolution-derived solely from onboard measurements. This residual representation provides a physically interpretable interface for high-level policy learning, enabling robust control under unseen wind conditions.

4.3 Residual-Conditioned Policy Learning via WA-TD3

Using the learned residual dynamics, we integrate the predicted external force into the reinforcement learning pipeline for adaptive quadrotor control under wind disturbances. TD3 serves as the backbone, extended with residual-conditioned state representations.

At each timestep, the policy observes the quadrotor state augmented with the predicted residual:

This provides explicit disturbance information for proactive compensation.

The actor πθ(˜st) outputs normalized control actions corresponding to total thrust and desired body angular rates, while two critics Qϕ1 , Qϕ2 evaluate residual-augmented states, mitigating overestimation bias. Actions at = [az, ωx, ωy, ωz] are mapped to thrust and body-rate references, tracked by a deterministic low-level controller to ensure stability and actuator compliance. This separation allows high-level policies to focus on disturbance-aware decision making.

The reward function encourages precise trajectory tracking by penalizing position and velocity tracking errors:

The learning procedure follows the standard TD3 algorithm with delayed policy updates and target smoothing. Given a transition (˜st, at, rt, ˜st+1), the target Q-value is computed as

where ϵ is clipped Gaussian noise for target policy smoothing. The critics are updated by minimizing the Bellman error, while the actor is updated less frequently by maximizing the expected Q-value estimated by the first critic. Incorporating the residual-augmented state ˜st ensures that learned policies proactively compensate for wind-induced disturbances.

5 Experiments and Results

We evaluate the robustness and generalization of WA-TD3 under both nominal and severe wind disturbances. Extensive experiments are conducted in high-fidelity simulation, with a particular focus on trajectory tracking accuracy under out-ofdistribution (OoD) wind conditions. WA-TD3 is systematically compared against six representative baselines spanning classical control, model-free reinforcement learning, hybrid learning-control frameworks, and disturbance-aware methods, providing a comprehensive assessment across diverse control paradigms.

Specifically, PID [Lopez-Sanchez and Moreno-Valenzuela, 2023] serves as traditional model-based controllers relying on manual tuning and linearized dynamics. TD3 [Fujimoto et al., 2018] is included as a model-free RL baseline without explicit disturbance modeling. Neural-Fly [O'Connell et al., 2022], linear method (Linear) and OoD-Control [Ye et al., 2024], and OMAC [Shi et al., 2021] represent recent learning-based and adaptive control approaches with varying levels of disturbance awareness and online adaptation. To isolate the effect of model expressiveness, we also include a lightweight linear residual estimator that maps short-horizon state history to external force estimates.

All methods are trained and evaluated under identical state observations, actuator constraints, and in-distribution wind conditions, ensuring fair and controlled comparisons.

5.1 Experimental Setup

All experiments are conducted in WindSim, our in-house quadrotor simulation environment with an aerodynamic wind-disturbance model. Wind is modeled as a piecewiseconstant 3D velocity vector updated every 2 s. During training, each Cartesian component is generated by sampling a nonnegative magnitude from a Gamma distribution and assigning it an independent random sign:

where st,i ∼ Unif{-1,+1}, i ∈ {x, y, z}, e is the episode index, and βe = 0.5((e mod 3) + 1). Thus, βe ∈ {0.5, 1.0, 1.5}, corresponding to mean component-wise wind magnitudes of 0.5, 1.0, and 1.5 m/s. During testing, each component is independently sampled as

where Wv denotes the prescribed test wind level. This setup enables evaluation of out-of-distribution generalization under distribution-shifted wind conditions.

Implementation Details and Hyperparameters. All methods are implemented in Python using PyTorch 1.13 and trained on a single NVIDIA RTX 3090 Ti GPU. Both the TD3 actor-critic networks and the residual estimator are implemented as multi-layer perceptrons with ReLU activations.

Phase I performs joint policy learning and online residual estimation under stochastic wind sampling, promoting coordinated adaptation of control and disturbance perception. Phase II freezes the policy and refines the residual estimator offline to enhance disturbance prediction accuracy. The history length is set to k = 1, corresponding to a 0.01 s temporal window at 100 Hz control frequency. Key hyperparameters are summarized in Table 1.

DynamicsModeling. The quadrotor dynamics follow standard Newton-Euler equations:

where pt, vt ∈ R3 are the position and linear velocity, qt ∈ R4 is the attitude quaternion, and ωt ∈ R3 is the angular velocity. The matrices R(qt) and J ∈ R3×3 denote the rotation matrix from body to world frame and the inertia matrix, respectively. Wind-induced effects are injected through Fwind,t and τwind,t. Motor dynamics are approximated using a first-order system.

Evaluation Metrics. Tracking performance is quantified using the translational position error, emphasizing disturbance rejection under wind perturbations. The primary metric is the Average Control Error (ACE):

where pt and pd,t denote the actual and reference positions at timestep t, respectively, and T is the total trajectory length. Lower ACE indicates better tracking accuracy and robustness against disturbances.

5.2 Overall Performance Comparison

Table 2 summarizes the ACE across four representative trajectories under increasing wind intensities. All learningbased methods are trained under Gamma-distributed wind and evaluated on unseen uniform headwinds, forming a strict OoD generalization scenario.

Across all trajectories and wind conditions, WA-TD3 consistently achieves the lowest ACE in all OoD test cases. On average, it reduces ACE by 62.4% compared to the strongest baseline (OoD-Control) and by 71.8% relative to PID. Under extreme Gale conditions (12 m/s), WA-TD3 maintains stable tracking with ACE values of 0.05-0.06 m, whereas PID and OoD-Control degrade to 0.31-0.35 m and 0.14-0.18 m, respectively. Moreover, WA-TD3 exhibits consistently lower variance, highlighting stable and repeatable behavior across diverse tasks.

Qualitative trajectory results in Fig. 3 further corroborate these findings: baseline methods suffer from drift and trajectory distortion under strong wind, whereas WA-TD3 closely follows reference paths with smooth and stable motion, demonstrating effective real-time wind compensation without the need for dedicated wind sensors.

5.3 Ablation Studies

We perform ablation studies to evaluate the contributions of two key design choices in WA-TD3: (i) the residual-based wind compensation, and (ii) the length of temporal state history used for residual estimation. These experiments quantify how each component affects trajectory tracking accuracy and robustness under varying wind conditions.

Effect of Wind Compensation Residual. We evaluate the impact of the proposed residual ˆrf by comparing WA-TD3 against vanilla TD3 across four trajectories (Hover, Sinforward, Figure-8, Spiral-up) under wind velocities from 0 to 12 m/s. Performance is quantified using ACE with standard deviation over ten random seeds.

As shown in Fig. 4, WA-TD3 consistently outperforms TD3. At 0 m/s, both methods achieve low ACE (TD3: ∼0.08 m, WA-TD3: ∼0.02 m). Under stronger winds (12 m/s), TD3's ACE rises sharply to 0.19-0.21 m, while WA-TD3 maintains 0.05-0.06 m, representing a ∼70% reduction. The advantage is particularly pronounced in complex trajectories (e.g., Spiral-up), where TD3 suffers from unmodeled drift, whereas WA-TD3 leverages ˆrf as a predictive disturbance proxy, enabling proactive compensation and robust OoD performance without additional sensors or tuning.

Moreover, WA-TD3 exhibits lower variance (std < 0.010 m vs. TD3's up to 0.036 m at 12 m/s), indicating more stable and predictable behavior. This improvement stems from explicitly disentangling disturbance dynamics from the control policy, avoiding overcompensation and high-frequency jitter common in implicit adaptation schemes.

These results demonstrate that incorporating explicit, sensor-free disturbance observables is critical for fast and reliable adaptation under extreme OoD wind conditions. By encoding unmodeled aerodynamics as a learnable residual, WATD3 achieves superior tracking precision, smoother control, and enhanced robustness without relying on online estimation, external sensors, or environment-specific tuning.

Impact of Temporal State History Length. We investigate the effect of state history length k on the residual-based wind compensation by ablating k ∈ {1, 2, 3, 4, 5} on the Sinforward trajectory under wind speeds of 0, 4, 8, and 12 m/s. The residual ˆrf is computed as the difference between the predicted and nominal external forces over the selected horizon, while ACE is evaluated as the mean position tracking error.

As shown in Fig. 5, the state history length k = 1 performs best under the tested wind models and control frequency, reaching 0.062 m at 12 m/s-15-30% lower than longer horizons. At low wind speeds (0-4 m/s), ACE is similar across k (0.038-0.065 m), indicating that a single-step residual suffices for effective disturbance observability. For higher wind intensities, longer histories (k ≥ 3) introduce delayed responses and noise accumulation, increasing ACE (e.g., k = 5: 0.088 m at 12 m/s) due to over-smoothing of transient gusts.

The optimal state history length k = 1 matches the control frequency and the temporal correlation of Dryden gusts: a single-step residual captures instantaneous model mismatch, enabling precise, low-latency compensation. Longer histories, though richer, dilute acute disturbances and increase sensitivity to velocity estimation errors. Overall, it demonstrates that the optimal state history length provides a lightweight, highly effective disturbance observable, maximizing robustness and responsiveness in severe, time-varying wind conditions without additional computational overhead.

6 Conclusion

We presented WA-TD3, a RL framework that incorporates wind disturbance awareness for robust quadrotor control under strong and out-of-distribution wind conditions. By leveraging dynamics residuals from onboard state evolution, WATD3 enables implicit wind perception without dedicated sensors or precise aerodynamic models. Integrating this residual into a perception-augmented TD3 policy allows proactive compensation of wind effects rather than treating them as unstructured noise. Experiments show that WA-TD3 greatly improves trajectory tracking accuracy and stability under varied wind strengths and novel disturbances, proving the value of residual-based wind-aware RL for adaptive UAV control. Future work will explore more turbulent conditions and multiagent systems to advance wind-aware UAV operations.

Acknowledgments

This paper is supported by the National Key Research and Development Program for Young Scientists (No. 2025YFB4711500), National Natural Science Foundation of China (No. 42474027, No. 62403085), Xiaomi Young Talents Program and Graduate Research and Innovation Foundation of Chongqing, China (No. CYB240066).