Beyond the Clouds: Reliable and Cloud-Aware Spatiotemporal Fusion via Adversarial Regression Wavelets

1 Introduction

Acquiring satellite imagery with both high spatial resolution and high temporal frequency is critical for monitoring dynamic Earth surface processes [Chamin´e et al., 2021]. Unfortunately, due to hardware constraints, existing sensors invariably suffer from a trade-off between spatial detail and revisit cycles [Xiao et al., 2023]. Spatiotemporal fusion (STF) mitigates this limitation by blending frequent coarseresolution data with sparse high-resolution observations [Li et al., 2025]. This technique provides synthetic time series that bridge the resolution gap, enabling precise tracking of rapid environmental changes such as crop phenology and urbanization [Guo and Shi, 2023].

∗Corresponding author.

As illustrated in Figure 1 (a), existing STF methods typically rely on an idealized clear-sky assumption and face two fundamental difficulties: spatial differences between heterogeneous sensors and temporal changes across dates. However, when reference images are contaminated by clouds, these models lack the necessary spatial guidance, which often leads to severe artifacts or blurred outputs. This problem is further intensified in the common but unexplored scenario where both high- and low-resolution references are simultaneously obscured.

Establishing a robust fusion framework under these compromised conditions is complicated by three further challenges (Figure 1 (b)). First, thick clouds and shadows cause significant spatial information loss by creating extensive data gaps that hinder the recovery of coherent textures. Second, spectral-structural mismatch arises when the inherent correlation between high- and low-resolution observations is disturbed by cloud pollution, hindering the unified spectral-spatial mapping. Third, temporal change confusion occurs when separating genuine land-cover variations from cloud artifacts, leading to potential semantic errors during feature propagation.

To bridge this critical gap, we present CLAW-GAN, a novel Cloud-Aware Wavelet Generative Adversarial Network designed for robust STF under atmospheric interference. The model first decouples spectral backgrounds from structural details via Regression Wavelet Analysis (RWA) [Amrani et al., 2016], establishing a unique pipeline where high-frequency (HF) and low-frequency (LF) components are processed independently. Guided by cloud masks and a reliability map from the Temporal Consistency Evaluator (TCE), the Frequency-Separated Fusion (FSF) model adaptively integrates decoupled feature streams by prioritizing stable landcover information. Furthermore, a multi-scale discriminator with the stationary wavelet transform (SWT)[Nason and Silverman, 1995] is used to enforce textural and spectral fidelity across scales. We also introduce the Global Cloud-shrouded Agricultural Regions (GCAR) benchmark to provide a largescale evaluation platform with authentic cloud conditions. By combining wavelet-domain processing with adversarial training, CLAW-GAN delivers fusion images that are both temporally consistent and geographically faithful. The primary contributions of this work are summarized as follows:

• We formalize the cloud-constrained STF task and propose the cloud-reliable CLAW-GAN framework. To support this research, we establish GCAR, a large-scale benchmark built with real cloud-contaminated imagery from worldwide agricultural zones.
• A Wavelet-Pyramid Generator is designed to decouple LF backgrounds and HF details via RWA. This approach, integrated with the FSF module, effectively minimizes cross-frequency interference and recovers landcover features with high fidelity, even under significant atmospheric interference.
• We develop a multi-scale discriminator that operates in the wavelet domain using SWT. It enforces consistency across directional HF subbands, thereby reducing synthetic artifacts and promoting geographically realistic outputs.
• Comprehensive experiments on the proposed GCAR benchmark and the cloud-simulated Daxing dataset demonstrate that CLAW-GAN achieves state-of-the-art (SOTA) performance and exhibits strong robustness across varying cloud coverage levels.

2 RelatedWork

2.1 Spatiotemporal Fusion for Remote Sensing

STF methods can be broadly categorized into five classes: weight function-based, unmixing-based, Bayesian-based, hybrid, and deep learning-based [Zhu et al., 2018].

Weight-based methods like STARFM [Gao et al., 2006] and ESTARFM [Zhu et al., 2010] propagate reflectance changes by calculating spectral-temporal similarity weights. Alternatively, unmixing approaches such as MMT [Zhukov et al., 1999] and OBSUM [Guo et al., 2024] utilize linear decomposition, which can cause spectral distortions in heterogeneous regions. While Bayesian frameworks offer a probabilistic foundation for fusion, their high computational cost limits large-scale deployment [Li et al., 2013]. Hybrid strategies, notably FSDAF [Zhu et al., 2016], combine multiple techniques but often rely on empirical parameters that restrict their generalizability.

Recent advances in STF have been significantly driven by deep learning. CNN-based models like EDCSTFN [Tan et al., 2019] and ECPW [Zhang et al., 2024] effectively extract hierarchical spatial features for detail enhancement. Transformerbased frameworks such as SwinSTFM [Chen et al., 2022] utilize self-attention mechanisms to model long-range dependencies and capture global contextual information. Diffusionbased frameworks reconstruct target images through an iterative denoising process that progressively transforms random noise into structured outputs [Huang et al., 2024]. GANbased methods, such as GAN-STFM [Tan et al., 2022] and OPGAN [Song et al., 2022], employ adversarial training to synthesize realistic textures and enhance spatial details.

Despite these advances, a key limitation of current STF methods is their reliance on the idealized cloud-free reference assumption. When input images are cloud-corrupted, traditional approaches lack reliable spatial guidance, which leads to inaccurate change propagation. While cloud-removal can be employed as preprocessing, such two-stage pipelines often introduce error accumulation and spectral shifts, ultimately degrading final fusion quality. Therefore, developing a unified framework that jointly handles cloud corruption and spatiotemporal reconstruction remains a significant research gap.

2.2 Wavelet Transforms

Wavelet Transform (WT) is a powerful tool for decomposing signals into localized spatial and frequency components. In image processing, the Discrete Wavelet Transform (DWT) [Mallat, 1989] separates data into frequency subbands through decimated filtering. In contrast, the SWT [Nason and Silverman, 1995] achieves translation invariance by omitting the down-sampling step, thereby better preserving edge information and mitigating artifacts. Furthermore, RWA [Amrani et al., 2016] integrates wavelet decomposition with regression modeling to enhance feature separation and textural clarity. These multi-scale capabilities are particularly suited for isolating and synthesizing features across heterogeneous resolutions [Ji et al., 2023], making wavelet theory a robust foundation for spatiotemporal fusion.

Wavelet-based approaches have demonstrated strong utility across multiple remote sensing tasks. WTSO [Yang et al., 2019] effectively captures multi-resolution features in hyperspectral image classification, while IRPruneDet [Zhang et al., 2025] enhances infrared target detection by improve robustness in identifying subtle features. DTWSR [Du et al., 2025] efficiently process spectral-spatial information for image super-resolution while preserving high-frequency details. Cloud removal [Zi et al., 2023] utilizes wavelet-based frameworks to effectively separate cloud artifacts from surface features across frequency bands. In STF, ECPW [Zhang et al., 2024] introduced a CNN-based network with DWT and employed a wavelet-aware loss to recover details.

3 Method

3.1 ProblemFormulation

Conventional STF assumes cloud-free high-resolution (HR) and low-resolution (LR) reference images, denoted as XH1

and XL1, respectively. As shown in Figure 1(a), the goal under this idealized setting is to generate a cloud-free HR image XH2 at the target time by fusing spatial details from the references with temporal changes from the LR observation XL2, where all images share the same dimensions RC×H×W .

In practice, however, clouds and other atmospheric contaminants frequently corrupt reference imagery. This work addresses the more realistic and challenging scenario of cloud-constrained STF, where both XH1 and XL1 are simultaneously contaminated (Figure 1(b)). We denote binary cloud masks MH1 and ML1, where values of 0 and 1 indicate cloudy and clear pixels, respectively. The reconstruction task is then formulated as learning a mapping function F that estimates the clean target from the corrupted observations:

3.2 Overall Framework of CLAW-GAN

CLAW-GAN follows a hierarchical pipeline built upon wavelet decoupling. As shown in Figure 2 (a), The WaveletPyramid Generator G first employs RWA to decompose each input source into LF spectral backgrounds and HF structural details. These components are then processed separately through parallel Swin Transformer (SwinT) encoders to capture multi-scale spatial contexts. Subsequently, separate gated units adaptively aggregate these streams, utilizing cloud masks and reliability maps to fill information gaps. Finally, a SwinT-based pyramidal decoder synthesizes the fused representations into a cloud-free estimate ˆXH2.

To ensure textural realism, a Multi-scale Discriminator D evaluates the results in the wavelet domain. Using the SWT, the discriminator extracts directional HF subbands from ˆXH2,XH2 and XL2, respectively. By evaluating these concatenated features across multiple scales, the discriminator identifies spectral artifacts and directional inconsistencies that are often overlooked in the image domain. This frequency-aware supervision guides the model toward generating geographically plausible content, ensuring high-fidelity reconstruction even under significant atmospheric interference.

3.3 Wavelet-Pyramid Generator

Wavelet-based Spectral Decomposition

Atmospheric interference distorts both spectral irradiance and spatial details, inducing artifacts during direct pixel-domain fusion. To mitigate this, we map multi-source inputs into the wavelet domain using RWA. Unlike conventional transforms, RWA exploits inter-band correlations via regressionbased prediction to achieve a more compact and physically consistent representation. For a spectral vector v at any pixel, RWA applies a lifting operation that splits it into odd (vo) and even (ve) samples:

where h captures spectral differences and l represents the local average. The operator ⌊·⌋ denotes the floor function. A regression model then estimates the HF component from its LF counterpart to yield a prediction ˆh. The final decomposed features for each source s ∈ {H1, L1, L2} are:

This mapping effectively isolates global spectral trends (FLF s ) from fine textures (FHF s ), providing a stable foundation for robust feature extraction under varying atmospheric conditions.

Hierarchical Spatial Modeling via SwinT Blocks

CLAW-GAN adopts a hierarchical SwinT backbone to capture non-local spatial dependencies efficiently. To handle high-resolution fusion with manageable computation, the SwinT block divides the feature maps into non-overlapping local windows of size M × M. Inside each window, multi-head self-attention (W-MSA) is computed as:

where Q,K, V ∈ RM2×d denote the query, key, and value matrices, d is the query dimension, and B is a learnable relative position bias. To enable information exchange across neighboring regions, a shifted windowing (SW-MSA) mechanism is utilized in alternating layers.

Through successive stages of resolution adjustment and channel expansion, this hierarchical design enables the generator to model geographic semantics across scales, effectively recovering missing content in cloud-occluded regions.

Temporal Consistency Evaluator

To manage land-cover variations between acquisition dates, the TCE processes the stacked low-resolution pair [XL1, XL2] through two 3 × 3 convolutional layers. With intermediate ReLU and final Sigmoid activations, the module outputs a single-channel reliability map R ∈ [0, 1]H×W , which quantifies surface stability at the original resolution. R highlights temporally consistent regions, thereby preventing the transfer of outdated or irrelevant features from the reference to the target reconstruction.

Frequency-Separated Fusion

The linear relationships required for a standard inverse RWA transformation are disrupted during multi-source fusion, making direct frequency-to-image recovery no longer feasible. We therefore introduce the FSF module to aggregate frequency components independently, preventing crossfrequency interference (Figure 2 (c)).

At each pyramid scale i, a mask-gated networkMgenerates weights Wi from the stacked cloud masks and reliability map:

For each frequency ϕ ∈ {LF, HF}, the fused features are calculated as a weighted sum of the three input streams:

This design dynamically rebalances information sources based on cloud coverage and temporal consistency. It prioritizes high-resolution textures fromXH1 in clear, stable regions while relying more on the temporal dynamics from XL2 when reference pixels are occluded or altered. This selective aggregation ensures that the fused features maintain both structural sharpness and spectral accuracy.

3.4 Multi-scale Discriminator inWavelet Domain

To ensure the fidelity of synthesized high-frequency details, we implement a multi-scale discriminator D that operates directly in the wavelet domain, as illustrated in Figure 2(b). For each multispectral channel, the input image is decomposed via the SWT into one LF approximation subband (LL) and three directional HF detail subbands (LH, HL, and HH). By maintaining the original spatial resolution, SWT ensures that the HF subbands (carrying horizontal, vertical, and diagonal textures) remain precisely aligned with its corresponding spatial location. This design is motivated by the observation that fine-grained textures and potential artifacts are most discriminative in the HF domain, whereas the spectral background in the LL subband is already well-constrained by the generator's reconstruction losses.

D adopts a hierarchical architecture that processes these wavelet subbands at three spatial scales: the original resolution, 1/2×, and 1/4×. Each sub-discriminator consists of residual blocks with spectral normalization to stabilize adversarial training. By verifying the consistency of directional HF subbands across multiple scales, D guides G to produce outputs with sharp edges and realistic spatial semantics.

3.5 Loss Functions

To achieve stable reconstruction under atmospheric interference, the CLAW-GAN framework is optimized within the Least Squares GAN (LSGAN) [Mao et al., 2017] paradigm, which provides stable gradient flow and mitigates the vanishing gradient problem. While D follows the standard LSGAN update, G is trained by minimizing a composite objective function that balances adversarial realism with reconstruction

where LMSE is the mean squared error that reduces overall pixel-wise discrepancies; LSSIM is a multi-scale structural similarity loss that preserves perceptual details; Lcos enforces spectral alignment across all channels via cosine similarity; and Ladv denotes the adversarial loss from the LSGAN formulation. By jointly optimizing pixel-level accuracy, structural consistency, and spectral alignment, CLAW-GAN effectively corrects cloud-induced artifacts while maintaining high spectral fidelity.

4 Experiment

4.1 Datasets

GCAR

As depicted in Figure 3, the GCAR dataset facilitates STF resilience assessment across 12 diverse biomes spanning six continents, such as the Amazon rainforest, Nile Delta, Siberian Taiga, and U.S. Corn Belt. It utilizes Landsat-8/9 OLI Surface Reflectance (30 m) as HR data and MODIS MOD09A1 (500 m) as LR data. Covering February 2022 to March 2025, the dataset features authentic reference-time cloud cover between 5% and 45%. In our experiments, the Pampas (Argentina) and Indo-Gangetic Plain (India) are reserved for testing, while the remaining regions are used for training. Further details on the dataset construction pipeline are provided in Appendix A.

Daxing

The Daxing dataset [Li et al., 2020] is a well-established benchmark containing 29 MODIS-Landsat pairs (2013-2019) across urban, cropland, and mountainous landscapes in Beijing, China. To simulate cloud-constrained scenarios, we superimposed 30 real cloud masks (5-45% coverage) extracted from contemporary Landsat-9 imagery onto the original cloud-free references. For each task, the reference pair was selected based on temporal proximity to the target date. Five image pairs from February 2016 to November 2017 are used for testing, with the rest for training, ensuring model evaluation under realistic atmospheric interference.

4.2 Settings

Implementation Details

All experiments used PyTorch with an NVIDIA RTX 3090 GPU. Images were processed as 256×256 patches with a stride of 128. Training used the AdamW optimizer (initial learning rate 2 × 10-4 ) for 200 epochs with a batch size of 10. For the loss function, α = β = 1 and γ = 0.01.

Evaluation Metric

Fusion quality is assessed using five metrics, including Root Mean Square Error (RMSE), Structural Similarity Index (SSIM) [Wang et al., 2004], Correlation Coefficient (CC) [Kaneko et al., 2003], Spectral Angle Mapper (SAM) [Yuhas et al., 1992], and Erreur Relative Globale Adimensionnelle de Synth`ese (ERGAS) [Khan et al., 2009]. Lower RMSE, SAM, and ERGAS values and higher SSIM and CC values indicate better performance.

Comparisons methods

We compare CLAW-GAN with seven representative STF methods, including traditional approaches (STARFM [Gao et al., 2006], OBSUM [Guo et al., 2024], FSDAF [Zhu et al., 2016]), a wavelet-integrated CNN (ECPW [Zhang et al., 2024], GAN-based architectures (OPGAN [Song et al., 2022], GANSTFM [Tan et al., 2022]) and a Transformerbased method (SwinSTFM [Chen et al., 2022]). All methods were trained and tested under identical settings for a fair comparison.

4.3 Qualitative Comparisons

Visual performance is evaluated on the GCAR benchmark and the Daxing dataset with 15% simulated cloud coverage, the test image pairs are illustrated in Figure 4. Figure 5 provides detailed comparisons including full-scene fused images, enlarged local patches, and their corresponding Average Absolute Difference (AAD) maps. For a more comprehensive assessment, additional visualizations, full-image AADmaps, and scatter plots are provided in the Appendix B.

Traditional STF methods struggle to reconstruct discernible ground features, typically resulting in severe blurring or spectral distortions in regions affected by clouds. While deep learning approaches improve sharpness, they frequently introduce inconsistent textures and artifacts along cloud boundaries. In contrast, CLAW-GAN generates coherent imagery with high fidelity, effectively preserving distinct field boundaries and authentic urban textures. Substantially lower AAD errors further validate the robustness of our model, showing that it maintains accurate inference of realistic land-cover semantics even under heavy cloud cover.

4.4 Quantitative Comparisons

Our method achieves SOTA on both the GCAR and Daxing datasets across all five evaluation metrics (Table 1). The detailed band-wise analysis is available in Appendix B. Under cloud-constrained conditions, the performance of traditional STF methods declines significantly. This limitation is primarily attributed to their reliance on linear mapping or weighted averaging assumptions, which are insufficient to address the complex nonlinear spectral-spatial distortions and extensive data gaps caused by cloud cover. In contrast, CLAW-GAN demonstrates superior robustness in both radiometric reconstruction and structural preservation despite substantial information loss.

4.5 Ablation Study

Ablation studies are conducted on the GCAR and Daxing datasets to comprehensively examine the impact of key components within the CLAW-GAN framework, focusing on frequency-domain representation and architectural design.

Effectiveness of Wavelet-Domain Processing

We first evaluate the necessity of frequency-domain processing by comparing CLAW-GAN against several spatial and alternative wavelet baselines. In the generator, the RWA-based decomposition is replaced by a direct spatial-domain baseline and the standard DWT. This comparison isolates the advantage of RWA in decoupling spectral and spatial information. Similarly, the contribution of the SWT in the discriminator is assessed by substituting it with DWT or standard spatial inputs. The omission or replacement of these components results in a consistent performance decline across all metrics (Table 2). These results confirm that regression-based decorrelation in RWA and shift-invariant supervision from SWT are both critical for maintaining reconstruction fidelity under cloud-constrained condition.

Contribution of Architectural Components

To assess the impact of individual architectural modules, we perform a series of ablation studys on the core components of the framework. Specifically, we remove TCE to examine its role in handling land-cover variations. FSF is replaced with a direct feature concatenation operation to evaluate its effectiveness in frequency-separated processing. Furthermore, the pyramidal hierarchy is tested by restricting fusion to a uniform resolution, while the multi-scale design of the discriminator is compared against a fixed-scale variant.

Removing or simplifying any of these components leads to a consistent decline across all evaluation metrics (Table 2). These findings confirm that the combined design, which integrates change awareness from the TCE, decoupled processing in FSF, and a hierarchical multi scale architecture, is essential for preserving spectral and structural integrity when reference data is obscured.

4.6 Case Study

To evaluate the performance boundaries of CLAW-GAN, we conducted a case study on the Daxing dataset with simulated cloud levels of 15%, 30%, and 45%. The corresponding cloud masks are shown in Figure 6. Quantitative results for deep learning methods are summarized in Figure 7, with bar charts for RMSE and line plots for SSIM. As expected, all models degrade with increasing occlusion, yet CLAW-GAN maintains the highest stability, with its metrics declining at a markedly slower rate than competing approaches.

Band-wise analysis (detailed in Appendix C) indicates that shorter wavelengths are more susceptible under heavy cloud cover. This sensitivity likely arises from the higher spatial complexity and subtler spectral variations in these bands, which complicate the recovery of fully obscured pixels. CLAW-GAN delivers consistently high accuracy across the spectrum and outperforms other models in overall averages, despite comparable results with SwinSTFM in a few isolated parameters. Visual AAD maps and scatter plots further confirm these findings, demonstrating that CLAW-GAN yields the smallest deviations from ground truth across diverse land-cover types. Together, these results underscore the reliable of our framework for continuous time-series monitoring, ensuring high-fidelity data reconstruction even under extreme atmospheric conditions.

5 Conclusion

This paper introduces CLAW-GAN, a wavelet-based adversarial framework designed for robust STF under cloud-contaminated conditions. The Wavelet-Pyramid Generator utilizes RWA to separate global spectral trends from fine-scale textures. The resulting components are independently fused by the FSF module, which is guided by a TCE to adaptively handle land-cover changes. A multi-scale discriminator operating in the SWT domain further enforces the geographical authenticity of reconstructed high-frequency details. Comprehensive evaluations on the proposed GCAR benchmark and the simulated Daxing dataset confirm that CLAW-GAN achieves SOTA accuracy and structural preservation, demonstrating strong robustness even under 45% cloud coverage. Future work will investigate mask-free and uncertainty-aware modeling to generalize across diverse atmospheric scenarios and application domains. This work establishes a reliable foundation for continuous Earth observation in suboptimal imaging conditions.

Figure 7: Performance trends of deep learning methods across varied cloud cover levels. Bar charts and line plots represent the mean RMSE and SSIM results, respectively, as the cloud coverage increases from 15% to 45%.

Ethical Statement

There are no ethical issues.

Acknowledgments

This work was supported by the National natural science foundation of China under Grant 42427801.