Complex-Valued Residual Diffusion with GRPO for Pansharpening

1 Introduction

Pansharpening aims to fuse a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LRMS) image to generate a high-resolution multispectral (HRMS) product that retains both fine spatial details and accurate spectral responses. The intrinsic difficulty stems from the spectral-spatial tension: LRMS largely determines the lowfrequency spectral content, whereas PAN provides abundant high-frequency structural cues that should be consistently injected into all spectral bands without introducing spectral distortions. Consequently, despite substantial advances with convolutional and transformer-based architectures [Masi et al., 2016; Yang et al., 2017; Xu et al., 2021; Wu et al., 2024], it remains challenging to achieve a robust balance between spatial sharpness and spectral consistency across diverse scenes and sensors.

∗Corresponding authors.

Diffusion models have recently shown strong capabilities in image restoration and conditional generation [Ho et al., 2020; Song et al., 2020; Saharia et al., 2022; Kawar et al., 2022], motivating their adoption for pansharpening. However, existing paradigms reveal a practical trade-off. Fig. 1 provides direct evidence: end-to-end CNN/Transformer approaches often achieve higher distortion fidelity (e.g., higher PSNR) but may yield less pleasing perceptual results, while diffusion-based methods tend to improve perceptual quality yet frequently sacrifice pixel-level fidelity. The contrasting behavior between PSNR (fidelity-oriented) and LPIPS (perceptual similarity) [Zhang et al., 2018] indicates that optimizing pansharpening with a single training objective is often insufficient to satisfy both fidelity- and perception-oriented criteria simultaneously.

Applying diffusion directly to full HRMS reconstruction further encounters two bottlenecks. First, diffusion training is typically driven by pixel-wise objectives (e.g., L1/L2 on noise or residual), which are not well aligned with how pansharpening quality is judged in practice-a combination of spectral consistency, structure/texture fidelity, and perceptual realism [Wald et al., 1997; Alparone et al., 2004]. Such losses may average out high-frequency details and remain insensitive to perceptual preferences. Second, learning diffusion on full HRMS signals can be unstable and sample-inefficient, since the model must simultaneously learn low-frequency multi-band spectra and high-frequency PAN-guided details, potentially leading to slow convergence and spectral artifacts. Meanwhile, recent evidence suggests that explicit spatial-frequency interaction is important for robust PAN-MS fusion [Tan et al., 2024; He et al., 2024], calling for denoisers that can model both domains more directly.

To address these challenges, we propose a two-stage framework that separates stable reconstruction from preference alignment. In Stage I, we construct a strong spectral base via a frozen pre-enhancement network and formulate pansharpening as residual diffusion conditioned on this spectral prior. Instead of denoising the full HRMS, the diffusion model learns only the residual between the prior and the HRMS target, which simplifies learning, stabilizes training, and reduces the tendency to hallucinate or drift the spectral base, while still focusing on injecting PAN-guided highfrequency structures through the residual. For the denoising backbone, we instantiate a Complex U-Net built upon complex-valued convolutions [Ronneberger et al., 2015; Trabelsi et al., 2018] and a spatial-frequency coupled block design. By jointly manipulating real and imaginary components, the model can capture amplitude-phase couplings and richer spectral-spatial interactions [Luo et al., 2025]. Moreover, PAN/LRMS cues (together with the spectral prior) are injected in a persistent multi-resolution manner, ensuring that structural guidance remains available throughout the iterative denoising trajectory.

In the second stage, we introduce Group Relative Policy Optimization (GRPO) [Shao et al., 2024] to fine-tune the conditional diffusion model toward task-relevant preferences. For each input, we generate a small group of candidate HRMS outputs via DDIM sampling and evaluate them with a composite reward that explicitly captures (i) spectral fidelity, (ii) PAN-aligned texture/edge consistency, and (iii) perceptual quality. We then compute group-relative advantages and optimize a softmax-weighted objective, yielding a stable preference-aligned learning signal [Schulman et al., 2017]. In contrast to purely likelihood- or pixel-driven training, GRPO directly aligns diffusion updates with taskrelevant quality preferences, improving the spectral-spatial trade-off in a controlled manner. To the best of our knowledge, this is the first work that introduces GRPO into the pansharpening domain.

The contributions of this paper are summarized as follows:

• We propose a spectral-prior conditioned residual diffusion framework for pansharpening and employ a complex-valued denoising network to strengthen spectral-spatial interaction modeling.
• We introduce Group Relative Policy Optimization into
• pansharpening for the first time to align diffusion training with multi-objective quality preferences.
• Extensive experiments on standard benchmarks demonstrate that the proposed method achieves a more balanced trade-off between spectral fidelity and perceptual quality than existing end-to-end and diffusion-based approaches.

2 RelatedWork

2.1 Pansharpening

Pansharpening aims to fuse a high-resolution panchromatic (PAN) image with a low-resolution multispectral (MS) image to obtain a high-resolution MS product. Traditional approaches are typically categorized into componentsubstitution [Gillespie et al., 1987; Haydn et al., 1982; Laben and Brower, 2000] and multiresolution-analysis [Vivone et al., 2015; Aiazzi et al., 2006] families, which are efficient but often suffer from spectral distortion under complex cross-sensor characteristics. Recently, deep learning has become the dominant paradigm by learning nonlinear mappings from paired data. Early CNN-based methods demonstrated that end-to-end training can significantly reduce artifacts and improve spectral fidelity compared with handcrafted fusion rules [Masi et al., 2016]. More recently, hypercomplex and complex-valued representation learning has been explored to better capture spectral-spatial and spatial-frequency interactions between PAN and MS; for example, quaternionbased modeling strengthens spatial-spectral interaction [Li et al., 2025a], while PanComplex builds a complex-valued pansharpening framework and reports strong performance with compact computation [Luo et al., 2025].

2.2 DiffusionModels for Image Generation and Restoration

Denoising diffusion models have emerged as powerful generative priors by learning a gradual denoising process [Ho et al., 2020]. To accelerate sampling, implicit formulations such as DDIM reduce the number of denoising steps while maintaining generation quality [Song et al., 2020; Lu et al., 2025]. Beyond synthesis, diffusion has been widely studied for inverse problems and image restoration, where the learned prior provides robustness under noise and illposed degradation [Saharia et al., 2022; Kawar et al., 2022; Chen et al., 2026]. These advances motivate diffusion-based formulations for remote sensing fusion, as they can model complex conditional distributions and produce visually plausible details under strong spectral constraints.

2.3 Complex-Valued Learning

Complex-valued neural networks (CVNNs) have been investigated as a principled way to jointly model amplitude and phase, offering richer representations for frequency-related patterns [Trabelsi et al., 2018]. In remote sensing fusion, complex-valued backbones are particularly appealing because the PAN image carries abundant high-frequency spatial cues while MS provides spectral content. PanComplex further validates the effectiveness of complex-valued convolution for pansharpening by explicitly leveraging complexdomain interactions [Luo et al., 2025].

2.4 Preference Optimization with Perceptual Rewards

Policy-gradient methods optimize objectives defined by task rewards, with REINFORCE providing an unbiased gradient estimator [Williams, 1992] and PPO improving stability through clipped updates [Schulman et al., 2017]. In parallel, preference optimization has recently gained popularity for aligning generative models to non-differentiable objectives without requiring explicit reward modeling in the deployment loop [Rafailov et al., 2023]. For low-level vision, perceptual criteria are frequently used to complement pixelwise losses: perceptual losses based on deep features [Johnson et al., 2016] and GAN-based super-resolution [Ledig et al., 2017] highlight the importance of human-aligned quality. Modern perceptual metrics such as LPIPS [Zhang et al., 2018], DISTS [Ding et al., 2020], and MS-SSIM [Wang et al., 2003] are therefore commonly adopted as objective surrogates for visual fidelity and structural consistency. These lines of work suggest that integrating preference optimization with perceptual and domain-specific rewards is a promising direction for high-quality pansharpening.

3 Method

3.1 Framework

Pansharpening takes a low-resolution multispectral image M ∈ R Hs × Ws ×C and a high-resolution panchromatic image P ∈ RH×W as input, and predicts a high-resolution multispectral image ˆ Y ∈ RH×W×C . The intrinsic difficulty lies in the spectral-spatial tension: LRMS mainly determines lowfrequency spectral content, whereas PAN provides the highfrequency structural cues that should be injected into all spectral bands without introducing spectral distortion. This tension explains why a single objective often leads to an undesirable compromise between fidelity and perceptual sharpness.

We address this challenge with a two-stage design (Fig. 2) that turns the trade-off into a structured optimization problem. Stage I focuses on stability and correctness: we first construct a strong spectral base and then train diffusion to model only the missing residual, so the generative model does not need to learn low-frequency spectra and high-frequency textures simultaneously. Stage II focuses on preference alignment: we further fine-tune the diffusion model via GRPO using a multi-objective reward that directly reflects what pansharpening should optimize in practice-spectral fidelity, PANaligned textures, and perceptual realism. This division of labor yields a coherent loop: Stage I ensures a reliable baseline reconstruction anchored by spectral prior, and Stage II improves the spectral-spatial trade-off in a controlled and explainable manner.

3.2 Spectral Prior Construction

A practical bottleneck of applying diffusion directly to HRMS reconstruction is that the model must simultaneously learn (i) low-frequency multi-band spectral content and (ii) high-frequency spatial details guided by PAN. In pansharpening, the low-frequency spectral component is relatively welldetermined by LRMS, while the ill-posed part is how to inject PAN textures into each band without breaking spectral consistency. To decouple these factors, we first build a spectral prior ˆM0 from LRMS:

where Fϕ is a pre-enhancement network and is frozen throughout Stage I and Stage II. We instantiate Fϕ using the spatial-frequency domain information integration (SFDI) design [Zhou et al., 2022a], which leverages complementary cues across domains to produce an HRMS-like estimate with strong spectral consistency. Freezing Fϕ is crucial: it turns ˆM0 into a stable spectral anchor, preventing diffusion from "re-learning" or drifting the spectral base during training. As a result, the subsequent generative modeling becomes substantially more stable and sample-efficient, and it empirically reduces spectral artifacts caused by over-injecting PAN structures.

3.3 Residual Diffusion Process

Given the spectral prior, we formulate pansharpening as a residual generation problem:

This choice is not merely a numerical trick; it reflects the underlying fusion logic. The prior ˆM0 already explains most low-frequency spectral information, so the residual r0 concentrates on (i) PAN-guided spatial details and (ii) subtle band-wise corrections needed for spectral consistency. Consequently, diffusion is trained on a cleaner and more structured target, which reduces low-frequency ambiguity and makes optimization less prone to over-smoothing or spectral drift.

We adopt the standard DDPM forward process [Ho et al., 2020] for residual diffusion:

The denoiser ϵθ is trained to predict the injected noise from a noisy residual rt under conditional guidance, using the standard noise-prediction objective:

For conditional guidance, we use PAN and LRMS cues (together with the spectral prior) to steer the denoising trajectory; this provides the denoiser with consistent structural information from PAN and band-wise context from LRMS, without requiring additional handcrafted transforms. At inference, we use DDIM sampling [Song et al., 2020] to obtain ˆ r0 efficiently and reconstruct ˆ Y = ˆM0 + ˆ r0.

3.4 Complex U-Net with Spatial-Frequency Interaction

A key question is how to design a denoiser that can repeatedly inject PAN structures during iterative denoising while maintaining cross-band spectral consistency. Conventional realvalued U-Nets mainly rely on spatial convolutions; they are effective for local edges but can struggle to model global frequency patterns (e.g., periodic textures and long-range correlations) and may overfit to PAN gradients, resulting in spectral distortion. Meanwhile, the PanComplex study validates that complex-valued convolutions can be beneficial for pansharpening by better exploiting amplitude-phase information in feature representations. Motivated by this evidence, we design the diffusion denoiser as a Complex U-Net (Fig. 3) that combines complex-valued feature learning with explicit spatial-frequency interaction inside each block.

Formally, a complex feature map is represented as x = x(r) + ix(i) . Given a complex kernel W = W (r) + iW (i) , complex convolution is computed by

This operation enables the network to jointly manipulate real and imaginary components, offering an expressive mechanism to encode amplitude-phase couplings that are difficult to capture with purely real-valued filters [Trabelsi et al., 2018].

Beyond the representation, the second key design is where and how we inject PAN/LRMS cues. As illustrated in Fig. 3, we inject these modalities at every scale of the encoder (and symmetrically propagate them through skip connections), rather than conditioning only once at the input. Let fℓ denote the feature at scale ℓ. We downsample PAN and LRMS (and the spectral prior) to the same resolution and concatenate themwith the current feature:

This persistent, multi-resolution injection is important for diffusion: since denoising is iterative, high-frequency guidance should remain available throughout the trajectory instead of being diluted after early layers.

The Complex Block (Fig. 3, right) then realizes explicit spatial-frequency interaction. The concatenated feature is split into modality-aware components and processed by two coupled branches: a spatial branch (complex convolutions) that captures local edge-aligned structures, and a frequency branch that transforms features via DFT/IDFT to model global interactions and long-range texture patterns [Xu et al., 2024; Li et al., 2024a; Li et al., 2025b]. We denote the frequency branch as

where F(·) and F -1 (·) are DFT/IDFT, and zms ℓ , zpan ℓ follow Fig. 3. Finally, we fuse the spatial and frequency outputs to obtain a feature that is simultaneously texture-rich and spectrally coherent:

By repeating this design across scales, the denoiser can (i) inject PAN details consistently at multiple resolutions, and (ii) correct spectral distortions through cross-band interactions that are explicitly supported by complex-valued and frequency-aware processing.

3.5 GRPO Fine-tuning withMulti-Objective Rewards

Stage I provides a stable residual diffusion backbone, but its learning signal is still dominated by pixel-level surrogates (noise prediction), which do not fully reflect how pansharpening quality is judged. In practice, good pansharpening simultaneously requires: (1) spectral consistency with the HRMS target, (2) faithful transfer of PAN textures/edges, and (3) perceptual naturalness. Optimizing only a single surrogate tends to over-emphasize one aspect and hurt the others, which underlies the persistent PSNR-LPIPS trade-off.

We introduce GRPO fine-tuning (Fig. 2) to explicitly align diffusion updates with these multi-faceted preferences. As illustrated in the lower part of Fig. 2, for each training input the denoiser generates K candidate outputs; these candidates are independently scored by three reward functions (spectral, texture, perceptual), and their normalized scores are converted to softmax weights that reweight the reconstruction loss-higher-reward candidates receive larger gradient contributions, steering the model toward outputs that jointly satisfy all quality criteria.

For each training input, we draw a small group of K candidate reconstructions { ˆ Y (k) }Kk=1 by DDIM sampling with different random seeds. We then assign each candidate a composite reward:

The spectral term Rspec emphasizes band-wise fidelity and spectral consistency; in practice we combine complementary criteria such as SAM and absolute deviation, so the model does not gain PSNR by introducing spectral angle distortions. The texture term Rtex enforces structural agreement with PAN; motivated by edge- and noise-aware cues in low-level vision and image forensics [Li et al., 2023; Zhu et al., 2024; Li et al., 2024b], we implement it by measuring edge/gradient alignment between the reconstruction (aggregated across bands) and PAN, encouraging the model to transfer textures while avoiding spurious edges. The perceptual term Rperc measures human-aligned similarity (e.g., LPIPS/DISTS) [Zhang et al., 2018; Ding et al., 2020], which is crucial for discouraging over-smoothed diffusion outputs and promoting visually pleasing details. Together, these terms turn the trade-off into an explicit objective rather than an implicit side-effect.

To keep optimization stable and reduce variance, GRPO uses group-relative weighting instead of trusting only the best sample. We normalize rewards within each group and convert them to softmax weights:

This mechanism emphasizes higher-preference candidates while still leveraging information from all samples, preventing unstable updates caused by occasional outliers. Conceptually, Stage I provides a strong and stable baseline anchored by the spectral prior, and Stage II refines the denoiser toward reconstructions that better satisfy spectral, texture, and perceptual preferences; together they yield a principled improvement of the spectral-spatial trade-off.

4 Experiments

4.1 Datasets and Benchmarks

We conduct experiments on three widely used pansharpening benchmarks: WorldView-II (WV2), WorldView-III (WV3), and Gaofen-2 (GF2). Following common practice, we construct training/validation/test splits under the Wald protocol [Wald et al., 1997], where LRMS and PAN pairs are generated in a reduced-resolution (RR) manner to provide ground-truth HRMS for supervised evaluation. For fullresolution (FR) evaluation without ground-truth HRMS, we report no-reference fusion metrics including Dλ, Ds, and QNR.

Baselines. To validate both reconstruction fidelity and perceptual quality, we compare against a compact yet representative set of baselines that are also used in our qualitative figure (Fig. 4). Specifically, we include classical end-to-end pansharpening networks PNN [Masi et al., 2016], GPPNN [Xu et al., 2021], and MutNet [Zhou et al., 2022b], as well as recent methods that emphasize stronger structure guidance such as HFIN [Tan et al., 2024]. We also include a diffusionbased baseline SSDiff [Zhong et al., 2024] as a representative diffusion paradigm for pansharpening. In addition, we report three traditional component-substitution baselines (Brovey [Gillespie et al., 1987], IHS [Haydn et al., 1982], and GS [Laben and Brower, 2000]) to contextualize the gain brought by learning-based fusion. Unless otherwise specified, all results are averaged over the corresponding test sets.

4.2 Implementation Details

All models are implemented in PyTorch and trained on a single NVIDIA GeForce RTX 4090 GPU. The multispectral images are pre-cropped into 128 × 128 patches with a scale factor of 4. We use the AdamW optimizer with a learning rate of 1 × 10-4 and batch size of 4. The diffusion process uses T=500 timesteps with a cosine noise schedule; 25step DDIM is adopted at inference. For GRPO, K=4 candidates are sampled per input with reward weights ωspec=0.5, ωtex=0.3, ωperc=0.2.

4.3 Comparison with State-of-the-art Methods

Evaluation on reduced-resolution scenes. Table 1 reports quantitative results on WV2/WV3/GF2 under the Wald protocol. We highlight the best results in bold and the secondbest in underline. Across all three benchmarks, our approach achieves a consistent improvement pattern: it increases distortion-fidelity metrics (PSNR/SSIM) and reduces spectral distortion (SAM/ERGAS). Together with the qualitative results and the ablation study on perceptual metrics, this indicates that the proposed two-stage training improves the fidelity-perception trade-off.

A closer look reveals a clear division among baselines. End-to-end methods such as PNN and GPPNN typically deliver competitive PSNR/SSIM but may struggle to preserve subtle PAN-aligned textures without introducing spectral bias. Conversely, diffusion-based methods (SSDiff) tend to yield visually more pleasing outputs and reduce perceptual errors, yet they often sacrifice pixel-level fidelity, which is reflected by a noticeable PSNR drop. Our method bridges this gap by separating responsibilities across stages: Stage I provides a strong fidelity-oriented foundation, while Stage II (GRPO fine-tuning) explicitly aligns optimization with multiobjective quality preferences. As evidenced by the comparison between Ours and Ours without GRPO, GRPO consistently improves perceptual metrics while keeping fidelity metrics at a strong level, validating the motivation introduced in Sec. 1.

Qualitative evaluation. Figure 4 presents visual comparisons on GF2 with the same set of methods as in Table 1. The first two rows show RGB renderings for intuitive inspection, while the last row visualizes per-pixel error maps w.r.t. GT using the same color scale. Several consistent phenomena can be observed. (i) End-to-end baselines (PNN/GPPNN) often produce locally over-smoothed textures or slightly biased colors in challenging regions (e.g., thin edges, small structures), indicating that optimizing with a single distortion-oriented objective may not fully capture perceptual preferences. (ii) The diffusion baseline (SSDiff) improves perceptual appearance in some regions, yet it may weaken fine structures or deviate in pixel-level fidelity, which is consistent with the quantitative trade-off seen in Table 1. (iii) Our reconstruction preserves sharper PAN-aligned structures while maintaining better spectral consistency, and the error maps show smaller discrepancies overall, suggesting that our method improves both structural transfer and spectral preservation.

Moreover, comparing Ours against Ours without GRPO isolates the effect of Stage II: GRPO fine-tuning yields cleaner local details and reduces perceptual errors in textured areas without introducing noticeable color shifts, supporting our claim that preference-aligned optimization is crucial for narrowing the fidelity-perception gap in diffusion-based pansharpening.

Evaluation on full-resolution scenes. Since real fullresolution scenes lack ground-truth HRMS, we further report no-reference metrics on full-resolution data. Table 2 shows that our method achieves the best Dλ, Ds, and QNR, indicating strong generalization in practical scenarios.

4.4 Ablation Experiments

We conduct ablation studies on WV2 to verify the contribution of each key component in our framework. Specifically, we examine: (i) the spectral prior (replacing it with a trainable prior or removing it), (ii) residual diffusion (directly diffusing HRMS), (iii) the Complex U-Net design (replacing complexvalued blocks with real-valued ones), and (iv) GRPO finetuning (removing preference alignment). Table 3 summarizes the results. Removing any component degrades performance, confirming that the proposed design is not a simple combination of tricks: the frozen spectral prior and residual diffusion stabilize training and suppress spectral artifacts, the Complex U-Net strengthens spectral-spatial interaction, and GRPO further improves the fidelity-perception trade-off.

5 Conclusion

We presented a two-stage pansharpening framework that improves the fidelity-perception trade-off by coupling spectralprior conditioned residual diffusion with preference-aligned optimization. In Stage I, a frozen spectral prior anchors lowfrequency spectral content and lets the diffusion model focus on PAN-guided high-frequency residuals, improving training stability and reducing spectral drift. To capture coupled spatial and frequency characteristics of PAN-MS fusion, we adopt a Complex U-Net denoiser with explicit spatialfrequency interaction. In Stage II, GRPO fine-tuning uses multi-objective rewards for spectral fidelity, PAN-aligned texture transfer, and perceptual quality, providing an alignment signal beyond pixel-wise surrogates. Experiments on standard benchmarks show consistent improvements over strong end-to-end and diffusion-based baselines, and ablations verify the contribution of each component. We believe this work provides a principled and practical direction for integrating diffusion modeling and preference optimization in future remote sensing image fusion tasks.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 62422609, and in part by the Fundamental Research Funds for the Central Universities under Grant WK2102026001.