HSRMamba: Contextual Spatial-Spectral State Space Model for Single Hyperspectral Image Super-Resolution

1 Introduction

Hyperspectral images (HSIs) typically comprise tens to hundreds of closely contiguous spectral bands spanning a broad spectral range, thereby capturing rich spectral and spatial information simultaneously [Xiao and Wei, 2023]. This capability enables a more accurate characterization of intrinsic spectral properties and subtle variations in materials, and consequently facilitates widespread applications in fields such as agriculture [Lu et al., 2020], medical diagnosis [Lu and Fei, 2014], and remote sensing [Deng et al., 2023]. However, achieving high spectral resolution often necessitates sacrifices in spatial resolution due to the inherent constraints of imaging hardware and procedures. Hyperspectral image super-resolution (HSISR) addresses the challenge by converting low-resolution (LR) HSIs into the high-resolution (HR) HSIs, thereby enhancing spatial detail while preserving rich spectral information.

∗Corresponding author

Generally, HSISR can be categorized into fusion-based approaches [Guo et al., 2023; He et al., 2023] and single hyperspectral image super-resolution (SHSR) [Zhang et al., 2023], depending on whether auxiliary information is employed. Although fusion-based strategies can deliver superior results when aided by precisely registered additional images like multispectral image and panchromatic image, acquiring auxiliary information is often a significant challenge in real-world scenarios. Consequently, SHSR has attracted increasing interest. Over the past decade, deep learning SHSR methods have demonstrated remarkable advantages over traditional priors [Chen et al., 2023a], owing to their potent capacity for modeling complex nonlinear relationships. Among these methods, Transformer-based networks [Liu et al., 2022; Chen et al., 2023b] have substantially enhanced HSISR performance by capturing long-range dependencies in spatial and spectral dimensions, thereby highlighting the critical importance of modeling long-range dependencies in HSISR.

However, these models often face a trade-off between the global receptive field and computational efficiency [Li et al., 2023; Zhang et al., 2025; Dai et al., 2024] due to the attention mechanism in Transformers, which severely limits further advancements in HSISR.

Recently, Mamba, an emerging state space model (SSM), has garnered significant attention for its ability to capture long-range dependencies with linear computational complexity. Despite its theoretical potential to address the challenge in attention mechanism, Mamba suffers from some issues for HSISR. Concretely, When transforming hyperspectral images into 1D sequences, the Mamba model overlooks the spatialspectral structure among locally adjacent pixels, severely limiting its performance. Moreover, the output of Mamba is highly dependent on the input order, and a simple unfolding process disregards the modeling of spatial-spectral relationships between highly similar pixels, which is crucial for hyperspectral image restoration tasks.

To address these challenges, we propose HSRMamba, a state space model designed to efficiently capture both local and global long-range spatial-spectral dependencies. To the best of our knowledge, HSRMamba is the first attempt to employ Mamba tailored for SHSR. Specifically, we design a local spatial-spectral partitioning (LSSP) mechanism, which divides the 3D features into local spatial and spectral windows. By introducing the bidirectional SSM [Fu et al., 2024] (BSSM), a local spaital-spectral Mamba module (LSSM) is constructed to establish causal relationships among neighboring pixels within these local 3-D windows, which enhances the network's local modeling capacity. Furthermore, we devise a global spectral reordering mechanism (GSRM), which rearranges the global spectra based on global spectral similarity. Subsequently we employ the Global Spectral Correlation Mamba module (GSCM) to strengthen causal modeling between highly similar pixels, facilitating detail reconstruction in hyperspectral images. This process further strengthens causal relationships among pixels in the spatial and spectral dimensions that share high similarity. In summary, the main contributions of our work are as follows:

• We propose HSRMamba, the first SSM tailored for SHSR, which efficiently establishes local and global long-range spatial-spectral causal relationships.
• We devise a local spatial-spectral partitionin mechanism that captures patch-wise long-range spatial-spectral dependencies within a 3D window, thereby alleviating the inherent local pixel forgetting issue.
• We develop a global spectral-correlation Mamba module that globally extracts long-range spatial-spectral features by reordering spectra based on global spectral similarity, thus bolstering causal modeling among highly similar pixels.
• Extensive experiments on various datasets demonstrates the superiority and effectiveness of our proposed technique over the state-of-the-art methods.

2 RelatedWorks

2.1 Single Hyperspectral Image Super-resolution

Without additional auxiliary data (e.g., panchromatic or multispectral images), SHSR exhibits broader applicability compared to fusion-based methods [Dian and Li, 2019; Vivone et al., 2022; Dong et al., 2023]. SHSR can be categorized into traditional methods [Dian et al., 2017] based on handcrafted priors and deep learning-based approaches [Wang et al., 2023]. Over the past decade, numerous learningbased approaches, such as 3D convolution-based methods [Mei et al., 2017; Li et al., 2020; Fu et al., 2021; Li et al., 2021], group strategy-based methods [Li et al., 2018; Jiang et al., 2020; Li et al., 2022; Wang et al., 2022b; Wang et al., 2022a], and Transformer-based methods [Wu et al., 2023;

Chen et al., 2024], have demonstrated significantly superior performance compared to traditional approaches.

In the above methods, convolution-based with 2D or 3D convolutions mainly focused on local spatial-spectral features, overlooking long-range spatial-spectral dependencies. Recently, Cai et al. [Cai et al., 2022] proposed a spectral-wise multi-head self-attention for HSI reconstruction. Wang et al. [Wu et al., 2023] introduced 3D-THSR, combining spectral self-attention with 3D convolutions to model spatial-spectral features in a global receptive field. Hu et al. [Hu et al., 2024] transfered the HSI SR to the abundance domain with spectral-wise non-local attention to effectively incorporates valuable knowledge. Nevertheless, the computational complexity of Transformer networks grows quadratically with the input size, which significantly increases the demand for hardware resources when dealing with high-dimensional data such as hyperspectral images. While window-based selfattention mechanisms reduce computational costs by limiting the attention range, they fail to fully address the issue of high complexity. To overcome this limitation, we propose the Mamba network, a linear modeling framework designed to efficiently achieve hyperspectral image super-resolution reconstruction.

2.2 State Space Model

State Space Model [Gu et al., 2022] are a mathematical framework designed to model temporal or sequential dependencies efficiently. More recently, Mamba, a SSMs-based model with linear computational complexity, has garnered significant attention for outperforming Transformers in natural language processing [Gu and Dao, 2023] and computer vision [Zhu et al., 2024; Huang et al., 2024] tasks. Subsequently, some Mamba networks designed for low-level vision tasks [Guo et al., 2024; Qiao et al., 2024; Xiao et al., 2024; Zhi et al., 2024; Fu et al., 2024] have been proposed. Nonetheless, these methods are not suited for hyperspectral images. Firstly, they fail to consider the rich spectral information and the correlations between spatial and spectral dimensions inherent in hyperspectral data. Secondly, the Mamba network suffers from a local pixel forgetting issue, limiting its ability to model the structural information across spatial and spectral dimensions. Lastly, the performance of the Mamba network is highly dependent on the input sequence order, making it unable to fully exploit the relationships between highly similar pixels, which are crucial for image restoration.

3 Method

3.1 Preliminaries: State Space Models

SSMs provide a mathematical framework for modeling systems governed by latent states and their transitions over time. SSMs effectively model temporal dependencies in sequential data, and can be formulated as linear ordinary differential equation:

where h(t) ∈ RN is the hidden state vector at time t, A ∈ RN×N , B ∈ RN×1 , C ∈ R1×N and D ∈ R are the weight parameters, and N is the hidden state size.

Then, Eq.1 can be discretized using the zeroth-order hold (ZOH) rule, which transforms the continuous parameters A, B to discrete parameters A, B by the the timescale parameter ∆. It can be defined as:

After discretization, the discrete version of Eq. 1 can be defined in the following RNN form:

Furthermore, the SSM computation can be extended to a convolutional form as:

where L is the length of the input sequence and ∗ is the convolution operation, and K ∈ RL respresents a structured convolutional kernel. Recently, leveraging a dynamic representation mechanism, Selective State Space Model (Mamba) enhances the long-range modeling capabilities of state space models with linear computational complexity.

3.2 OverviewArchitecture

As shown in Fig. 2, the overall architecture of HSRMamba consists of three primary components: a shallow feature extraction module, a deep feature extraction module, and an upsampling module. Assuming the input LR hyperspectral image is ILR ∈ RH×W×B , where H, W, and B represent the spatial height, width, and the number of spectral bands, respectively. The SR hyperspectral image is denoted as ISR ∈ RsH×sW×B , where s is the scaling factor for superresolution reconstruction. The shallow features is extracted by using 3 × 3 convolutional layer. This process captures the structural information and adjusts the feature channel dimensions, which can be denoted as:

where fs(·) represents 3 × 3 convolution, and F0 denotes the extracted shallow features. Next, the shallow features are passed through the deep feature extraction module, consisting of multiple cascaded contextual spatial-spectral Mamba group (CSMG). The long-range spatial-spectral dependencies in deep features are extracted locally and globally. The process is defined as:

where Hn(·) denotes the function of the n-th CSMG, and Fn represents the deep spatial-spectral features extracted by the n-th CSMG. Each CSMG consists of multiple consecutive contextual spatial-spectral Mamba modules (CSSM), each of which is composed of a local spatial-spectral Mamba block (LSSB) and a global spectral-correlation Mamba block (GSCB). Additionally, convolution is employed at the tail of each group to introduce inductive bias into the network.

By leveraging long skip connections between shallow and deep features, the network focuses on learning highfrequency information beneficial for image reconstruction, improving both training efficiency and reconstruction performance. The merged deep features are then passed through a pixelshuffle layer to produce the upsampling deep features. Finally, SR HSI is obtained through long skip connections between the input features after bicubic upsampling and deep features.

3.3 Local Spaital-Spectral MambaModule

As the performance of Mamba is heavily reliant on the order of the input, converting a hyperspectral image into a 1-D sequence for sequential scanning poses significant challenges in effectively establishing causal relationships between locally adjacent pixels across spatial and spectral dimensions. This inherent limitation greatly hinders the Mamba network's effectiveness in HSISR. Therefore, we design a local scanning mechanism capable of precisely capturing detailed connections between locally adjacent pixels, effectively addressing

the limitations of Mamba in local causal modeling.

As shown in Fig. 2, LSSB consists of layer normalization, LSSM, channel atteniton (CA), and linear mapping layer. LSSM extract long-range spatial-spectral information from locally scanned regions. Additionally, CA mechanism is introduced to add inductive bias property into the network. Finally, the processed information is refined through linear layers, enhancing the module's capability for deep feature extraction. Assuming the input of the j-th local spatial-spectral Mamba module in the contextual spatial-spectral Mamba component is Fj, the above process can be expressed as:

where LN(·) denotes the layer normalization function, LSSM(·) represents the local spatial-spectral Mamba block, CA(·) denotes the channel attention mechanism, MLP(·) represents the linear layer function, and F jL is the final output of the module. LSSM consists of a LSSP and a BSSM [Fu et al., 2024]. As illustrated in Fig. 3, the spatial-spectral partitioning mechanism divides the input features into Hh × Ww × Cc local 3D feature blocks, each of size h × w × c, along the spatial and spectral dimensions. BSSM captures long-range spatial-spectral dependencies within each 3D local feature block. This process effectively addresses the local pixel forgetting issue inherent in traditional Mamba modules.

Forward 1 2 3 4 5 6 7 8 WHC 1 2 3 4 5 6 7 8 1 2 5 6 3 4 7 8 WCH 1 2 5 6 3 4 7 8 1 3 2 4 5 7 6 8 HWC 1 3 2 4 5 7 6 8 1 3 5 7 2 4 6 8 HCW 1 3 5 7 2 4 6 8 1 5 3 7 2 4 6 8 CHW 1 5 3 7 2 4 6 8 1 5 2 6 3 7 4 8 CWH 1 5 2 6 3 7 4 8 Backward In this paper, we introduce a BSSM [Fu et al., 2024] as the foundational unit for global spatial-spectral feature extraction. As shown in Fig. 4, the BSSMmodule takes the feature sequence from LSSP or GSRM as input of bidirectional branches and models comprehensive global spatial-spectral causal relationships from different directions via bidirectional spatial-spectral continuous scanning (BSSC). This design complementarily enhances contextual long-range dependency modeling, making it suited for hyperspectral images.

3.4 Global Spectral CorrelationMambaModule

In the Mamba network, the output of the current input relies solely on the preceding input in the data sequence. However, the recovery of fine details during the super-resolution process heavily depends on global similar pixels. To address this limitation, we propose a global spectral reordering mechanism and design the global spectral correlation Mamba.

As shown in Fig. 2, GSCB consists of layer normalization, the designed GSCM module, CA, and linear mapping layer. GSCM enhances glboal spatial-spectral causal modeling through the proposed global spectral reordering mechanism. Additionally, this module incorporates a channel attention mechanism to introduce inductive bias into the network. Finally, a linear layer is employed to enhance the representation capabilities of this paper.

Assuming the input to the j-th global spectral correlation Mamba module in the contextual spatial-spectral Mamba component is Fj, the above process can be expressed as:

where LN(·) denotes the layer normalization function, GSCM(·) represents the global spectral correlation Mamba block, CA(·) denotes the channel attention mechanism, and MLP(·) represents the linear layer function. ˆ F jG is the intermediate output of the j-th global spectral correlation Mamba module, and F jG is the final output of the module.

As illustrated in Fig. 3, the global spectral correlation Mamba consists of the GSRM and the BSSM. The global spectral reordering mechanism first computes the correlation coefficient matrix between spectral features. It then calculates the average of correlation coefficients among each spectrum as the global correlation value. Finally, the module reorders the spectral features based on their global correlation values, ensuring that pixels with high spectral correlation are closer in the spatial-spectral dimensions. This process significantly enhances the Mamba module's performance in extracting long-range spatial-spectral features.

3.5 Loss Function

The network is optimized using three losses: l1 loss, spectral angle mapper (SAM) loss, and gradient loss in both spatial and spectral domains. The l1 loss calculates the absolute pixel-wise difference between the reconstructed and original hyperspectral images, encouraging sharper and more detailed results compared to l2 loss. The SAM loss ensures spectral consistency by considering the spectral characteristics of the data. Gradient loss enhances image sharpness by focusing on differences between adjacent pixels. The total loss is formulated as:

where λs and λg are hyper-parameters that balance the losses, with λs = 0.3 and λg = 0.1 used empirically. The detailed function can be expressed as:

where N is the batch size, Hn hr andHn sr represent the n-thHR and SR hyperspectral images, and M(·) denotes gradients in the horizontal, vertical, and spectral directions.

4 Experiments

4.1 Experimental Settings

Datesets

We conducted experiments on three hyperspectral image datasets: Chikusei [Yokoya and Iwasaki, 2016], Houston2018, and Pavia Center [Huang and Zhang, 2009]. The experiments on Pavia Center are presented in the supplementary materials. The Chikusei dataset, captured with the Headwall Hyperspec-VNIR-C sensor, includes 128 spectral bands over agricultural and urban areas in Japan, with a spatial resolution of 2517 × 2335 pixels. The Houston 2018 dataset is a part of the 2018 IEEE GRSS Data Fusion Contest, acquired by the ITRES CASI 1500 imager, covers the University of Houston and surrounding urban areas with 48 spectral bands and a resolution of 4172 × 1202 pixels.

Compared Methods and Metrics

We compare the proposed method with 8 deep learning approaches, including Transformer-based methods SwinIR [Liang et al., 2021] for natural images, Mamba-based methods MambaIR [Guo et al., 2024] for natural images, and hyperspectral image group-based methods such as GDRRN [Li et al., 2018], SSPSR [Jiang et al., 2020], RFSR [Wang et al., 2022b], and GELIN [Wang et al., 2022a]. We also include Transformer-based SHSR methods like AS 3 ITransUNet and MSDformer [Chen et al., 2023b]. The performance of these methods is evaluated using six commonly used metrics in both spatial and spectral dimensions, including peak signalto-noise ratio (PSNR), structure similarity (SSIM), spectral angle mapper (SAM), cross correlation (CC), root-meansquared error (RMSE), and erreur relative global adimensionnellede synthese (ERGAS).

Implementation Details

The kernel size of the convolution is set to 3 × 3. We set the number of channels C to 64, the number of CSMG to 4, and the number of CSSM to 2. The initial learning rate is 1e-4 , halving every 100 epochs until reaching 400 epochs. Following [Zhang et al., 2018], the reduction ratio in channel attention (CA) is set to 16. During training, the Adam optimizer with Xavier initialization is used with a mini-batch size of 8. For image reconstruction, we use a progressive upsampling strategy via PixelShuffle [Shi et al., 2016] to reduce parameters. The model is implemented in Pytorch and trained on NVIDIA RTX 4090 GPUs.

4.2 Comparison Results

Experiments on the Chikusei Dataset

For Chikusei dataset, 4 non-overlapping images with the size of 512 × 512 × 128 are cropped from the top region. The remaining area is cropped into overlapping HR images for training (10% randomly selected for validation). The spatial size of the LR for training is 32×32, while the corresponding HR sizes at scale factors ×4, and ×8 are 128 × 128, and 256 × 256, respectively. All LR patches are generated by Bicubic downsampling at different scales.

Table 1 presents the quantitative results of our method and the compared approaches at different scale factors on the Chikusei dataset. The best results are highlighted in bold, while the second-best results are underlined. Our method surpasses SSPSR by 0.29 dB in PSNR and 0.32 in SAM at ×4, as group-based methods like SSPSR fail to leverage global spatial and spectral information effectively. Methods for natural images like SwinIR and MambaIR fail to fully utilize the rich spectral information and spatial-spectral correlations in hyperspectral images, resulting in poor spectral performance. Transformer-based SHSR methods like MSDformer and AS 3 ITransUNet achieve better SR performance than the above methods by modeling long-range dependencies. Notably, HSRMamba outperforms other methods for all metrics at scale factors ×4, and ×8, demonstrating the superiority and effectiveness of our method.

As shown in Fig. 5, group-based methods like GDRRN, often introduce blur details due to the limited spatial-spectral modeling. Transformer-based methods, such as MSDformer, consider global spatial and spectral information simultaneously, resulting in clearer boundaries and fewer artifacts compared to methods designed for natural images, such as MambaIR. HSRMamba reconstructs HR hyperspectral images with clearer and sharper details, which demonstrates that our HSRMamba can efficiently model long-range spatialspectral dependencies. Additionally, mean error maps across all spectra in Fig. 5 show the reconstruction accuracy for individual pixels, with bluer regions indicating higher accuracy. And the mean spectral difference curves at ×4 in Fig. 7 evaluate the SR results from a spectral perspective. From the above visual results, it is evident that our method achieves results closer to the ground truth in both spatial and spectral dimensions compared to other methods.

Experiments on the Houston 2018 Dataset

Similar to Chikusei dataset, 8 images from the Houston2018 dataset with the size of 256 × 256 × 48 are cropped from the top region for testing. The spatial resolution of LR and HR training patches is consistent with the Chikusei dataset.

The quantitative results of all methods on Houston dataset are shown in Table 1. HSRMamba outperforms the compared methods for all metrics at different scale factors. The visual results and mean error maps of all algorithms are presented in Fig. 6. We can also observe that HSRMamba provides more accurate results compared with other approaches.

4.3 Ablation Study

In this paper, we conduct the ablation experiments at factor scale ×4 on the Chikusei dataset. Additional ablation studies are listed in the supplementary materials.

Effects of LSSP and GSRM

LSSP effectively addresses the issue of local forgetting and GSRM relieves the sensitivity of the Mamba network to input order. As shown in Table 2, when LSSP is removed, PSNR of the network decreases by 0.09 dB, demonstrating the effectiveness of LSSP. The PSNR of model without GSRM decreases by 0.12 dB compared to the original method, demonstrating the effectiveness of GSRM. Finally, when both LSSP

and GSRM are removed, the performance drops significantly, further confirming the effectiveness of our method.

Effects of the Number of Groups

HSRMamba consists of multiple consecutive Mamba groups, and Table 3 shows the impact of the number of Mamba groups N. When N = 2, the performance is the weakest. As N increases to 4, the quantitative metrics improve. However, setting N = 6 leads to a decline in performance. This is mainly due to the increased network depth, causing the model to overfit. Therefore, considering both the experimental results and the model parameters, we set N = 4 in the paper.

Parameter and Complexity Analysis

To evaluate the computational efficiency of the proposed HSRMamba, we compare the model parameters, FLOPs, and PSNR results for different methods. As shown in Fig. 1, our method achieves better SR results with lower computational complexity and fewer parameters compared to other methods, demonstrating the effectiveness and efficiency of our approach. This indicates that our method strikes an excellent balance between model complexity and performance.

5 Conclusion

This paper presents HSRMamba, a contextual spatialspectral relationship modeling algorithm designed for efficient HSISR. To address the issue of local pixel forgetting in hyperspectral images, we propose the LSSP to establish patch-wise long-range spatial-spectral correlations. Additionally, to overcome the challenge of insufficient causal modeling between highly similar pixels, we leverage the GSRM that rearranges the spectral dimension based on global spectral correlations. Using these algorithms, we construct the CSSM to efficiently capture long-range spatial-spectral dependencies in hyperspectral images. The CSSM module is composed of LSSM and GSCM, which enhance causal modeling by considering both local and global perspectives. Finally, extensive comparative experiments on different datasets and ablation studies validate the superiority and effectiveness of our proposed approach.

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant 62431020, the Foundation for Innovative Research Groups of Hubei Province under Grant 2024AFA017, the Fundamental Research Funds for the Central Universities under Grant 2042025kf0030.