DPMamba: Distillation Prompt Mamba for Multimodal Remote Sensing Image Classification with Missing Modalities
DPMamba: Distillation Prompt Mamba for Multimodal Remote Sensing Image Classification with Missing Modalities
Yueguang Yang, Jiahui Qu, Ling Huang, Wenqian Dong
Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence
Main Track. Pages 2224-2232.
https://doi.org/10.24963/ijcai.2025/248
Multimodal remote sensing image classification (RSIC) has emerged as a key focus in Earth observation, driven by its capacity to extract complementary information from diverse sources. Existing methods struggle with modality absence caused by weather or equipment failures, leading to performance degradation. As a solution, knowledge distillation-based methods train student networks (SN) using a full-modality teacher, but they usually require training separate SN for each modality absence scenario, increasing complexity. To this end, we propose a unified Distillation Prompt Mamba (DPMamba) framework for multimodal RSIC with missing modalities. DPMamba leverages knowledge distillation in a shared text semantic space to optimize learnable prompts, transforming them from ``placeholder" to ``adaptation" states by enriching missing modality information with full-modality knowledge. To achieve this, we focus on two main aspects: first, we propose a new modality-aware Mamba for dynamically and hierarchically extracting cross-modality interactive features, providing richer, contextually relevant representations for backpropagation-based optimization of prompts; and second, we introduce a novel text-bridging distillation method to efficiently transfer full-modality knowledge, guiding the inclusion of missing modality information into prompts. Extensive evaluations demonstrate the effectiveness and robustness of the proposed DPMamba.
Keywords:
Computer Vision: CV: Recognition (object detection, categorization)
Computer Vision: CV: Multimodal learning
Machine Learning: ML: Classification
