Always Clear Depth: Robust Monocular Depth Estimation Under Adverse Weather

1 Introduction

Monocular depth estimation (MDE) is a fundamental task in computer vision that aims to predict the depth from a single image. It has wide-ranging applications in autonomous driving [Sch¨on et al., 2021; Xue et al., 2020], robot navigation [H¨ane et al., 2011], and 3D reconstruction [Yu and Gallup, 2014; Yin et al., 2022]. MDE methods can be roughly divided into supervised methods and self-supervised methods [Arampatzakis et al., 2023]. The former [Bhat et al., 2023; Ranftl et al., 2020; Ranftl et al., 2021] directly learns depth mapping from the RGB input with paired samples (depth maps or 3D sensors like LiDAR). While achieving impressive performance, these methods heavily rely on highquality ground truth data, which is time-consuming and laborintensive for collection. By contrast, the latter [Godard et al., 2017; Godard et al., 2019; Zhou et al., 2017] uses only image sequences captured by a single camera and the corresponding camera parameters, and employs two assumptions (photometric constancy and rigid motion) [Godard et al., 2019] to produce the supervised signal. Specifically, these methods assume that scene information remains largely unchanged as the viewpoint shifts, relying on the geometric consistency between consecutive frames to provide depth supervision. While existing self-supervised approaches [Zhao et al., 2022; Zhang et al., 2023; Lyu et al., 2021] have shown significant success in outdoor scenes, their performance deteriorates under adverse conditions, such as low-light and rain-haze conditions. In these situations, insufficient lighting and significant motion perturbations (reflections of raindrops and streetlights) violate the assumptions mentioned above, leading to an unreliable estimation.

∗Corresponding Author

Recent studies [Gasperini et al., 2023; Saunders et al., 2023; Tosi et al., 2025] have explored ways to enhance the robustness of models under adverse conditions. Some technologies elaborate modules to promote robustness in specific scenes, such as nighttime and rain-haze conditions [Shi et al., 2023; Zheng et al., 2023; Yang et al., 2024a]. However, due to the limitation for scene-specific representation, these methods struggle to generalize to complex and diverse environments, frequently encountered in real-world applications. To promote generalization, some efforts harmonize the merits of distillation learning, contrastive learning, and data augmentation strategies to create more generalized models that improve model performance across varied scenes [Wang et al., 2024b; Wang et al., 2024a; Saunders et al., 2023]. For example, md4all [Gasperini et al., 2023] generates the [normal, degraded] sample pairs with a GAN-based model and uses distillation learning to extract supervised signals from clear depth maps to train the student model. However, the degraded characteristics in the translated image are unrealistic and unnatural (shown in Figure. 1(a)), which significantly affects the generalization from normal scenarios to real adverse weather. In addition, existing methods are often vulnerable to disturbances in complex scenes due to slack constraints between the teacher and student models, leading to incomplete knowledge transfer. Consequently, as shown in Figure. 1(b), md4all suffers from nonnegligible performance decline when there are significant domain differences between scenes, making it far from generalizing to various degraded scenarios.

Overall, two critical issues are imperative to train a generalizable and reliable depth estimation model: (1) the lack of a high-quality multi-tuple dataset covering diverse degradation types; (2) slack constraints between the teacher and student model, resulting in incomplete knowledge transfer.

To mitigate the problem of data scarcity, the author in [Tosi et al., 2025] utilize depth maps and text descriptions as control conditions to generate [normal, degraded] sample pairs with T2I-Adapter [Mou et al., 2024]. This provides an alternative solution with diffusion models for multi-tuple dataset generation [Zhu et al., 2017; Parmar et al., 2024; Sauer et al., 2025]. However, the translated image shows obvious inconsistencies and less authenticity of scene contents, which is unacceptable for training generalizable depth estimation models.

To address these issues, we propose to optimize the data generation and domain adaptation learning, and construct the degradation-agnostic robust monocular depth estimation method (ACDepth). Specifically, we fully explore the potential generation capability of diffusion model and employ LoRA adapters to encourage the network to generalize to diverse scene generation. Meanwhile, we integrate circular consistency loss and adversarial training to guarantee the naturalness and consistency of translated images. To achieve full transfer and alignment of capabilities, we propose the multigranularity knowledge distillation strategy (MKD), which borrows priors from the teacher model to provide comprehensive supervision and guidance to the student model. Besides the commonly used feature and result distillation learning, we pioneer the ordinal guidance distillation mechanism (OGD). In summary, the contributions are as follows:

• We propose a practical multi-tuple degradation dataset generation scheme, and develop a novel robust monocular depth estimation framework, termed as ACDepth for high-quality depth estimation under adverse weather.
• We propose the multi-granularity knowledge distillation strategy (MKD) to achieve the complete transfer and alignment of capabilities from the teacher model to student model. In addition, we introduce the ordinal guidance distillation mechanism (OGD) to heartens the network to focus on uncertain regions through differential ranking.
• Extensive experiments demonstrate the effectiveness of ACDepth, surpassing md4all-DD by 2.50% for night scene and 2.61% for rainy scene on the nuScenes dataset in terms of absRel metric.

2 RelatedWork

2.1 Monocular Depth Estimation

Before the advent of deep neural networks, traditional depth estimation methods primarily rely on handcrafted priors to explore the limited physical and geometric properties. Deep learning methods [Sun et al., 2012; Liu et al., 2015b; Liu et al., 2015a] have emerged as a preferable alternative due to their ability to learn generalizable priors from largescale data, such as depth maps from LiDAR or RGB-D cameras. However, due to the high cost to obtain the highquality annotation, self-supervised learning technologies, deriving depth information from stereo pairs [Garg et al., 2016; Godard et al., 2017] or video sequences [Zhou et al., 2017; Godard et al., 2019; Bian et al., 2019] have drawn growing interest. Unfortunately, a significant portion of the effort is focused on the normal scene. The depth estimation under adverse weather, such as low-light and rain-haze conditions, is barely explored, which is the common scenario in autonomous driving. Some studies [Zheng et al., 2023; Wang et al., 2021; Guo et al., 2020] divide depth estimation in adverse scenes into denoising and estimation, but these methods show poor generalization under unknown degradations. Further, researchers achieve more robust depth estimation through data generation and knowledge distillation [Zhu et al., 2023; Gasperini et al., 2023], and integrate multi-level contrastive learning with diffusion model for robust feature representation [Wang et al., 2024b]. However, the data quality and completeness of constraints in these methods significantly affect the generalization from normal scenarios to real adverse weather.

2.2 Distillation Learning

Early distillation learning methods primarily focus on model compression and acceleration. A classic example of distillation learning is to guide the training of student models by softening the output probability distribution of the teacher model [Hinton, 2015]. Subsequently, several studies have successfully applied distillation learning to monocular depth estimation. For example, Song et al. [Song and Yoon, 2022] selectively distill stereo knowledge for monocular depth estimation, using learned binary masks to pick the best disparity or pixel-wise depth map. More recently, md4all [Gasperini et al., 2023] trains the teacher model on clear samples by selfsupervised learning, and transfers the ability or priors to the student model under adverse weather. However, during the distillation process, the student model struggles to fully reproduce the ability of the teacher model, relying solely on depth-derived pseudo-labels or specific priors. To address these challenges, we propose a multi-granularity knowledge distillation strategy that enhances the knowledge transfer process by borrowing priors from multiple teachers to provide comprehensive supervision and guidance.

3 Method

3.1 Preliminary

MDE aims to estimate the depth map Dt from a single RGB image It. Restricted by high-quality paired samples in real-world scenarios, especially for adverse weather, the self-supervised learning strategy [Zhu et al., 2023; Wang et al., 2024b] provides an alternative solution to obtain supervised signals of the target image It from adjacent frames It′ ∈ [It-1, It+1]. Specifically, the pose network and depth network are jointly optimized. The former estimates the relative pose of camera motion Tt→t′ , from the target image to the adjacent frame (It′ ). Combining with the intrinsic parameters K of camera, it allows the network to synthesize the reconstruction image (It′→t) of It using the adjacent frame It′ , depicted as:

where the 〈〉 denotes the pixel sampling operator, and we constrain the depth by calculating the photometric reconstruction loss between It and It′→t, formulated by:

where SSIM is the structural similarity index measure. Compared with the pixel-wised L2 loss, it can better reflect the structural similarity between images. Additionally, edgeaware smoothness loss is also used to constrain the continuity of the depth:Le (D) = |∂xD′ | e∂xI + |∂yD′ | e∂yI , (4)

where D′ refers to the normalized inverse depth of D. ∂x and ∂y represent the horizontal and vertical gradients. Similar to [Gasperini et al., 2023], the above theoretical foundations of self-supervised learning are used to train the teacher model in this work.

3.2 Overall Architecture

Our ultimate goal is to achieve the robust MDE under adverse weather. Thus, we elaborate an ACDepth approach, as shown in Figure. 2, which involves the data generation and robust model training. In the first part, we utilize the LoRA adapters to fine-tune the pretrained diffusion model, where the circular consistency loss and adversarial training are used to guarantee the naturalness and consistency of the translated image. In this way, the trained generator can produce a multi-tuple degradation dataset under different weather conditions. And then, ACDepth takes the multi-tuple degradation dataset as input, and proposes the multi-granularity knowledge distillation strategy (MKD) to borrow priors from the teacher model to optimize the student network. The process is depicted as:

where Ld denotes the distillation loss between the teacher and student models, Lr refers to the ordinal guidance distillation between the Depth Anything V2 model and our ACDepth, and Lc is the feature consistency loss between the teacher encoder and the student encoder. λ1 and λ2 are the weight parameters to balance the loss components. Detailed explanation of losses (Ld, Lr and Lc) is provided below.

3.3 Data Generation

Given a normal input (ei), which is captured under good lighting and visibility conditions, the previous studies employ GAN-based or diffusion-based models to generate the degraded images (hci ). c represents various weather scenes, such as rain, night, and fog. However, besides requiring a large number of real multi-tuple pairs for training, the translated images generated by these technologies [Gasperini et al., 2023; Tosi et al., 2025] exhibit obvious differences and unnaturalness from the real samples, leading to poor generalization of depth estimation models across different scenes.

Inspired by recent image translation technologies [Parmar et al., 2024], we explore the content generation capability of stable diffusion, while employing adversarial learning and cycle consistency loss to train LoRA [Hu et al., 2021] adapters to promote the naturalness and consistency of translated images. Specifically, we take ei as input, along with the corresponding text prompt Pc to learn the specific LoRA adapters for each scene transformation, which can complete the conversion from the source domain to the target domain. The aforementioned process is depicted as:

where SDT (·) refers to the Stable Diffusion Turbo model, and Fc(·) is the translator that transforms the normal sample into the corresponding adverse scene with the condition c and text prompt Pc. Detailed experiments related to data generation can be found in the appendix.

3.4 Robust Model Training

To achieve full transfer and alignment of capabilities between the teacher model and student model, we propose the multigranularity knowledge distillation strategy (MKD) to achieve the robust model training of the student model, detailed as follows.

Distillation Learning. Similar to the existing technology [Gasperini et al., 2023], the commonly used multi-scale feature distillation loss between the teacher model (FT (·)) and student model (FS(·)) is employed to facilitate the perception of the student model to the adverse weather. For the given normal input ei and the produced degraded sample (hci ), the distillation loss function is defined as:

where S denotes the number of different scales and mi represents a sample randomly selected from the mixed training set.

Ordinal Guidance Distillation. 1) Uncertain Region Definition: We use the output from Depth Anything V2 model as the supervisory signal because it ensures accurate depth prediction while maintaining acceptable inference speed. DT and DS represent the inverse depth of the teacher model and student model, respectively, and Dv represents the output depth of Depth Anything V2. We first identify the regions with significant discrepancies between DT and DS for refinement:

where ¯D reflects the difference between the output of the teacher model and the student model. We normalize the depth difference ¯D to obtain ˆD, which takes values between 0 and 1. We define U as the region of focus for the loss, and the region of focus is determined by the following equation:

where γ is the threshold for dividing the key regions, set to 95% of ˆD. These key regions correspond to areas with impaired perception in challenging scenes, as shown in Fig 3(e).

2) Depth Ordinal Strategy: We sample a pixel from U ∗DS , with the corresponding depth value p0, and another pixel from (∼ U ) ∗ DS, with the corresponding depth value p1. The ranking loss [Xian et al., 2020; Sun et al., 2023] for the pair [p0, p1] is computed as follows:

The ordinal label ℓ is calculated as follows:

where τ is a hyperparameter used to control the selection of pixel pairs for the sorting. p∗0 is sampled from the region U ∗ Dv, and p∗1 is sampled from the complementary region (∼ U ) ∗ Dv. These two pixels, [p∗0 , p∗1 ], form an ordinal pair. The ranking loss aims to enhance the student model's performance by focusing on poorly predicted regions. However, we found that the constraints imposed by the distillation loss sometimes conflicted with the ranking loss. To mitigate this issue, after identifying each ordinal pair, we compute the average depth value of the surrounding 5×5 pixel region for each sample point and use this average value to calculate the ranking loss. Considering the increase in computational overhead, we control the number of selected points in each iteration by setting τ = 0.15 to select fewer sample points. Similarly, we set the proportion of sample pairs selected from U to 0.05. Finally, we randomly sample ordinal pairs from the global depth map by a ratio of 0.01, and the ranking loss calculated from these sampled points is used to refine the global depth information. The sampling process is illustrated in Figure. 3. Thus, our final loss becomes:

where Zg and Zl represent the global and local ordinal sampling sample sets, respectively.

Feature Consistency Constraint. Since a robust model can provide the precise perception on both the normal and degraded scenarios, we propose the feature consistency constraint to mitigate the impact of image degradation, thus generalizing to various adverse weather. Specifically, we dynamically adjust the feature alignment strategy based on the input image ei. If ei is the degraded image hci , we minimize the error between the feature maps FS(h) and FT (e), as well as the error between FS(h) and FS(e), where FT and FS represent the feature extraction by the teacher and student models, respectively. If ei is a clear sample e, we minimize the error between the feature maps FT (e) and FS(e), respectively extracted by the teacher and student models. This form can be written as:

where f represents L1 loss for aligning features, and the horizontal line denotes that no gradient backpropagation is applied. The above process enables the student model to not only acquire semantic features from the teacher model but also progressively improve its feature extraction capabilities across both normal and challenging scenarios, ensuring a seamless transition from clear to complex environments.

4 Experiments

4.1 Datasets

In this study, the commonly used nuScenes [Caesar et al., 2020] and RobotCar [Maddern et al., 2017] datasets are used for training and comparison. NuScenes is a comprehensive dataset featuring diverse outdoor scenes, specifically tailored for autonomous driving research, where the multi-frame sequence is supplemented with detailed radar annotations. Following [Gasperini et al., 2023], we adopt 15,129 generated samples (day-clear, day-rain, night) for training and 6,019 samples (including 4449 day-clear, 1088 rain, and 602 night) for testing. RobotCar is a large outdoor dataset collected in Oxford, UK. Following [Gasperini et al., 2023], we adopt 16,563 generated samples (day, night) for training and 1,411 samples (including 702 day 709 night) for testing. During the data preprocessing stage, images in the nuScenes dataset are resized to 320 × 576, while those in the RobotCar dataset are resized to 320 × 544. We record metrics within a range of 0.1m to 50m for RobotCar and 0.1m to 80m for nuScenes.

4.2 Implementation Details

All experiments are conducted on the same ResNet18 architecture [He et al., 2016]. We train the student model and teacher model on a single NVIDIA 3090 GPU with a batch size of 16, using the Adam optimizer. We set initial learning rate to 5e-4, reducing it by a factor of 0.1 every 15 epoch. The student model are trained for 25 epochs. Following the experimental protocol of [Gasperini et al., 2023], we maintain identical hyperparameter settings for self-supervised learning. Through experimental validation of different parameter combinations, the weights for the loss functions are set to λ1 = 0.01, λ2 = 0.02. Considering both the effectiveness of supervision signals and training efficiency, we select the small version of Depth Anything V2 [Yang et al., 2024b] as our supervision prior for ordinal guidance distillation. Further details on the model training procedure and dataset translation process are provided in the appendix.

4.3 Comparison with SoTAs

In this section, we evaluate our model on the nuScenes and RobotCar datasets. Qualitative and quantitative comparisons are shown below.

Comparison on the nuScenes. In Table 1, we compare our method with existing depth estimation models, including the typical MDE technologies (Monodepth2 [Godard et al., 2019], PackNet-SfM [Guizilini et al., 2020]) and robust depth estimation approaches (RNW [Wang et al., 2021], md4all [Gasperini et al., 2023] and DMMDE [Tosi et al., 2025]). As shown in Table 1, our proposed ACDepth shows significant competitiveness, particularly for the night condition, respectively reducing 2.50% and 4.49% in terms of absRel and RMSE metrics, when compared to robust depth estimation method md4all-DD. DMMDE [Tosi et al., 2025] uses different depth models as prior information to guide image translation, and achieves improvements over md4all-DD. However, due to the scale-and-shift-invariant loss, the scaling relation of depth map is ignored and corrupted during alignment, degrading the final performance. For better convincing,

�D� QX6FHQHV

�E� 2[IRUG 5RERW&DU

Table 1: Evaluation of self-supervised methods on nuScenes [Caesar et al., 2020] validation set. v1,v2,v3: depth maps from md4all-DD, DPT, Depth Anything in [Tosi et al., 2025]. Supervisions (sup.): M: via monocular videos, *: test-time median-scaling via LiDAR, v: weak velocity, s: test-time scaling via LSE criterion [Ranftl et al., 2020]. Training data (tr.data): d: day-clear, n: night, r: rain, a: all. T†: Translated via GAN, T‡: Translated via Diffusion. Visual support: 1st and 2nd.

Figure. 4(a) presents qualitative results. md4all-DD is particularly sensitive to water reflections in rainy scenes. For example, in the first and second rows of Figure. 4(a), the obvious errors of depth estimation are observed in the regions of water surface reflections. In addition for nighttime scenes, md4allDD is hard to predict the real depth of regions hidden in the darkness. In contrast, our ACDepth method can produce reasonable and reliable predictions of depth information in night scenes, such as trees and cars under low-light conditions.

Comparison on the RobotCar. To further verify the effectiveness, we compare our ACDepth with existing state-of-theart approaches on the RobotCar benchmark. As expected in Table 2, our method achieves the best performance in almost all metrics. Compared to md4all-DD, our approach reduces RMSE by 8.80% on the standard benchmark and 11.99% on the challenging benchmark, highlighting the superiority of our approach. To demonstrate the effectiveness of our method, Figure. 4(b) provides the visual comparisons, showing that our ACDepth can produce more precise depth maps over those of md4all-DD on both day and night scenes. We speculate that these considerable improvement of this study stems from the more reliable data generation and comprehensive prior constraints, which contribute more to the robust model training and optimization.

4.4 Ablation Study

In this section, we conduct detailed ablation experiments on nuScenes and RobotCar datasets to demonstrate the individual effectiveness of the proposed components.

Evaluation on Major Design Components. Our baseline is trained on daytime scenes [Gasperini et al., 2023]. Based on this, we introduce the distillation learning (DL) to promote training with more robust feature representation, reducing the absRel metric by 22.16% for night scene and 10.69% for rainy scene on the nuScenes. Then we construct another model with additional ordinal guidance distillation (OGD), which provides more precise supervision signals for degraded conditions. By contrast, it achieves the comprehensive improvements in model performance, reducing the absRel metric by 0.54% for day scene and 0.73% for night scene on the RobotCar. In addition, we introduce the feature consistency constraint (FCC) to evaluate its effect for robust feature representation. As expected, the model with FCC significantly enhances the accuracy of depth estimation in challenging conditions (such as nighttime and rainy scenes), reducing the absRel metric by 0.37% for night scene and 2.62% for rain scene on the nuScenes. We also provide the visual comparisons between the improved versions. As shown in Figure. 5, the complete model equipped with DL, OGD and FCC strategies shows significant superiority over its imperfect versions.

These optimization strategies not only provide reliable supervised signals but also guide the network to focus on uncertain regions, resulting in more robust depth estimation.

Evaluation onOrdinal Guidance Distillation. In this section, we investigate the impact of different sampling methods on ordinal guidance distillation, with the experimental results presented in Table 4. Our OGD strategy employs two sampling methods: global and local sampling. We observed that model performance became more reliable as both sampling methods were progressively incorporated. Additionally, our experiments demonstrate that using windowed sampling regions enables the model to learn more robust feature representations, significantly reducing the risk of overfitting.

5 Conclusion

In this paper, we propose a novel approach named ACDepth for robust monocular depth estimation under adverse weather conditions. In addition, we introduce an elaborate data generation scheme to produce the multi-tuple depth dataset with diverse degradations, which significantly mitigates the problem of data scarcity. Meanwhile, we construct an effective multi-granularity knowledge distillation (MKD) strategy to achieve the robust model training, which facilitates the complete transfer and alignment of capabilities between the teacher model and student model. Extensive experimental results and comprehensive ablation demonstrate the effectiveness of our ACDepth, demonstrating its superior performance compared to SoTA solutions.

Acknowledgments

This research was financially supported by the Natural Science Foundation of Heilongjiang Province of China for Excellent Youth Project (YQ2024F006), the National Natural Science Foundation of China (U23B2009) and Open Research Fund from Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) (GML-KF-24-09).