Reliable and Calibrated Semantic Occupancy Prediction by Hybrid Uncertainty Learning

1 Introduction

The goal of semantic occupancy prediction is to obtain a comprehensive voxel-based representation of the 3D scene from either LiDAR point clouds [Roldao et al., 2020; Xia et al., 2023] or camera images [Cao and de Charette, 2022; Huang et al., 2023; Li et al., 2023], which is crucial for perception systems in autonomous driving and robotic platforms. Initially, LiDAR-based models [Roldao et al., 2020;

* Corresponding authors.

Cheng et al., 2021; Yan et al., 2021; Xia et al., 2023] dominated the field due to their ability to provide accurate geometric cues. Researchers nowadays prefer to learn 3D occupancy information from images owing to the low cost and widespread availability of camera sensors. Recent progress [Cao and de Charette, 2022; Huang et al., 2023; Li et al., 2023; Yao et al., 2023; Mei et al., 2024] has significantly narrowed the gap in accuracy between camera and LiDAR-based approaches. However, their performance in terms of reliability remains under-explored, which becomes paramount in safety-critical scenarios.

Traditionally, the occupancy labeling relies on accumulated LiDAR scans and their corresponding point-wise semantic labels [Behley et al., 2019; Tian et al., 2024; Wang et al., 2023; Wei et al., 2023]. With the development of vision-centric approaches [Cao and de Charette, 2022; Huang et al., 2023; Yao et al., 2023; Li et al., 2023; Mei et al., 2024] using images, questions arise regarding the reliability of predictions solely derived from cameras without accurate depth information. Since the overall accuracy of occupancy networks is relatively low, exploring the reliability and uncertainty of their predictions can provide valuable reference information for downstream tasks in driving, such as decision-making and planning [Zheng et al., 2024; Hu et al., 2023; Albrecht et al., 2021].

With the above considerations, we conduct a thorough evaluation of existing semantic occupancy prediction models based on a reliability standpoint. To achieve this, we introduce the misclassification detection and calibration metrics from both geometric and semantic dimensions for evaluating model that utilize camera or LiDAR data. Our findings reveal that camera-based models often lag behind their LiDAR-based counterparts in terms of reliability despite improvements in accuracy.

To mitigate this disparity, RELIOCC is proposed to improve the reliability in occupancy networks by a new hybrid uncertainty learning scheme. Our approach optimizes uncertainty by taking into consideration of perturbations in individual voxels (absolute uncertainty) and the relative relationships in mix-up voxels (relative uncertainty) during model training. By integrating multiple sources of information for uncertainty learning, our method enhances the reliability of camera-based models without sacrificing inference speed or accuracy. Moreover, we provide an uncertainty-aware calibration strategy to utilize the learned uncertainty in offline mode, further enhancing model's reliability. Through extensive experiments across diverse configurations including online and offline modes, our method achieves competitive performance compared against the state-of-the-art models.

Our main contributions can be summarized as follows:

• A comprehensive evaluation is conducted on existing semantic occupancy prediction models from a reliability perspective, which provides a series of misclassification detection and calibration metrics across both geometric and semantic dimensions.
• We provide a systematic scheme for adapting the existing methods to achieve reliable and calibrated occupancy networks. Furthermore, RELIOCC is proposed to enhance the reliability of camera models. A novel hybrid uncertainty learning approach is presented to combine the variance from individual and mix-up voxels.
• Extensive experiments on online uncertainty learning and offline model calibration across diverse settings demonstrate the effectiveness of our approach in bridging the reliability gap between camera and LiDAR-based methods under general conditions, while showcasing robust performance in adverse scenarios such as sensor failures and noisy observations.

2 RelatedWork

Semantic Occupancy Prediction. Semantic occupancy prediction is also known as semantic scene completion and firstly explored in indoor scenes [2017; 2022]. In outdoor scenarios, SemanticKITTI [2019] stands as the first large-scale dataset, providing abundant data resources. Recently, several other datasets [2022; 2023; 2023; 2024] have been constructed to explore this task owing to its importance. LiDARbased methods [2020; 2021; 2021; 2022; 2023] have dominated this field in accuracy. MonoScene [2022] is the first occupancy prediction method that utilizes single image as input. Subsequent studies [2023; 2023; 2023; 2024; 2024; 2024] have effectively improved the performance of camerabased models. However, there is a lack of research on reliability of occupancy predictions, posing potential risks to the safety of downstream tasks in driving [2021; 2023; 2024]. RELIOCC fills this gap by investigating the reliability of occupancy networks by uncertainty learning.

Uncertainty Learning and Model Calibration. The uncertainty in machine learning consists of aleatoric uncertainty from data noises and epistemic uncertainty from model parameters [2017; 2018]. Data uncertainty is widely explored in face field [2017; 2019; 2020; 2021]. Cai et al. [2023] propose a probabilistic embedding model to estimates the data uncertainty for point cloud. Model uncertainty is usually obtained from the statistics of multiple predictions through methods including model ensembling [2017], bootstrapping [2001], and bagging [1996]. Model calibration is another line to improve reliability in model prediction [2023], which provides a post-processing scheme applied to the non-probabilistic output from a trained model. Model calibration was initially studied in image classification [2017; 2019; 2021] and has since been widely applied to object detection [2020; 2022] and semantic segmentation [2021; 2025]. Our method adopts uncertainty as a learning objective and can support both online uncertainty estimation and offline model calibration simultaneously.

3 Preliminaries

3.1 ProblemFormulation

Occupancy Prediction. Given inputs x from LiDAR or camera sensors, occupancy networks Vθ(x) generate dense features V ∈ Rd×L×W×H in a pre-defined volume, where L, W, and H represent the length, width, and height, respectively. d is the dimension of dense features. For any voxel vi ∈ Rd within this volume, the prediction involves with two components. One is a binary indicator that specifies whether the voxel is occupied or not. The other is the semantic label of the voxel if the voxel is occupied. Generally, such process can be formulated by estimating the probability p(yi = y|vi) for vi, where y ∈ {0, 1, ..., S}. Here, 0 denotes that the voxel is unoccupied, and S is the total number of semantic classes. Misclassification Detection. For a reliable classifier, we expect it to accurately reject those incorrect predictions with low-confidence. Therefore, misclassification detection is introduced to measure the gap between the actual trained model and the ideal one, which can be evaluated by rejection curves [Fumera and Roli, 2002; Hendrycks et al., 2021]. To avoid the tendency of higher precision models, we adopt the same strategy as [Malinin et al., 2019; de Jorge et al., 2023] to normalize the area under the curve and deducts a baseline score.

Calibration. As a long-standing problem in machine learning, the goal of model calibration is to ensure that predicted confidence of a model aligns accurately with the actual likelihood of correctness [Niculescu-Mizil and Caruana, 2005; Guo et al., 2017], thereby producing more reliable predictions. Within the framework of multi-class classification, a model is deemed perfectly calibrated if p(yi = y|ci = c, vi) = c. Here, the model not only predicts a discrete label y but also generates a confidence score c ∈ [0, 1]. This score c should ideally reflect the true probability that the predictions are correct.

3.2 EvaluationMetrics

Canonical Metrics. Following common practice [Song et al., 2017; Behley et al., 2019], we employ the Intersection over Union (IoU) metric to assess the accuracy of geometric occupancy prediction that is typically treated as a binary classification task. Additionally, we utilize the mean Intersection over Union (mIoU) across multiple categories to evaluate the quality of semantic predictions. These two metrics are calculated using discrete predictions by applying argmax operation to the logits. Although IoU and mIoU effectively reflect the performance of model in accuracy aspect, they do not assess its reliability.

In this paper, we mainly evaluate the reliability of occupancy prediction on misclassification detection and calibration, which are assessed by following metrics.

Prediction Rejection Ratio (PRR). The Prediction Rejection Ratio (PRR) [Malinin et al., 2019] is defined through rejection curves for misclassification detection. To construct a rejection curve, we initially sort predictions based on a specific criterion, such as predicted confidence or oracle confidence (where predictions are labeled 1 if correct and 0 otherwise). Subsequently, a threshold is set and predictions below this threshold are rejected, allowing us to calculate a rejection rate. As this threshold is incrementally adjusted, we obtain a rejection curve to illustrate how the classification error (depicted on the y-axis) decreases in tandem with the rejection rate (represented on the x-axis). The PRR metric is then quantitatively defined as follows

where AUC represents the Area Under the Curve. Here, AUCrandom = 0.5 corresponds to the AUC for randomly generated confidences. A perfectly reliable model would achieve PRR = 1. For occupancy networks, we report both PRRgeo for geometric predictions and PRRsem for semantic predictions, respectively.

Expected Calibration Error (ECE). Expected Calibration Error (ECE) [Naeini et al., 2015; Guo et al., 2017] assesses the calibration of probabilistic predictions made by machine learning models. It measures the difference between predicted probabilities and observed frequencies across various confidence levels. Intuitively,

A perfectly calibrated model yields eECE = 0. Eq. (2) is a continuous integration over c ∈ [0, 1]. Practically, we approximate this integration by discretizing c into M small bins. Denoting the set of samples falling into the m-th bin as Bm, the expectation can be calculated as

where acc(·) denotes the mean accuracy, and conf(·) is mean confidence of Bm. N is the number of samples. We set the number of bins M = 15 by default. As with PRR, we report both ECEgeo and ECEsem for geometric and semantic predictions, respectively.

4 Adaptation with ExistingMethods for Occupancy Networks

Reliable predictions are paramount in occupancy networks, especially in critical applications such as autonomous driving and robotics where safety is a strict requirement. Despite their importance, methods for enhancing the reliability and calibration of occupancy networks are still under-explored in the existing literature. To address this gap, we begin by reviewing existing uncertainty learning and calibration methods, which are mostly developed for traditional tasks. Then, we adapt them for the recent occupancy prediction networks.

We categorize these methods into two paradigms. One is training uncertainty predictor cσ|ϕ based on the dense features V concurrently with Vθ from scratch, which is termed online uncertainty learning. Another is training scaling factor cf|ϕ on top of a fixed Vθ, which is termed offline model calibration. In the experimental section (see §6.2 and §6.3), we provide extensive evaluations on these methods to compare their effectiveness in boosting the reliability.

4.1 Online Uncertainty Learning

Uncertainty estimation is a long-standing problem in the context of Bayesian deep learning [Tishby and Solla, 1989; Denker and LeCun, 1990; Gal and Ghahramani, 2016]. Prior arts can be classified into ones concerning epistemic (model) uncertainty [Lakshminarayanan et al., 2017; Jungo and Reyes, 2019] and ones concerning aleatoric (data) uncertainty [Kendall and Gal, 2017; H¨ullermeier and Waegeman, 2021]. Although explicit uncertainty estimates are obtainable, we do not directly evaluate these estimates in online mode. Instead, since the uncertainty is learned concurrently with the model's predictions from scratch, we use them as a regularization term to help the model become more reliable.

For each voxel feature vi, we compute a logit vector zi ∈ RS+1 using a linear layer, where S represents the number of semantic classes.

Heteroscedastic Aleatoric Uncertainty (HAU) [Kendall and Gal, 2017] is a data-dependent uncertainty learning method. We employ the classification form, which modifies upon a deterministic model by placing a Gaussian over the logit: ˆzi|ϕ ∼ N(zi, (σϕ i )2 )1 . The sampled logit vector ˆzi is then passed through a softmax operator and cross entropy loss is computed. Here, σϕ i is the predicted uncertainty parameterized by ϕ. Optimization of ϕ can be done with backpropagation using the re-parameterization trick [Kingma and Welling, 2013]: ˆzi = zi + σϕ i ϵ, ϵ ∈ N(0, I). Note that the uncertainty predictions vary for different voxel i.

Data Uncertainty Learning (DUL) [Chang et al., 2020] shares a similar spirit with HAU with two distinctions. Instead of using the logit ˆzi, DUL models the feature ˆvi as a Gaussian distribution by ˆvi = vi + σW i ϵ. Moreover, DUL introduces a regularization term in the loss function that minimizes the Kullback-Leibler (KL) divergence between the predicted Gaussian and a standard Gaussian.

MC Dropout (MCD) [Jungo and Reyes, 2019] is proposed to explore the epistemic (model) uncertainty. Differently from the above methods, MCD does not require additional parameters to learn uncertainty. Instead, it incorporates multiple dropout layers into the original network during training. For inference, the occupancy prediction of each voxel is obtained by ˆzi = 1K ∑Kk=1 zk,i, where zk,i is the model output at the k-th test. The normalized entropy of K predictions is adopted as the model uncertainty. To fully explore the uncertainty within the model, we set K = 40 in our experiments.

For above online uncertainty learning methods, the calibrated confidence is set as the softmax output of the sampled

1 We omit predicting the mean µϕ (zi) and use zi for simplicity. Empirical results are similar.

⋅

Figure 1: (a) Overview of proposed RELIOCC. Besides the original objective of an occupancy network, we introduce an uncertainty estimation branch and supervise it with absolute and relative uncertainty learning losses. (b) Absolute uncertainty learning. Deterministic logits are replaced with ones sampled from predicted distributions. (c) Relative uncertainty learning. We leverage the relative relationships between uncertainty pairs to further enhance uncertainty learning.

logit and then determined by taking the maximum probability across all classes: ci = maxs S(ˆzi)(s) , where S denotes the softmax function.

4.2 Offline Model Calibration

Offline calibration methods build on pre-trained Vθ and need to learn a scaling function, which typically employ the following formulation

where fϕ is the scaling function applied to zi parameterized by the learnable parameters ϕ. In the absence of explicit uncertainty estimation, uncertainty σi is set to 1 - ci.

Temperature Scaling (TempS) [Guo et al., 2017] employs a scalar parameter T, termed as temperature, to scale the logits zi, by fϕ(zi) = S( zi T ). T is data-independent, which is shared across all classes.

Dirichlet Scaling (DiriS) [Kull et al., 2019] assumes that the model's output follows a Dirichlet distribution. Based on this assumption, they propose the Dirichlet scaling, fϕ(zi) = S(W· log(S(zi))+b). Here, learnable parameters ϕ include weight W ∈ R(S+1)×(S+1) and bias b ∈ RS+1 .

Meta-Calibration (MetaC) [Ma and Blaschko, 2021] proposes to employ the entropy of model prediction -ci log(ci) to select different calibrators. Specifically, an identical fϕ(zi) = S( zi T ) as in TempS is used when -ci log(ci) is smaller than the predefined threshold η. Otherwise, the calibration function fϕ(zi) is set to the constant value 1 S+1 . MetaC introduces new randomness into predictions, which lead to variations in accuracy, making it less practical for safety-critical tasks such as occupancy prediction.

Depth-Aware Scaling (DeptS) [Ma and Blaschko, 2021] is an improved variant upon MetaC, which is specially designed for LiDAR segmentation. Depth di of each point or voxel is encoded into the calibration function fϕ by a linear mapping αi = k1 · di + k2, where k1 and k2 are learnable parameters. When prediction entropy -ci log(ci) is greater than the threshold η, fϕ(zi) = S( zi α·T1 ). Otherwise, fϕ(zi) =

S( zi α·T2 ) Both T1 and T2 are temperature parameters, where T1 is initially set higher than T2.

5 RELIOCC

Method Overview. Inspired by investigation on prior arts, we propose RELIOCC, a plug-and-play method tailored for the 3D semantic occupancy prediction task. RELIOCC has two main improvements over existing methods. Firstly, beyond traditional uncertainty learning, we propose to utilize the relative relationships between voxel pairs of uncertainty estimates in order to further refine the uncertainty estimation process. Secondly, we present a unified framework that integrates the methodologies of uncertainty learning with scalingbased calibration, which demonstrates that their synergy offers substantial benefits. As shown in Figure 1(a), we predict a scalar uncertainty σi from a voxel feature vi using an MLP, which receives supervision from both the individual and relative voxels.

Absolute Uncertainty. By absolute here we mean the uncertainty σi is only determined by the individual vi itself while ignoring relative relations between a pair of vi and vj. We adopt a similar formulation as in HAU [Kendall and Gal, 2017]. Specifically, We randomly sample the logit ˆzi based on the predicted uncertainty σi by ˆzi = zi +σiϵ, ϵ ∈ N(0, I) as illustrated in Figure 1(b). We denote this absolute uncertainty loss as Lau, which is computed with the re-sampled logit ˆzi and its corresponding ground truth.

Relative Uncertainty. A potential drawback of absolute uncertainty is that the optimization of σi tends to plateau once it reaches a small scale. To address this issue, we introduce the concept of relative uncertainty learning for occupancy prediction. The fundamental principle of relative uncertainty learning involves enforcing comparisons between uncertainty pairs vi and vj . This approach ensures that optimization does not plateau, even when σi and σj are small.

Concretely, we shuffle voxel features in V, paired the shuffled features with the original ones and obtain random pairs (vi, vj) at each iteration. Inspired by the mix-up [Zhang et al., 2018] learning principle, we blend the paired voxel features with ˆ v = λvi +(1-λ)vj . Correspondingly ˆ v is trained with a blend of the label pairs using cross-entropy loss. The blended label y = yi + yj, where yi, yj are one-hot label encodings 2 . We employ the predicted uncertainty for the weighting: λ = σi σi+σj ∈ [0, 1]. We denote the loss computed with blended label and mixed output from the shared completion head as the relative uncertainty loss Lru, as shown in Figure 1(c).

An intuitive understanding emerges when considering that relative uncertainty adaptively modulates the learning dynamics between the feature pair (vi, vj). Specifically, if the model exhibits greater confidence in the prediction associated with vi, it suffices for the mixup feature to incorporate a smaller portion of vi while still achieving a reduced loss Lru. In contrast, a lower confidence in vj necessitates a greater inclusion of vj within the mixup to diminish the loss. Consequently, this process enables the model to effectively differentiate between the uncertainties σi and σj, typically resulting in a smaller σi and a larger σj. Importantly, this differentiation does not hinge on the absolute magnitudes of σi,j. Rather, it is the relative relationship between them that is central to the learning process. In driving scenarios, there are rich relative relationships between voxels, including distance, occupancy, surface and interior properties. This focus on relative differences ensures that the model's adjustments are robust to the absolute scales of the uncertainties.

Uncertainty-Aware Calibration. Using the above uncertainty estimation objectives, RELIOCC is capable of learning uncertainty with existing occupancy models in an online setting. We introduce a scaling-based calibration objective to make it also compatible with the offline one. A variant form of TempS [Guo et al., 2017] is adopted, and the uncertaintyaware temperature Tσ is a linear transform of σi:

where k1 and k2 are learnable parameters, and b is the bias. W is initialized as the identity matrix and only the elements on the diagonal are optimized. The calibration loss is denoted as Lcalib.

Training and Inference. RELIOCC supports both online uncertainty learning and offline model calibration settings. In the online setting, the uncertainty predictor is trained concurrently with the occupancy network from scratch. The total loss function comprises Locc, Lau, and Lru. Here, Locc represents the primary loss for the occupancy network. During inference, the model operates consistently with the original network design. In the offline setting, the occupancy network is well trained and frozen, eliminating the need for Locc and introducing the calibration loss Lcalib instead. The inference process incurs a minimal increase in computational overhead due to the addition of the calibrator.

2 Different from the original mix-up [Zhang et al., 2018] paper, we omit weighting the labels with λ for stable training.

6 Experiments

6.1 Benchmark Results

Datasets and Evaluation. SemanticKITTI [Behley et al., 2019] is the first large-scale outdoor dataset for semantic occupancy prediction containing 64-beam LiDAR scans and front camera images as inputs [Geiger et al., 2012]. The dataset comprises 22 sequences, where 00-10 (excluding 08) are used as the training set, 08 is the validation set, and 11-21 are the test set. Since the ground truth for the test set is not publicly available, we cannot measure our newly introduced metrics on it. Therefore, we primarily evaluate the existing methods on the validation set (val.). As described in §3.2, mIoU and IoU are used to measure the model's accuracy in semantic and geometric completion, respectively. For misclassification detection and calibration metrics including PRR and ECE, we also report the corresponding results from both geometric and semantic perspectives.

Re-evaluated Methods. We reproduce and evaluate existing publicly available methods on the SemanticKITTI benchmark, including five LiDAR-based models [Song et al., 2017; Roldao et al., 2020; Yan et al., 2021; Mei et al., 2023; Xia et al., 2023] and five camera-based models [Cao and de Charette, 2022; Huang et al., 2023; Li et al., 2023; Yao et al., 2023; Mei et al., 2024]. All results are obtained using the official implementation and the configurations are kept consistent for inference. As shown in Tab. 1, we find that although the accuracy of camera-based methods has been continuously improved and gradually approaches the baseline accuracy of LiDAR methods, their reliability metrics, particularly the ECE, have not shown corresponding improvements. In cases of lower accuracy compared to LiDAR, the camerabased models' reliability is also quite poor, which undoubtedly poses significant safety risks for autonomous driving.

SGN Framework

15.52

44.72

5.69

15.52

46.90

2.68

15.52

48.20

2.61

14.71

46.38

4.06

15.52

45.45

2.14

15.52

47.40

2.09

45.45

39.78

4.85

45.45

37.25

2.35

45.45

43.04

2.51

40.37

37.97

3.65

45.45

34.99

1.42

45.45

43.80

2.43

Table 3: Quantitative results of offline model calibration (§6.3) on SemanticKITTI (validation set).

6.2 Online Uncertainty Learning

Base Architectures and Competing Methods. Considering the potential applications of camera-based methods and their current limitations, we adopt the state-of-the-art vision-based methods including VoxFormer [Li et al., 2023] and SGN [Mei et al., 2024] as our base architectures to conduct relevant experiments. We report the results of some existing methods on the two baseline frameworks for comparison, including HAU, DUL, and MCD (see §4.1).

Implementation Details. We follow the original training setting and add additional uncertainty learning parameters without altering the network structure. The inputs consist of the current image from the left camera and previous 4 frames. The image size is copped into 1220 × 370. VoxFormer [Li et al., 2023] and SGN [Mei et al., 2024] with online uncertainty

estimation are trained for 20 epochs and 40 epochs, respectively. The loss coefficients (α, β) for Lau and Lru are set to (4.0, 6.0) for both frameworks.

Evaluation Results. In Tab. 2, we provide the comparison among the methods with the same framework for fairness. Compared to data uncertainty-based HAU and DUL, as well as model uncertainty-based MCD, our method shows significant improvements in the new evaluation metrics for reliability. The calibration errors (ECE) in both semantic and geometric aspects are significantly reduced compared to the existing uncertainty estimation methods. The improvement in PRR also indicates a notable enhancement in model reliability. Our method maintains stability in terms of the original accuracy (mIoU and IoU) across the two different frameworks SGN

RELIOCC (Ours)

Figure 2: Visual results of the error map and uncertainty map from the prediction by SGN and RELIOCC. In the uncertainty map, a closer proximity to yellow indicates a higher level of uncertainty.

and even surpasses LiDAR-based methods in PRR. Furthermore, we visualize the error map with corresponding uncertainty map of SGN and our approach. The uncertainty for vanilla SGN is obtained by subtracting the confidence from 1. As shown in Figure 2, when the network's predictions exhibit large areas of error, SGN's uncertainty map still shows low uncertainty, indicating high confidence in prediction. In contrast, our proposed RELIOCC displays high uncertainty in most of the error regions, providing more reliable information for downstream tasks.

6.3 Offline Model Calibration

In this section, VoxFormer and SGN are also adopted as baseline frameworks. We primarily compare our method with scaling-based model calibration approaches including TempS, DiriS, MetaC, and DeptS (see §4.2).

Implementation Details. We select the best-performing model checkpoints on the validation set from the pre-trained VoxFormer and SGN as the targets for calibration. During the calibration process, the parameters of the original network are frozen, and only the parameters ϕ in the calibration function fϕ and uncertainty learning layers are trainable. For both frameworks, these methods are trained on 8 GPUs for 20 epochs with a learning rate as 0.001 and AdamW optimizer [Zhao et al., 2022]. The batch size is set to 1 per GPU. For our method, the loss weights (α, β, γ) for uncertainty learning (Lau, Lru) and model calibration (Lcalib) are set to 1.5, 1.0, and 4.0, respectively.

Evaluation Results. As illustrated in Tab. 3, all calibration methods demonstrate improvements compared to the baselines, particularly in calibration error (ECE). MetaC [Ma and Blaschko, 2021] loses the characteristic of maintaining accuracy in calibration due to the introduction of new random classifications. Our approach with uncertainty-aware design achieves competitive performance on both ECE and PRR metrics without depth information even compared with the stateof-the-art DeptS [Kong et al., 2025].

6.4 Diagnostic Experiments

Ablation of Online Uncertainty Learning. We provide ablation experiments on the effect of absolute uncertainty and relative uncertainty during the whole model training. The experiments are conducted with VoxFormer [Li et al., 2023] on the validation set of SemanticKITTI. As shown in Tab. 4, the first row presents the baseline results. The inclusion of individual absolute uncertainty and relative one both contribute to the improvement of the model's reliability, albeit with a modest enhancement. When our proposed hybrid uncertainty learning module is incorporated, the PRR and ECE metrics of model's prediction achieve the best results.

Ablation of Offline Model Calibration. Further ablations are also conducted in offline mode. With the pre-trained VoxFormer, we found that employing standard scaling strategies such as TempS can achieve good calibration results as illustrated in second row of Tab. 5. However, it impacts the improvement of misclassification detection metrics (PRR) and even leads to a decline in geometry. Our introduced relative uncertainty learning can further improve calibration performance and enhance misclassification detection. Moreover, the combination of absolute and relative uncertainties achieves the best performance in misclassification detection, although it is marginally less effective in calibration due to the distinct focus of the PRR and ECE metrics.

Robustness Analysis. Figure 3 presents the robustness analysis results of RELIOCC compared to the baseline model SGN [Mei et al., 2024]. We simulate four potential out-ofdomain scenarios during the inference, including sensor failures (frames drop), strong sunlight, foggy and rainy conditions, to evaluate the model's robustness [Dong et al., 2023]. Each adverse scenario provides weak(w) and strong(s) modes of perturbation. As the noise increases in various conditions, our method not only maintains stability in reliability metrics but also demonstrates more improvement in accuracy compared to the baseline.

7 Conclusion

In this paper, we address the issue of assessing reliability in semantic occupancy prediction for the first time. The reliability is evaluated from two aspects including misclassification detection and calibration. Extensive evaluation on existing LiDAR and camera-based methods is provided. Besides, we propose a new scheme RELIOCC that integrates hybrid uncertainty from the individual and relative voxels into existing occupancy networks without affecting accuracy or inference speed. Both online and offline modes are designed to illustrate the generalization capability of our learned uncertainty. Extensive experiments are conducted under various settings, demonstrating RELIOCC is effective in improving the reliability and robustness of semantic occupancy models.

Acknowledgments

This work is supported by National Natural Science Foundation of China (Grant No. 62376244) and Program of China Scholarship Council (Grant No. 202406320418). It is also supported by Information Technology Center and State Key Lab of CAD&CG, Zhejiang University.