Enhancing Sampling Protocol for Point Cloud Classification Against Corruptions

1 Introduction

In the rapidly evolving field of 3D data perception via deep learning [Qi et al., 2017a; Qi et al., 2017b; Guo et al., 2020], point cloud sampling serves as a critical component in the standard learning and recognition pipeline [Hu et al., 2020; Qian et al., 2022; Yu et al., 2022; Zhang et al., 2022b]. Following the legacy of pioneer works [Qi et al., 2017a; Qi et al., 2017b], existing sampling protocols are primarily designed and optimized for clean data, without taking into account corruptions. However, due to the high complexity of real-world, point cloud data are almost always incomplete and with noise in practice [Ren et al., 2022], posing threats to 3D deep learning applications. For example, noisy background points or slight perturbations generated by inaccurate processing or sensor error can significantly decrease the deep model performance [Ren et al., 2022; Sun et al., 2022]. Such performance drops can lead to serious safety consequences, especially in critical 3D applications like autonomous driving. Therefore, it is necessary to rethink and redesign point cloud sampling protocols with a focus on robustness against corruptions to ensure reliable 3D deep learning in real-world conditions.

∗Corresphonding author

One crucial limitation of established sampling protocols is that they are sub-optimal under the corrupted data distribution. For instance, the protocol samples a fixed number of points in data preparation, namely, Fixed Sample Size (FSS) [Qi et al., 2017a]. This convention overlooks the facts that point clouds in the real world naturally vary in size and density. These varied sizes are even obvious in particular corruptions such as occlusions and density-related noise [Sun et al., 2022; Ren et al., 2022]. The misclassfication results on various corrupted data are illustrated in Figure 1. Another aspect is that the widely used Farthest Point Sampling (FPS) [Eldar et al., 1997] for key points selection is especially vulnerable to outliers due to its inherent basis of Euclidean distance and sensitivity to sparse points [Yan et al., 2020]. Several works have considered updating a specific step to deal with this issue, like PointASNL [Yan et al., 2020] and ADS [Hong et al., 2023]. The learning-based methods put extra effort into module training and may be potentially overfitting. Overall, none of them propose a comprehensive and alternative solution to overcome the sampling protocol limitations.

To overcome these limitations, we propose an enhanced point cloud sampling protocol, PointSP, by revising key points selection and full points resampling.The implementation of the proposed protocol involves randomizing the sampling during training and processing noisy point clouds during inference. To achieve this, the key point sampling process assisted by point reweighting is applied to ensure that potential outliers are not captured. The point weight is named as isolation rate evaluating the extent of local isolation for a point. Moreover, the proposed full points resampling randomizes the sample size during training; and restores insufficient point clouds in the inference stage. We achieve an local-global-balanced downsampling offering a continuous spectrum between local and global sampling. Inspired by shape-invariant perturbation [Huang et al., 2022], we realize an lightweight upsampling via a tangent plane interpolation technique that enhances the density of point cloud data while preserving local geometry.Overall, the enhanced sampling protocol is learning-free thus straightforward to implement and can be seamlessly integrated into the existing point cloud analysis pipeline.

Our contributions are summarized as follows:

• We first comprehensively revisit the long-existing sample protocol for point cloud learning through the lens of data corruption. Based on the analysis, we propose an alternative protocol to enhance the robustness of point cloud learning.
• We develop three learning-free techniques as the key of protocol, point reweighting, local-global balanced sampling, and local-geometry-preserved interpolation which can deal with point cloud corruption in different aspects. The whole proposed protocol is free of model architecture change and extra learning, thus it can be implemented to replace the current protocol with minimal pains and fits almost all 3D deep models.
• Extensive experiments are conducted on synthesis and real corrupted 3D point cloud datasets. The results have demonstrated that the proposed protocol is able to improve the robustness of 3D point cloud classification and outperform the latest methods.

2 RelatedWork

Point Cloud Sampling. Point cloud sampling techniques typically consist of: 1) downsampling, also known as "simplification" [Dovrat et al., 2019], and 2) upsampling [Zhang et al., 2022b]. These techniques are divided into non-learning and learning-based methods [Zhang et al., 2022b]. Traditional non-learning-based downsampling techniques include Farthest Point Sampling (FPS) [Eldar et al., 1997], Random Sampling (RS) [Hu et al., 2020], Poisson Disk Sampling (PDS) [Ying et al., 2013], and voxelization [Lv et al., 2021]. Conversely, learning-based downsampling methods account for downstream tasks [Dovrat et al., 2019; Lang et al., 2020; Qian et al., 2020; Qian et al., 2023]. Upsampling is categorized into learning-based [Yu et al., 2018; Qiu et al., 2022; He et al., 2023a; He et al., 2023b] and non-learning-based approaches [Alexa et al., 2003; Huang et al., 2013; Wu et al., 2015a]. The non-learning-based sampling techniques are particularly susceptible to outliers due to their inherent structural limitations; meanwhile, the learning-based methods are either also sensitive to noise or dependent on downstream tasks and prone to overfitting. This paper introduces simple yet effective point cloud sampling techniques to overcome these challenges.

Point Cloud Classification. PointNet [Qi et al., 2017a] has been a trailblazer in utilizing deep learning for point cloud analysis, with notable extensions such as PointNet++ [Qi et al., 2017b], GDANet [Xu et al., 2021], Point Transformer (PCT) [Guo et al., 2021], and CurveNet [Xiang et al., 2021]. However, the performance of these models significantly deteriorates with corrupted real-world data [Uy et al., 2019; Ren et al., 2022; Sun et al., 2022]. To tackle this issue, existing literature offers three main types of solutions. The first focuses on modifying the model by altering its structure or training strategies, such as pooling operations based on sorting [Sun et al., 2020] and model aggregation [Dong et al., 2020]. The second type includes certified methods, exemplified by Pointguard, which theoretically enhances model robustness through certified classification [Liu et al., 2021]. The third type is data-driven approaches that directly cleanse corrupted data, with notable methods including IF-defense [Wu et al., 2020] and DUP-Net [Zhou et al., 2019]. This paper aims to advance robustness from a new perspective by refining point cloud sampling protocol during data preparation, while the non-trivial architecture modification is avoided.

Point Cloud Data Augmentation. Point cloud augmentation is a widely recognized practice in the deep learning community, employed to improve the generalization capabilities of neural networks. Traditional augmentation methods, including random scaling, rotation, and jitter, are somewhat limited in their effectiveness for point cloud analysis [Zhu et al., 2024]. Recent advancements have introduced sophisticated techniques such as PointCutMix [Zhang et al., 2022a], PointAugment [Li et al., 2020], PointMixup [Chen et al., 2020], and PointWOLF [Kim et al., 2021]. However, they suffer from various limitations. For instance, while PointMixup [Chen et al., 2020] and PointWOLF [Kim et al., 2021] largely rely on predefined transformations, PointAugment [Li et al., 2020] emphasizes global transformations, often at the expense of local geometric details. To our knowledge, the sampling augmentation of point cloud for robust classification has been largely unexplored. In this work, we aim to enhance sampling protocols specifically tailored for robust point cloud classification, addressing this critical gap.

3 Proposed Sampling Protocol

3.1 Existing Sampling Protocol

Mainstream 3D point cloud classification frameworks employ a standardized protocol predicated on processing clean point clouds of predetermined size, e.g., 1,024. Formally, the input point cloud is defined as P = {pi}Ni=1 , where each point pi ∈ R3 and N represents a fixed size. A common technique of this protocol is the Farthest Point Sampling (FPS) algorithm, which identifies key points that serve as anchors for subsequent feature aggregation or pooling operations as in [Qi et al., 2017b]. FPS iteratively constructs a subset S ⊆ P by selecting points st according to:

where t = 1, · · · , |S| denotes the iteration index.

This conventional protocol, however, exhibits significant limitations when confronted with real-world scenarios where point clouds are invariably corrupted and of variable size. We can characterize a corrupted point cloud P ′ through the following formulation:

where Ps represents points removed from the original clean cloud P, while O comprises introduced noise points or outliers, typically arising from occlusion effects and sensor imperfections. This formulation exposes three critical vulnerabilities in the current protocol:

• FPS inherently selects outliers (O) as key points due to its distance-based criterion.
• Missing points (Ps) lead to information loss without compensatory mechanisms.
• The variable size of Ps and O violate the fixed-size input constraint.

Recent empirical studies [Ren et al., 2022; Sun et al., 2022] have confirmed these shortcomings, showing substantial performance degradation under corruption scenarios. This necessitates a fundamental redesign of the sampling protocol to enhance robustness in point cloud processing.

3.2 New Sampling Protocol

We propose the enhanced and learning-free sampling protocol towards key points selection and full points preprocessing, respectively. The new point cloud sampling protocol (PointSP) is visualized in Figure 2.

Key Points Sampling. We propose distinct sampling strategies for training and inference stages to select key points. During training, we employ a stochastic sampling approach based on point-wise weights by st ∼ Cat(P;w), where Cat(·; ·) denotes the categorical distribution, and w = {w1, · · · ,wN} represents the point weights. These weights are derived from the isolation rate of each point (detailed in Section 3.3), with more isolated points receiving lower weights to reduce their sampling probability. During inference, we introduce Filtered FPS (FFPS), a deterministic sampling method that modifies the conventional FPS by incorporating binary weights:

where ˆwi is a binary weight determined by thresholding point weights at the ω-th quantile (typically ω = 0.95) of all weights. This effectively filters out the most isolated points, which are often outliers or noise. The binarization simplifies implementation while maintaining effectiveness. Detailed weighting calculation is presented in Section 3.3.

Full Points Resampling. Our resampling protocol differentiates training and inference stages. During training, we deliberately introduce size variations to enhance model robustness against real-world point clouds often with non-uniform sizes. Given an input point cloud P with N points, we randomly adjust its size to N +∆N, where the random variable ∆N can be either positive (upsampling) or negative (downsampling). The modified point cloud ˜ P is obtained through:

During inference, our protocol ensures consistent point cloud size by applying conditional upsampling. Specifically, for any input point cloud P with insufficient points (i.e., |P| < N), we supplement it with additional points via upsampling to reach the target sizeN, namely, ˜ P = P∪∆P. The specific techniques for generating additional points (∆P) in upsampling and downsampling are detailed in Section 3.3.

3.3 Proposed Sampling Techniques

We introduce three key techniques of the new sampling protocol, namely, designed downsampling and upsampling techinques for full points resampling, and the point reweighting techinque for key point sampling.

Point Reweighting. The point-wise weight in key points sampling can be defined by the concept of Isolation Rate. At first, we calculate the radius of a sphere containing k nearest neighbors of each point in P, which is given by ri = maxqj∈N k i ∥pi -qj∥2 , where N k i ⊆ P is the set of k neighbors of i-th point pi. We further define Isolation Rate for each point as w′i , given by,

where ¯ r = Median({ri}Ni=1 ) is the median of all radius and PrY (X) is the probability of X given condition Y . The isolation rate of a point suggests the extent of a point being isolated, i.e., far from others in a probability way. Although a few associated concepts were proposed to calculate the exact local radius of points [Sotoodeh, 2006] and identify outliers, the isolation rate is naturally fit for point weighting due to the probability representation. In our key points sampling, we implement wi = 1 - w′i and ˆwi = 1(w′i ⩽ ω) for training and inference, respectively, where 1 is the indicator function.

Downsampling: Local-global-balanced downsampling. For ∆N ⩽ 0 in (4), we remove points through a flexible neighborhood-based approach. Specifically, we randomly select a center point pi and consider its ˜ k nearest neighbors N ˜k i , where ˜ k is randomly sampled from [|∆N|,N]. The points to be removed are selected according to:

Algorithm 1 Local Geometry Preserved Interpolation

Require: Point cloud P, Point cloud normal {ni}Ni=1 , A query point pi ∈ P, Integer k

Ensure: A new point ˆpi

• 1: N k ← kNN(P, pi, k) ▷ k nearest neighbors of pi
• 2: δmed ←Median({∥pi -q∥ : q ∈ N k }) ▷ Compute the median of local distances as interpolation norm.
• 3: q∗ ∼N k ▷ Sample a random neighbor
• 4: vi ← (I -nin⊤ i ) · (q∗ -pi) ▷ Compute the interpolation direction on tangent plane.
• 5: ˆpi ←pi + δmed vi ∥vi∥ ▷ Create interpolated point
• 6: return ˆpi.

4 Experimental Studies

4.1 Experimental Setup

Dataset and Model. We utilize models trained on ModelNet40 [Wu et al., 2015b] to conduct experiments on three corrupted datasets: ModelNet40-C, PointCloud-C, and OmniObject-C. The ModelNet40-C [Sun et al., 2022] and PointCloud-C [Ren et al., 2022] are datasets applying 15 and 7 distinct corruptions to ModelNet40's test set, totaling 2,468 objects. The OmniObject-C, based on OmniObject3D [Wu et al., 2023], has real-scanned 362 objects corrupted by the methods proposed in [Ren et al., 2022]. For 3D deep models, we employ PointNet [Qi et al., 2017a], PointNet++ [Qi et al., 2017b], GDANet [Xu et al., 2021], CurveNet [Xiang et al., 2021], PCT [Guo et al., 2021], following the pipeline in ModelNet40-C including batch size and training protocol. We note that all experiments are run on NVIDIA GeForce

This strategy offers a continuous spectrum between local and global downsampling: when ˜ k = |∆N|, it removes a concentrated local patch; when ˜ k = N, it performs global random sampling; and intermediate values of ˜ k provide balanced local-global downsampling. This flexibility helps simulate various real-world point cloud corruptions.

Upsampling: Local-geometry-preserved Interpolation. For ∆N > 0, we propose a interpolation method that generates new points. For each point, we perform interpolation between it and a randomly selected neighbor on their shared tangent plane, as detailed in Algorithm 1. This approach ensures the preservation of local geometric features, particularly surface normals, while increasing point density. The upsampled points are sampled from the interpolation set ˆ P by

This interpolation-based strategy effectively balances point cloud densification with geometric preservation, resulting in more natural and structurally coherent augmented data.

Computational Cost. We design the sampling protocol in the way that poses minimal computational effort beyond the original protocol. Particularly, the implementation of FPS in conventional protocol involves the calculation of point paired distances with complexity of O(N 2 ). The proposed point reweighting and interpolation (for all points) techinques can utilize the same paired distances and induce extra operations with O(kN) complexity. Such extra computation effort is minor since k ≪ N.

RTX 3090 GPUs.

Parameters Setting. The number of nearest neighbors k used in point weight computation is set to 20, which follows the common setup. During the inference phase, the key points sampling applies a threshold ω of 0.95, exploring the learning as depicted in Figure 4, meaning that FFPS filters out points within the lowest 5% of point weights.

Evaluation Protocol. We report the error rates (ER) and mean error rates (mER) across multiple corruptions on the three corrupted datasets for performance evaluation. A smaller ER indicates a superior performance.

4.2 Main Results

Overall Results. Mean error rates (mERs) for the three corrupted datasets are presented in Table 1. To facilitate a comprehensive comparison, we include multiple baseline models. The results clearly indicate that the proposed PointSP significantly enhances PCT and CurveNet; mERs decrease by approximately 10% across all datasets, with the most substantial improvement observed in PointCloud-C.

Results on ModelNet40-C. Extensive evaluations of PointSP on ModelNet40-C utilizing five 3D deep models revealed its superiority. Compared to five enhancement techniques (CutMix-R [Zhang et al., 2022a], CutMix-K [Zhang et al., 2022a], Mixup [Chen et al., 2020], Rsmix [Lee et al., 2021], and PGD [Sun et al., 2021]), PointSP significantly improved all models. Across multiple corruption types, PointSP consistently achieved the lowest error rates: 24.1% for "Density", 9.5% for "Noise", and 11.1% for "Transform", demonstrating robustness. Notably, its unique randomized size sampling in resampling and FFPS in downsampling effectively tackled "Density" and "Noise" corruptions, enhancing resilience and eliminating outliers, respectively.

Results on PointCloud-C and OmniObject-C. Table 3 compares the performance of PointSP with data augmentation methods on the PointCloud-C and OmniObject-C datasets. On PointCloud-C, PointSP enhances classification accuracy across various corruption scenarios, outperforming other methods, with the FFPS technique achieving an error rate of 7.5% under additive corruption types due to its effective outlier filtering. However, PointSP does not perform optimally under the "Jitter" corruption, where the PGD strategy excels due to its robust feature learning mechanism. For drop-type corruptions, methods like CutMix and Rsmix demonstrate superior robustness, likely due to their data-mixing strategies. On OmniObject-C, PointSP excels in improving outof-distribution (OOD) robustness, achieving the lowest mER for CurveNet (57.9%) and the best results for "Jitter" (61.4% mER). It also outperforms other methods on PointNet++ for "Drop-G" and "Add-G" corruptions, and is highly competitive with CutMix-R on PCT. Overall, these results validate PointSP's effectiveness in enhancing both OOD robustness and generalization.

4.3 Results for Part Segmentation

The proposed sampling protocol has been evaluated on a classification task. To demonstrate its broader applicability, we also applied it to part segmentation tasks, which are critical for robotic manipulation, using the ShapeNet-C dataset [Ren et al., 2022]. The results, shown in Figure 3, clearly indimethods across three 3D deep models.

cate that the proposed PointSP protocol provides a significant improvement.

4.4 Ablation Studies

In the ablation studies, we use PCT [Guo et al., 2021] and CurveNet [Xiang et al., 2021] as 3D deep models on two datasets: ModelNet40-C [Sun et al., 2022] and PointCloud-C (PCC) [Ren et al., 2022].

Sampling Protocols in Training. We compared various sampling protocols during the training phase. As shown in Table 4, the combination of random up&downsampling (RUD) and stochastic weighted sampling (SWS) consistently delivers the best performance. Removing RUD or substituting the proposed SWS method with FPS obviously degrades the performance. FFPS also achieves the second-based performance, highlighting the importance of point reweighting.

Random Upsampling and Downsampling. We investigate the impact of various resampling techniques during training. As shown in Table 5, we explore different methods for increasing and reducing sample sizes. The results indicate that the proposed local-glocal-balanced method plays a crucial role in enhancing performance. This suggests that stochastically determining the localness of the point dropping, i.e, neighbor size, can improve robustness against corruptions. Additionally, we compare our upsampling method in Algorithm 1 with Shape-invariant perturbation (SI), which conducts per-point perturbation on the tangent plane [Huang et al., 2022]. The superiority of our method over SI shows that perturbation direction based on neighbor points preserves more local information than random directions.

Effect of Quantile-based Threshold ω of FFPS. As illustrated in Figure 4, retain 95% of the points (i.e., filtering out the 5% of points with the smallest weights) results in the best performance. Moreover, when nearly half of the points are removed, the mER peaks, likely due to the loss of critical information within the point cloud. As the number of removed points decreases, the error rate also decreases, reaching its weights lowest when about 5% of the points are filtered out, before rising again. It suggests that an optimal balance is achieved.

Figure 4: mERs of FFPS's different threshold ω on ModelNet40-C (left) and PointCloud-C (right).

4.5 Visualization Results

Isolation Rate. In Figure 5, we visualize the distribution of point-wise isolation rates for three example objects. The proposed rate effectively identifies boundary points and outliers, thereby enhancing subsequent point cloud sampling and improving learning robustness against corruption.

Local-geometry-preserved Interpolation. Figure 6 visually compares the results of three upsampling techniques on four example objects. It is evident that both Jitter and SI [Huang et al., 2022] struggle with corrupted data, particularly when it is sparse and non-uniform. In contrast, the proposed LGP method effectively combines completion and uniformity in the upsampling process.

Neighborhood Size ˜ k in Local-global-balanced downsampling. Stochastically determining the sample size is a critical aspect of the resampling protocol. As shown in Figure 7, a smaller ˜ k leads to local drops (second row), while a larger ˜ k results in more global removals (last row). A stochastic ˜ k would closely mimic real-world corruption, contributing to the robust improvement of the proposed protocol.

5 Conclusion

This work highlights the limitations of current sampling protocols for corrupted 3D point clouds and proposes PointSP, a robust solution that mitigates outliers and restores incomplete clouds without additional training. Experiments show that PointSP outperforms state-of-the-art methods in 3D classification, paving the way for more reliable 3D deep learning in critical applications like autonomous driving.

Acknowledgments

This research was partially supported by National Natural Science Foundation of China (Grant No. 62202395 & 62176221), Sichuan Science and Technology Program (Grant No: 2024NSFTD0036 & 2024ZHCG0166), and the Fundamental Research Funds for the Central Universities (Grant No: 2682025ZTZD013).