Rotation Invariant Spatial Networks for Single-View Point Cloud Classification

1 Introduction

Point cloud is three-dimensional (3D) data collected by depth devices, which provides a direct representation of the environment and objects [Qian et al., 2022; Liu et al., 2023]. Point cloud processing, including techniques such as classification and segmentation, is a fundamental requirement for unmanned systems to understand 3D scenes [Fei et al., 2022; Fan et al., 2023].

Since the introduction of PointNet [Qi et al., 2017a], learning-based architectures have emerged with the ability to directly process point cloud data. Recently, an increasing number of studies have employed deep neural networks for processing point clouds to achieve high-precision classification [Wei et al., 2023; Li et al., 2018; Liang et al., 2024; Lin et al., 2022; Qiu et al., 2022; Yu et al., 2022]. However, most methods presume a complete, prescribed pose point cloud as network input. In the real world, the captured point cloud is usually single-viewed and partial. For example, single-view and arbitrary pose features of objects in the real world are shown in Fig. 1 (a) and (b).

∗Corresponding author

Applying rotational enhancement during training could alleviate the impact of arbitrary poses on classifiers [Qi et al., 2017a; Qi et al., 2017b; Wang et al., 2019]. However, due to the large number of degrees of freedom for 3D data, the effect of the above rotational enhancement is limited. Rotation-invariant models are key to accurately classifying objects for point clouds, which focus on designing features with rotational invariance [Poulenard et al., 2019; Chen et al., 2019; Deng et al., 2018; Zhang et al., 2022; Zhao et al., 2022]. However, the performance of these methods tends to be suboptimal compared to translationally invariant methods, since a portion of spatial information of objects is often ignored or lost during the construction of rotationinvariant features.

The single-view point cloud data of objects with arbitrary poses results in a performance decline of classification, as presented in Fig. 1 (c). To solve this problem, we propose a rotation-invariant network for point cloud classification. Specifically, a RISpaNet block is designed for extracting rotation-invariant features from point clouds, which includes a rotation-invariant property generation module (RIPGM), a pooling module and a hybrid encoder. In RIPGM, a highly expressive rotation-invariant tetrahedron property (RITP) is developed. RITP is constructed as a local tetrahedron centered on a reference point to efficiently capture spatial structure of objects. The pooling module is used to extract multiscale features. A hybrid encoder is designed to generate rotation-invariant features through different encoding mechanisms including convolution, attention mechanisms and state space models. For single-view point clouds, a complete point cloud auxiliary branch and a part-whole correlation module are designed to obtain complete point cloud features from partial point clouds.

To summarize, our main contributions are:

• Rotation-invariant Tetrahedron Properties. A highly expressive rotation-invariant property is designed by constructing local tetrahedra to capture spatial information of objects.
• RISpaNet. RISpaNet is a fundamental block in our model. This block includes a rotation-invariant property generation module, a multi-scale pooling module and a hybrid encoder. Different coding mechanisms are embedded in the hybrid encoder to extract the integrated rotation-invariant features fromRITPs.
• Two-branch design. The complete point cloud auxiliary branch is designed to acquire complete point cloud features during the training phase. The part-whole correlation module is used to achieve the mapping from missing point cloud features to complete point cloud features. The two-branch structure can improve the single-view point cloud classification accuracy.

We performed experiments on the ModelNet40, MVP, PCN and ScanObjectNN datasets. Compared to state-of-theart methods, our method achieved competitive or even the best performance.

2 RelatedWork

2.1 Learning-based Point Cloud Analysis

Recently, deep learning methods have become popular for processing point cloud data [Albert and Tri, 2023; Wang et al., 2024; Shen et al., 2024]. Initially, researchers used 3D voxel grids to process point clouds, which are easy to understand [Shi et al., 2023; Li et al., 2024; Singh and Yadav, 2024]. However, the voxelization process inevitably leads to information loss, which is closely related to the chosen resolution. PointNet [Qi et al., 2017a] is a pioneer in point cloud feature extraction using multi-layer perceptions and global pooling operations. PointNet++ [Qi et al., 2017b] was further proposed for multi-scale local feature encoding. Currently, the transformer framework has received widespread interest in point cloud processing. An efficient point selfattention layer was designed in Point Transformer [Zhao et al., 2021]. The dual-channel structure has been proposed in a dual-transformer network [Han et al., 2023] for efficient feature extraction. Recently, the Mamba framework based on state-space models has gained a lot of attention [Yang et al., 2024; Ju et al., 2024; Zhou et al., 2025]. Mamba is a novel architecture that provides an efficient solution through its innovative design of a selective state-space model and linear time complexity [Albert and Tri, 2023]. Point-Mamba [Liang et al., 2024] successfully applied Mamba to point cloud processing. However, the above methods are difficult to apply to the real world since the rotation of the object and the singleview features are not considered.

2.2 Single-view Point Cloud Analysis

Most of the inputs to current learning-based point cloud classifiers are complete point clouds. However, in the real world, the acquired 3D data are often partial point clouds from a single viewpoint. The predicted complete point cloud generated by point cloud completion methods can be used as input to classifiers [Cheng et al., 2022; Sarmad et al., 2019; Pan et al., 2023]. Studies directly addressing single-view point cloud classification are still relatively few. A singleview point cloud classifier called SVP-classifier [Mohammadi et al., 2022] was proposed for partial point cloud classification. SVP-classifier achieved classification based on global features, which were multi-view features mapped from single-view features. PAPNet [Xu et al., 2023] was a singleview classification network that learned object pose changes during its implementation. However, PAPNet still has its limitations, as noted in the literature [Xu et al., 2023], with inaccuracies in pose estimation and vulnerabilities to adversarial attacks, potentially leading to the failure of the perception system. In this paper, we aim to design an efficient point cloud classification network that directly processes singleview point clouds.

2.3 Rotation-invariant Analysis

Objects can be viewed from different perspectives and their poses are arbitrary in the real world. Therefore, rotation invariance is critical for practical applications in 3D object classification [Chen et al., 2019]. Several methods [Zhang et al., 2019; Zhang et al., 2022] based on rotation-invariant features have been proposed. RIConv [Zhang et al., 2019] constructed rotation-invariant features based on the center of mass from a local point set formed by k nearest neighbor points. RIConv++ [Zhang et al., 2022] designed rotationinvariant descriptors based on the theory of Local Reference Frames (LRFs). The LRF can preserve global rotation invariant features. Note that the LRF may be unstable due to changes in the centroid, especially if the shape contains disturbances. RISurConv [Zhang et al., 2025] constructed a locally triangulated surface to capture detailed surface structures for extracting highly expressive rotation-invariant surface properties. However, the classification accuracies of these methods decrease in single-view point cloud classification tasks. We aim to design a point cloud classification network that is robust to single views and arbitrary rotations.

3 Method

To improve the accuracy of classifiers for arbitrary pose and single-view features in real scenarios, we propose a rotationinvariant point cloud feature extraction module and a dualbranch network architecture.

3.1 Rotation-invariant Tetrahedron Properties

RITP is used to capture detailed spatial structure by constructing local tetrahedra around a reference point pi. The process of generating RITP is shown in Fig. 2 (a). For a reference point pi, k nearest neighbors could be obtained to form a local point set. Then a tetrahedron is formed based on the point pi and the nearest neighbors xi and other two adjacent points, xi-1 and xi+1. RITP can be expressed as:

where, Li represents the distance from the reference point pi to the other three vertices of the tetrahedron. Φi represents the relationship between the three edges in the tetrahedron with pi as vertex. Li and Φi are specified as:

θ0, θ1, θ2, α0, α1, β0, β1, γ0, γ1 represent the relationship between point normals and edges in a tetrahedron:

where -→ nj represents the normal of the corresponding vertex. ω0, ω1, ω2, ω3 represent the relationships between the four faces that make up the tetrahedron:

There is a significant difference in mathematical logic between RITP and other methods, which RITP exhibits high dimensional considerations. The specific difference is shown in Figs. 2 (b) and (c). The rotation-invariant property in RIConv++ is at the point level. RISurConv is at the surface level. Our RITP is at 3D structure level.

3.2 RISpaNet Block

RISpaNet is the core in our network (shown in Fig. 3), which is designed to extract the rotation-invariant features from point clouds. RISpaNet consists of three modules: the RIPGM, the pooling module and the hybrid encoder.

By constructing tetrahedra from nearest points, RIPGM generates RITP. A multi-scale pooling module (MSPM) is used to extract multi-scale features from RITP. As shown in Fig. 3, the pooling module consists of a convolution module and two pooling layers. RITP is processed by the pooling module to obtain multi-scale features FRITPmul. MSPM enables network to contain global, local and contextual information compared to single pooling.

A hybrid encoder is designed to further process FRITPmul to obtain rotation-invariant features. As shown in Fig. 3, hybrid encoder consists of three different structures of encoders: Mamba, self-attention mechanism (SA) and convolutional neural network (CNN). These three components are chosen because they each represent a different encoding way. Each of them is good at extracting different information. Mamba is a novel network architecture based on state-space models (SSMs), designed to efficiently capture long-range dependencies while preserving linear computational complexity. SA is a mechanism for capturing the contextual relationships of data by calculating the correlations between queries (Q) and keywords (K) and weighting the values (V ) with these correlations as weights. CNN could extract local features from data through convolutional operations. The sum of the outputs from these three encoders is processed with a convolutional layer to obtain rotation-invariant features. The specific frameworks of Mamba and SA are shown in Fig. 4.

3.3 Two-branch Network Structure

To improve classification accuracy of partial point clouds, we design a complete point cloud auxiliary branch (CPCAB) to obtain complete point cloud features from partial. The network structure is shown in Fig. 5. The input to the CPCAB is the complete point cloud from the dataset used. The CPCAB is used only in the training phase. In the inference phase, we only use the partial point cloud branch. The input to the partial channel is a partial, single-view point cloud captured by the depth device. Five RISpaNet blocks are used to extract rotation-invariant features.

The parameters of RISpaNet blocks in the auxiliary and partial branch are shared. The outputs of the two branches are the complete point cloud feature fcom and the partial point cloud feature fpart. To achieve the mapping of partial point cloud features to complete point cloud features, we design a part-whole correlation module (PWCM). The structure of PWCM is similar to the multi-scale pooling module, except that there is no pooling layer. Finally, the predicted complete point cloud feature fcompre provided by PWCM is processed by the decoder to obtain the predicted object label y. The structure of the decoder used is shown in Fig. 4.

3.4 Loss

The loss function of our network consists of two parts: the class loss Lcls and the feature loss Lfea. The loss of the overall network is Ltotal = Lcls +λLfea, where λ is a tradeoff parameter. The cross-entropy loss is used in Lcls, where Lcls = Fcross(ygt, y). ygt is the true label. The mean square error loss is used in the feature loss Lfea, where Lfea = Fmse(fcom, fcompre).

4 Experiment

4.1 Application Details

The network was tested on an NVIDIA RTX 3080 Ti GPU with four datasets: MoldeNet40 [Wu et al., 2015], MVP [Pan et al., 2023], PCN [Yuan et al., 2018] and ScanObjectNN [Uy et al., 2019]. The inputs to the network are 1024 3D coordinates and their corresponding point normals. It is worth noting that normals are optional for our method. In cases of no normals, we use the weighted eigenvector corresponding to the smallest eigenvalue as the normals. To evaluate the classification results, we followed the existing work on rotation invariance and performed experiments in three cases: (1) both training and test data were rotated around the z-axis (z/z), (2) training data were rotated around the z-axis and test data were arbitrarily rotated (z/SO3), and (3) both training and test data were arbitrarily rotated (SO3/SO3).

4.2 Complete Point Cloud Classification

The proposed method was initially validated on complete point clouds fromModelNet40. We trained and tested on all 40 classes in ModelNet40. We simply used the partial point cloud branch (PPCB) to connect directly to the decoder for training and testing. For simplicity, the single-branch way is called Ours-PPCB. The Adam optimizer was used. The learning rate started at 0.00033 and was reduced by 20% every 20 epochs. The batch for training was 56. We compared two types of methods: rotation-sensitive methods and rotationinvariant methods. The results are shown in Table 1. The results of PointNet [Qi et al., 2017a], PointCNN [Li et al., 2018], PointNet++ [Qi et al., 2017b], DGCNN [Wang et al., 2019], RIConv [Zhang et al., 2019] and RIConv++ [Zhang et al., 2022] are from RISurConv [Zhang et al., 2025]. The results of the SF-CNN [Rao et al., 2019], Li et al [Li et al., 2021], LGR-Net [Zhao et al., 2022] and Wang et al. [Wang et al., 2024] are fromWang et al. [Wang et al., 2024] . The results of RISurConv [Zhang et al., 2025] and PointMamba [Liang et al., 2024] were trained on the corresponding code provided on the official website.

For the rotation-sensitive method, Table 1 shows a significant drop in classification accuracy when the testing rotation differs from the training rotation. In contrast, the classification accuracy of the rotation-invariant methods decreases only slightly in this case. Our method achieves the highest classification accuracy as shown in Table 1, even without normal vectors.

4.3 Single-view Point Cloud Classification

We performed the single-view partial point cloud classification task on two datasets: the MVP and the PCN. MVP comprises 16 classes of point clouds captured by 26 fixedpose depth cameras, while PCN includes 8 classes of point clouds from 8 viewpoints. For training in the two-branch way, the Adam optimizer was used. The learning rate started at 0.00044 and was reduced by 15% every 10 epochs. The batch for training was 40. As for λ, it was set to 0.001 for the first 10 epochs, 0.0001 for epochs 11 to 20 and 0.00001 for subsequent epochs.

All comparison methods were trained using the official code provided in the corresponding literature. As shown in Table 2, our method performs excellently in all cases, even in the single-channel way. This demonstrates that RISpaNet block is effective in extracting rotation-invariant features. Compared to ours-PPCB, the classification accuracy of our two-channel method is significantly improved. This demonstrates the effectiveness of CPCAB and PWCM. Our method achieves 93.4% classification accuracy in MVP and 94.7% in PCN, achieving a significant improvement on the classification results of existing methods.

4.4 Real World Object Classification

For classification task in the real world, we performed experiments on the ScanObjectNN dataset, which is a 3D point cloud dataset with 15 classes captured by an RGB-D camera. In our experiments, we used the processed files and choose the hardest variant PB T50 RS to train our network with the partial point cloud branch. Due to the fact that normal were not provided in PB T50 RS, 3D coordinate form was used as input. The results are presented in Table 3. It can be seen that the classification accuracy of our method achieves as high as 94.8%, which is significantly superior to the performances of existing methods.

4.5 Ablation Experiments

RITP. To validate the effectiveness of the designed RITP, ablation experiments were performed on rotation-invariant properties, comparing with rotation-invariant surface properties (RISP) in RISurConv [Zhang et al., 2025] and informative rotation-invariant features (IRIF) in RIConv++ [Zhang et al., 2022]. All three rotationally invariant features mentioned above were subsequently extracted using the pooling module and the hybrid encoder in RISpaNet. Both training and test data are arbitrarily rotated (SO3/SO3) on MVP. It can be seen from Table 4 that the RITP-based method achieves the highest classification accuracy, demonstrating its highexpressive rotation-invariant property.

RISpaNet. To validate the effectiveness of the pooling module and hybrid encoder in the RISpaNet block, ablation experiments were performed on feature extraction. Both training and test data are arbitrarily rotated (SO3/SO3) on MVP. The results are shown in Table 5. The accuracy of Model A is higher than that of Model B, indicating that the multi-scale pooling module is beneficial for classification. Hybrid encoders can efficiently extract rotation-invariant features by comparing the accuracy of model B with model D and model A with model C.

Efficiency Analysis. We performed a comprehensive analysis for network efficiency during training and testing, as shown in Table 6. We compared our method with RIConv++ [Zhang et al., 2022], RISurConv [Zhang et al., 2025] and LGR-Net [Zhao et al., 2022]. The training and inference speeds of our network are generally comparable to existing methods, while the number of parameters is kept at a reasonable level.

Parametric Analysis. λ was obtained experimentally. We performed sensitivity tests for λ, as shown in Table 7. λ is appropriate between 10-3 and 10-5 . Based on the results of the sensitivity experiments, it is shown that the network is sensitive to λ in Loss.

5 Conclusion

In this paper, we propose a novel rotation-invariant network for point cloud classification. A highly expressive rotationinvariant property, called RITP, is designed to capture object spatial information by constructing tetrahedra. The RISpaNet block is used to extract rotation-invariant features of objects, including a rotation-invariant property generation module, a multi-scale pooling module and a hybrid encoder. Furthermore, for single-view point clouds, the joint use of a complete point cloud auxiliary branch and a part-whole correlation module is developed to obtain complete point cloud features from partial point clouds. Extensive experimental validation demonstrates the superior performance of our network for point cloud classification tasks, particularly achieving classification accuracies of 93.4% and 94.7% on the MVP and PCN benchmarks, respectively.

Our method is designed to accommodate the requirements of practical scenarios, with profound implications for future research and applications. In the future, we plan to apply this method to specific manipulation tasks to provide a cognitive foundation for robots.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (No. 62088101), in part by the Science and Technology Commission of Shanghai Municipality (No. 2021SHZDZX0100, 22ZR1467100), and the Fundamental Research Funds for the Central Universities (No. 22120240291).