Point Cloud Data
In the context of automotive perception, this unstructured data mostly stems from LiDAR sensors, which measure the distance between the vehicle itself and the surrounding space either through the mainstream method of time-of-flight measurement or through a frequency-modulated laser signal. Autonomous LiDAR systems serve as a sensory backbone of autonomous driving, delivering a continuous stream of complex geometrical structures, which are susceptible to range-dependent sparsity and jitter through atmospheric processes.
Perceptual Neural Network Paradigms for 3D Understanding
The landscape of 3D deep learning contains three primary categories, each operating at different levels of spatial and semantic granularity.
3D Point Cloud Classification
This global task focuses on predicting a single semantic label for an entire point cloud. It serves as the foundational architectural baseline for evaluating feature extraction capabilities. A more detailed explanation is provided in Section 2.3.1.
3D Object Detection
Moving beyond global shape recognition, object detection aims to find and categorize specific objects within a point cloud or point cloud sequence. This is typically achieved by drawing 3D bounding boxes and thus predicting an object’s spatial measurements, like translation and volumetric dimensions. Modern approaches like VirConv leverage multimodal fusion of LiDAR sensor inputs and camera data.
To process the unordered and unstructured nature of point sets, classification networks have brought forth several core methodologies, each introducing several trade-offs in their computational efficiency and geometric fidelity. The next section provides a brief overview of current methods. For further study, the paper by Zhang et al. is recommended.
Multi-View Architectures
Multi-view architectures circumvent the unstructured nature of 3D point cloud data by projecting the raw coordinates onto one or multiple predefined 2D planes, and therefore transforming the problem into the image-based domain. Once projected, the image can be categorized by highly efficient 2D CNNs. This strategy reduces the processing anomalies caused by varying point cloud densities.
However, multi-view methods require heavy geometric preprocessing and introduce irreversible depth information loss during projection. To reduce the risk of perspective occlusion, the network needs to rely on an array of camera viewpoints, elevating the computational cost even higher.
Explainability Methods in 3D Deep Learning
As neural network architectures grow more and more complex, they fundamentally function as “black boxes” where it is not clear why a certain instance was classified exactly in this way and not in another. In safety-critical domains such as autonomous driving and medical diagnosis, it is key to establish trust in the models. One cannot blindly rely on the model’s high validation accuracy but must also understand how the underlying architecture arrives at this geometric interpretation, such as identifying trains based on nearby tracks or on the vehicle itself.
To open up these black boxes, two primary classes of local explainability methods have emerged.
Gradient-Based Methods
Frameworks try to explain the underlying PointNet model by calculating a feature gradient during the backward pass, through systematically shifting points towards the centroid of the model instance to isolate their structural importance. These methods are computationally very efficient as they only require a single backward pass. However, they are strictly model-dependent, meaning they can only be applied to fully differentiable architectures.
Local Surrogate-Based Methods
Approaches such as LIME-3D are model-agnostic, meaning they treat the underlying network as a “black box.” They group the model into local clusters and generate perturbed variants by removing clusters and thus observing the changes in the network’s output classification. Therefore, these methods are highly interpretable for humans but suffer from immense computational cost and “contribution neutralization,” where positive and negative point activation in one cluster cancel each other out.
Ultimately, these local explainability methods excel at the point-wise individual level, but they are fundamentally limited as they cannot globally explain whole semantic classes. For explaining classes globally, other methods already exist.
Methods in Point Cloud Downsampling
In real-world applications, 3D point clouds can be comprised of millions of points, especially when they are acquired from LiDAR scanners. This introduces an immense data pipeline, which often exceeds edge hardware capabilities. Downsampling serves as a vital geometric preprocessing phase designed to systematically drop points from the initial point set in order to reduce memory consumption and overall execution time while maintaining high accuracy. Literature primarily divides downsampling methodologies into three distinct algorithmic approaches.
Bridging the Sensor Reality Gap
While the aforementioned existing implementations demonstrated the viability of deploying PointNet on edge hardware, their evaluation relied exclusively on uniformly sampled instances from the ModelNet dataset. These instances are completely free of environmental noise. To close this existing gap, this workflow leverages BLAINDER, an open-source add-on for Blender designed for depth-sensing simulation, such as Sonar and LiDAR. By using physically based raycasting methods, instead of simple primitive coordinates, we construct a robust dataset for network training and validation. It inherits geometrical noise and imperfections.
To thoroughly evaluate the classification boundaries of PointNet, we derived two distinct datasets that are publicly available to reproduce and hopefully improve our results.
Generic LiDAR Dataset
Serving as a control group, this dataset uses the generic preset presented by BLAINDER, fixed on a single perspective. Here, a “clean” and a “noisy” subset exist as well, as in the rotational dataset.
As the data-generation logic is built upon an adaptive script, it is totally possible to construct tailored solutions. Adaptations in the pipeline can be made to incorporate weather-dependent noise, such as rain, or to drive up mean and standard deviation, or swap sensor profiles completely—depending on the target application.
Feature-Driven Data Thinning: Critical Point Layer
To extract features in the PointNet architecture, the points are first processed independently in a shared MLP to extract further features. These features are then processed through a global max-pooling operation. A key insight of the authors was that a model’s final classification capability solely depends on a small subset of the input data—the Critical Points. These critical points have the highest activation in the max-pooling operation. Furthermore, they represent global geometric boundaries and landmarks of the shape.
While dropping up to the unspecified proportion of input points, the authors noted that the classification performance only dropped by the unspecified amount when using random sampling and thus still achieved an adequate classification performance. This proved that PointNet is highly robust to missing data, as long as critical points are not discarded. Because one point can dominate many columns in the max-pooling operation, the actual number of unique critical points that are driving the classification is quite small.
CPL Training
The CPL frontend was built up by a pointwise shared MLP, expanding from 3 to 64 over 128 to 1024 channels. Afterwards, a column-wise max-pooling block was used to extract indices with the highest scores. The entire network chain was trained with the Adam optimizer and a learning rate of 0.001 over 200 epochs. As the gradient flows strictly through the selected indices during backpropagation, the frontend operates as a discrete bottleneck rather than a continuous function.
It has to be noted that a single point of a point cloud can inhabit many distinct features if it is a prominent landmark. For example, a wingtip edge or nose tip can achieve peak activations across multiple feature channels at the same time. Therefore, the unique rows selected by the argmax operator are significantly lower than the channel depth. It is possible that a point cloud with 1024 points can be compressed down to as few as 40 points.
To maintain desired output point sizes, our proof-of-concept implementation duplicates the resulting points. If, on the other hand, fewer points are desired, the number of points is truncated. In later implementations, weighted sampling could be used instead of naive row repetition—effectively weighting how “valuable” a point is in the network—and repeat coordinates proportionally. This idea was already proposed by the authors of the publication our method is based on. It would also be possible to execute secondary pooling passes until the desired target density is reached.
Profiling the isolated CPL frontend over 5,000 test iterations on the same instance reveals an average execution latency of the unspecified latency in milliseconds for a 128-point target. Compared to the unspecified latency in milliseconds runtime required by FPS for the identical subset size, our implementation is nearly three times faster.
Geometric and Perceptual Comparison
To analyze whether the feature-driven downsampler aligns with human perception, we conducted a comparison between several instances from the airplane and person classes of the ModelNet40 dataset. The instances were first randomly sampled to have a point count of 1024, then LIME-3D was applied to see which points positively contributed to the classification. At last, CPL was applied to the downsampled point set. It can be noted that CPL learned to isolate extrema and sharp boundaries. For airplane instances, the layer consistently selects points distributed across both wing edges.
To also quantitatively evaluate the point clouds, a similarity measurement on the basis of the Chamfer Distances (CD) was conducted. Here, the CPL points were compared against the results of LIME-3D as well as a random point cloud with the same number of points as the CPL ones. As the results of RS can vary, we computed the CD 1000 times for RS.
Across all evaluated classes, RS consistently achieved a lower CD to LIME-3D, maintaining metrics nearly twice as low as CPL. This indicates that CPL introduces a higher geometric dissimilarity to the local explainer. This highlights the trade-off between classification utility and human interpretability, while exposing boundaries of both frameworks.
Local surrogate models like LIME-3D cannot evaluate every single point; they rely on clustering, producing explanations that are optimized for human perception.
CPL acts as an information funnel—strictly optimized for classification. It isolates a minimum of points that are necessary to maximize classification confidence. What a neural network deems critical for classification must not align with human perception.
Despite the stark difference in CD, CPL maintains operational utility. When testing our fused PointNet and CPL pipeline, where CPL was downsampled to 64 points, the network achieved an instance accuracy of the unspecified accuracy across 5,000 test instances. This performance is on par with our baseline metrics, where the network was trained on 64 random points (the unspecified baseline accuracy). Thus, it introduces minimal or non-negative degradation to PointNet’s classification performance, functioning as a capable edge accelerator.
Conclusion
We therefore showed the viability of deploying spatial deep learning architectures to edge processors. The contribution of our research is twofold, addressing the reality gap and algorithmic optimization.
First, we established a robust data-generation pipeline utilizing BLAINDER to generate realistic synthetic LiDAR data based on the ModelNet40 dataset. By emulating existing LiDAR sensors and introducing point-wise Gaussian noise, this pipeline simulates real-world sensor traits, such as view-dependent density. Cross-evaluation demonstrates that networks that are trained on pristine CAD models fail when subjected to sensor anomalies, but models that are trained on noisy data can abstract and achieve accurate results even on clean data. Furthermore, evaluations on variable densities reveal that PointNet is highly robust to data sparsity and maintains stable classification results even when aggressive data thinning is incorporated.
Second, we implemented a pre-trained Critical Point Layer as a preprocessing filter. Our results demonstrate that this frontend can deterministically compress a raw 1024-point cloud down to a sparse subset of 40–60 unique coordinates. When tested on the Raspberry Pi’s CPU, this method bypasses the timings set by the FPS algorithm, proving that it is possible to run in real time on low-powered edge devices.
Outlook
Several compelling avenues remain for optimization and architectural improvements.
To scale this pipeline to handle millions of points simultaneously that are generated by an outdoor LiDAR scanner, future work can explore offloading the CPL shared-MLP onto a Field-Programmable Gate Array (FPGA) or specialized low-powered NPUs. FPGAs are well-suited for massive parallel execution, which would free up further CPU resources.
To resolve the extreme feature collapse where only a handful of points exist after the compression, it would be desirable to test continuous and fully differentiable operators, such as the Gumbel-Top-k approximation. Instead of hard filtering and row duplication to fill the rows, this optimization would compute an importance value for every point and could be sampled based on this score.
Frequently Asked Questions
What problem does synthetic LiDAR data generation address? It addresses the gap between clean ModelNet data and noisy, imperfect sensor data acquired in real-world environments.
What is the Critical Point Layer? The Critical Point Layer is a feature-driven preprocessing filter that selects points with the highest activation in PointNet’s max-pooling operation.
How many points can the CPL retain from a 1024-point cloud? The CPL can compress a 1024-point cloud to approximately 40–60 unique coordinates.
Why is CPL useful for edge devices? It provides deterministic point cloud compression and bypasses the runtime of FPS on low-powered edge hardware.