深度学习点云质量增强方法综述

With the development of 3D detection technologies，point clouds have gradually become one of the most common data representations of 3D objects or scenes that are widely used in many applications，such as autonomous driving，augmented reality（AR），and virtual reality（VR）. However，due to limitations in hardware，environment，and occlusion，the acquired point clouds are usually sparse，noisy，and uneven，hence imposing great challenges to the processing and analysis of point clouds. Therefore，point cloud quality enhancement techniques，which aim to process the original point cloud to obtain a dense，clean，and structurally complete point cloud，are of great significance. In recent years，with the development of hardware and machine learning technologies，deep-learning-based point cloud quality enhancement methods，which have great potential to extract the features of point clouds，have attracted the attention of scholars at home and abroad. Related works mainly focus on point cloud completion，point cloud upsampling（also known as super-resolution），and point cloud denoising. Point cloud completion fills the incomplete point clouds to restore the complete point cloud information，while point cloud upsampling increases the point number of the original point cloud to obtain a denser point cloud，and point cloud denoising removes the noisy points in the point cloud to obtain a cleaner point cloud. This paper systematically reviews the existing point cloud quality enhancement methods based on deep learning to offer a basis for subsequent research. First，this study briefly introduces the fundamentals and key technologies that are widely used in point cloud analysis. Second，three types of point cloud quality enhancement technologies，namely，upsampling，completion，and denoising，are introduced，classified，and summarized. According to the types of input data，point cloud completion methods can be divided into voxel- and point-based algorithms，with the latter being further sub-divided into two types depending on whether the encoder-decoder structure is exploited or not. The encoder-decoder-structure-based algorithms can be further divided according to whether the generative adversarial network（GAN）structure is used. Point cloud upsampling methods can be classified into convolutional neural network（CNN）-based algorithms，GAN-based algorithms，and graph convolutional network（GCN）-based algorithms. Point cloud denoising methods can also be divided into two types based on whether the encoder-decoder structure is exploited or not. Third，the commonly used datasets and evaluation metrics in point cloud quality enhancement tasks are summarized. The performance evaluation metrics for geometry reconstruction mainly include chamfer distance，earth mover's distance，Hausdorff distance，and point-to-surface distance. This paper then compares the state-of-the-art algorithms of point cloud completion and upsampling on common datasets and identifies the reasons for the differences in their performance. The recent progress and challenges in the field are then summarized，and future research trends are proposed. The findings are summarized as follows：1）the point cloud features extracted by existing deep learning-based algorithms are highly global，which means that the local features related to the detailed structure cannot be captured well，thus resulting in poor detail reconstruction. Traditional geometric algorithms are known to effectively represent data features based on geometric information. Therefore，how to combine geometric algorithms with deep learning for point cloud quality enhancement is worth exploring. 2）Most algorithms are for dense point clouds of single objects，and only a few studies have focused on sparse LiDAR point clouds containing large-scale outdoor scenes. 3）Most of the related studies only consider the point cloud processing of a single frame and ignore the temporal correlation of point cloud sequences. Therefore，how to utilize the spatial-temporal correlation to improve quality enhancement performance warrants further investigation. 4）In existing methods，the proposed network models are often complex and the inference speed is relatively slow，which fail to meet the real-time requirements of several applications. Therefore，how to further reduce the scale of the model parameters and improve the inference speed is a research direction worth exploring. 5）Most of the existing methods only process the geometric information（3D coordinates）of point clouds and ignore the attribute information（e. g. ，color and intensity）. Therefore，how to simultaneously enhance the quality of geometric and attribute information needs to be explored. Project page：https：//github.com/LilydotEE/Point_cloud_quality_enhancement.