FGCN: Image-Fused Point Cloud Semantic Segmentation with Fusion Graph Convolutional Network

• Published in: Sensors (Basel, Switzerland) Vol. 23; no. 19; p. 8338
• Main Authors: Zhang, Kun, Chen, Rui, Peng, Zidong, Zhu, Yawei, Wang, Xiaohong
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01-10-2023; MDPI
• Subjects: Accuracy; Cameras; Datasets; FGCN; graph attention convolution; multi-modal data; multi-scale features; point clouds; Semantics; Teaching methods
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23198338

In interpreting a scene for numerous applications, including autonomous driving and robotic navigation, semantic segmentation is crucial. Compared to single-modal data, multi-modal data allow us to extract a richer set of features, which is the benefit of improving segmentation accuracy and effect. We propose a point cloud semantic segmentation method, and a fusion graph convolutional network (FGCN) which extracts the semantic information of each point involved in the two-modal data of images and point clouds. The two-channel k-nearest neighbors (KNN) module of the FGCN was created to address the issue of the feature extraction’s poor efficiency by utilizing picture data. Notably, the FGCN utilizes the spatial attention mechanism to better distinguish more important features and fuses multi-scale features to enhance the generalization capability of the network and increase the accuracy of the semantic segmentation. In the experiment, a self-made semantic segmentation KITTI (SSKIT) dataset was made for the fusion effect. The mean intersection over union (MIoU) of the SSKIT can reach 88.06%. As well as the public datasets, the S3DIS showed that our method can enhance data features and outperform other methods: the MIoU of the S3DIS can reach up to 78.55%. The segmentation accuracy is significantly improved compared with the existing methods, which verifies the effectiveness of the improved algorithms.