Improving urban tree species classification by deep-learning based fusion of digital aerial images and LiDAR

• Published in: Urban forestry & urban greening Vol. 94; p. 128240
• Main Authors: Ferreira, Matheus Pinheiro, dos Santos, Daniel Rodrigues, Ferrari, Felipe, Filho, Luiz Carlos Teixeira Coelho, Martins, Gabriela Barbosa, Feitosa, Raul Queiroz
• Format: Journal Article
• Language: English
• Published: Elsevier GmbH 01-04-2024
• Subjects: Convolutional neural networks; Deep learning; Optical-LiDAR fusion; Surface normals; Tropical urban forests; Deep learning; Optical-LiDAR fusion; Convolutional neural networks; Tropical urban forests; Surface normals
• ISSN: 1618-8667
• DOI: 10.1016/j.ufug.2024.128240

Accurate information on tree species distribution in urban areas can offer insights into how street trees provide ecosystem services, such as air pollution mitigation and surface cooling. This article presents a method to improve tree species classification in a tropical urban area using LiDAR-derived structural properties of individual tree crowns (ITCs) and digital aerial images. We extracted four LiDAR features, including surface normals of tree leaves, intensity, tree height, and leaf area index (LAI). We conducted two experiments: In the first, we trained encoder-decoder convolutional neural networks using a stack of optical bands and one LiDAR feature at a time. In the second, we developed an optical-LiDAR fusion strategy that combined feature maps from two encoder-decoder networks. One network was trained with optical bands only, while the other was trained with the LiDAR features that improved classification accuracy in the first experiment. Our experiment results demonstrated the usefulness of surface normals and intensity in discriminating among tree species. We found that the optical-LiDAR fusion strategy increased the average F1-score by 12.6 percentage points compared to only optical bands. We also employed the new segment anything (SAM) model to automatically delineate ITCs. SAM outlined ITCs with a boundary F1-score of 98%. The SAM-delineated ITCs were used to improve raw model predictions and produce reliable species maps. This study contributes to mapping and monitoring urban tree species in tropical areas.
•We show an approach to improve urban tree species classification.•Our approach is based on fusing aerial images and LiDAR with deep-learning.•Our fusion strategy improved the average F1-score by 12.6% points.•Surface normals from 3D LiDAR point clouds are useful to discriminate species.•Our study contributes to mapping and monitoring urban tree species in tropical areas.