Multi-scale Feature Fusion and Transformer Network for urban green space segmentation from high-resolution remote sensing images

• Published in: International journal of applied earth observation and geoinformation Vol. 124; p. 103514
• Main Authors: Cheng, Yong, Wang, Wei, Ren, Zhoupeng, Zhao, Yingfen, Liao, Yilan, Ge, Yong, Wang, Jun, He, Jiaxin, Gu, Yakang, Wang, Yixuan, Zhang, Wenjie, Zhang, Ce
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01-11-2023; Elsevier
• Subjects: Deep learning; High-resolution remote sensing images; Multi-scale feature fusion; Urban green space; Vegetation feature; Deep learning; Vegetation feature; Multi-scale feature fusion; High-resolution remote sensing images; Urban green space
• ISSN: 1569-8432; 1872-826X
• DOI: 10.1016/j.jag.2023.103514

•Multi-scale feature is fused into Transformer network to extract urban green space.•Spatial coordinate, context information and channel correlation are fully utilized.•Fusion module improves recognition of greenfield spatial and edge information.•Adding vegetation features improves the effect of green space segmentation.
Accurate extraction of urban green space is critical for preserving urban ecological balance and enhancing urban life quality. However, due to the complex urban green space morphology (e.g., different sizes and shapes), it is still challenging to extract green space effectively from high-resolution image. To address this issue, we proposed a novel hybrid method, Multi-scale Feature Fusion and Transformer Network (MFFTNet), as a new deep learning approach for extracting urban green space from high-resolution (GF-2) image. Our method was characterized by two aspects: (1) a multi-scale feature fusion module and transformer network that enhanced the recovery of green space edge information and (2) vegetation feature (NDVI) that highlighted vegetation information and enhanced vegetation boundaries identification. The GF-2 image was utilized to build two urban green space labeled datasets, namely Greenfield and Greenfield2. We compared the proposed MFFTNet with the existing popular deep learning models (like PSPNet, DensASPP, etc.) to evaluate the effectiveness of MFFTNet by the Mean Intersection Over Union (MIOU) benchmark on Greenfield, Greenfield2, and a public dataset (WHDLD). Experiments on Greenfield2 showed that MFFTNet can achieve a high MIOU (86.50%), which outperformed deep learning networks like PSPNet and DensASPP by 0.86% and 3.28%, respectively. Meanwhile, the MIOU of MFFTNet incorporating vegetation feature (NDVI) was further achieved to 86.76% on Greenfield2. Our experimental results demonstrate that the proposed MFFTNet with vegetation feature (NDVI) outperforms the state-of-the-art methods in urban green space segmentation.