Image augmentation to improve construction resource detection using generative adversarial networks, cut-and-paste, and image transformation techniques

• Published in: Automation in construction Vol. 115; p. 103198
• Main Authors: Bang, Seongdeok, Baek, Francis, Park, Somin, Kim, Wontae, Kim, Hyoungkwan
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01-07-2020; Elsevier BV
• Subjects: Artificial neural networks; Augmentation; Construction resource detection; Construction sites; Data augmentation; Datasets; GAN; Generative adversarial networks; Image acquisition; Image reconstruction; Object recognition; On-site management; UAV; Unmanned aerial vehicles; Data augmentation; UAV; GAN; Construction resource detection; On-site management
• DOI: 10.1016/j.autcon.2020.103198
• EISSN: 1872-7891
• ISSN: 0926-5805

The paper proposes an image augmentation method to construct a large-size dataset for improving construction resource detection. The method consists of three techniques: removing-and-inpainting, cut-and-paste, and image-variation. The removing-and-inpainting technique arbitrarily removes objects from images and reconstructs the removed regions via generative adversarial networks (GAN). The cut-and-paste technique extracts objects from the original dataset and places them into the reconstructed images via the previous technique. The image-variation technique applies three image transformation techniques, intensity-, blur- and scale-variation, to the images. To evaluate the method, 656 unmanned aerial vehicle (UAV)-acquired construction site images were used as the original dataset. A faster region-based convolutional neural network (Faster R-CNN) trained with the augmented training dataset achieves better performance, which is higher than that of a network trained with the original dataset. These results prove that the method is optimal for improving construction resource detection in UAV-acquired images.
•The method constructs a dataset for improving construction resource detection.•The removing-and-inpainting removes objects and reconstructs regions via GAN.•The cut-and-paste extracts objects and places them into the images.•The image-variation applies intensity-, blur- and scale-variation to the images.•A faster R-CNN trained with the augmented dataset achieves the best performance.