ARQGAN: An evaluation of generative adversarial network approaches for automatic virtual inpainting restoration of Greek temples

• Published in: Expert systems with applications Vol. 180; p. 115092
• Main Authors: Nogales, Alberto, Delgado-Martos, Emilio, Melchor, Ángel, García-Tejedor, Álvaro J.
• Format: Journal Article
• Language: English
• Published: New York Elsevier Ltd 15-10-2021; Elsevier BV
• Subjects: Architecture; Deep Learning; Generative Adversarial Networks; Greek temples; Image inpainting; Image reconstruction; Image restoration; Image segmentation; Segmented training; Temples; Virtual restoration; Greek temples; Generative Adversarial Networks; Segmented training; Image inpainting; Virtual restoration; Deep Learning
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2021.115092

•A dataset formed by images of 3D models of Greek temples has been created.•Two Deep Learning approaches for automatic image restoration of ruined temples.•Both approaches have been evaluated with mathematical metrics with good results.•Restored images have been evaluated by people in the field of architecture.
Image reconstruction has received much attention and has advanced in recent years with the rise of deep learning. Deep neural models have been able to perform image-to-image translation by transferring pictorial styles, coloring old photographs or filling in missing parts. This last technique is known as image inpainting and enables restoration of damaged or missing parts of an image or photograph to obtain the complete picture. However, it is not always possible to properly define which parts are missing or to identify where they are missing, as in the case of superimposing new information on an already complete image. In this paper, we propose the use of generative adversarial networks (GANs), a well-known deep learning model, for virtual inpainting restoration of artificial landscape images containing archaeological remains of Greek temples. The network identifies key features determined by the internal logic of the architectural style denoted by the ruins and adds the missing architectural elements to obtain an image of the restored building. Unlike other studies, it does not receive any information on which elements should be added or where. Virtual inpainting restoration is capable of representing a building’s envelope but also integrates particular aspects of the building related to the architectural language used for its design. The restoration of the fundamental parts of the classical Greek order was consistent, and the results were evaluated with objective metrics and through a subjective survey between academics and architects. They showed that adding segmented images to the training dataset gives better results.