DDGANSE: Dual-Discriminator GAN with a Squeeze-and-Excitation Module for Infrared and Visible Image Fusion

• Published in: Photonics Vol. 9; no. 3; p. 150
• Main Authors: Wang, Jingjing, Ren, Jinwen, Li, Hongzhen, Sun, Zengzhao, Luan, Zhenye, Yu, Zishu, Liang, Chunhao, Monfared, Yashar E., Xu, Huaqiang, Hua, Qing
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01-03-2022
• Subjects: Computer vision; Deep learning; Discriminators; Excitation; fusion image; Generative adversarial networks; Image contrast; Image processing; infrared image; Infrared imagery; Modules; Neural networks; Performance measurement; squeeze-and-excitation net-works; Teaching methods; Texture; Training; visible image; Visual system
• ISSN: 2304-6732; 2304-6732
• DOI: 10.3390/photonics9030150

Infrared images can provide clear contrast information to distinguish between the target and the background under any lighting conditions. In contrast, visible images can provide rich texture details and are compatible with the human visual system. The fusion of a visible image and infrared image will thus contain both comprehensive contrast information and texture details. In this study, a novel approach for the fusion of infrared and visible images is proposed based on a dual-discriminator generative adversarial network with a squeeze-and-excitation module (DDGANSE). Our approach establishes confrontation training between one generator and two discriminators. The goal of the generator is to generate images that are similar to the source images, and contain the information from both infrared and visible source images. The purpose of the two discriminators is to increase the similarity between the image generated by the generator and the infrared and visible images. We experimentally demonstrated that using continuous adversarial training, DDGANSE outputs images retain the advantages of both infrared and visible images with significant contrast information and rich texture details. Finally, we compared the performance of our proposed method with previously reported techniques for fusing infrared and visible images using both quantitative and qualitative assessments. Our experiments on the TNO dataset demonstrate that our proposed method shows superior performance compared to other similar reported methods in the literature using various performance metrics.