Unified gradient- and intensity-discriminator generative adversarial network for image fusion

• Published in: Information fusion Vol. 88; pp. 184 - 201
• Main Authors: Zhou, Huabing, Hou, Jilei, Zhang, Yanduo, Ma, Jiayi, Ling, Haibin
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01-12-2022
• Subjects: Generative adversarial network; Gradient; Image fusion; Intensity; Gradient; Intensity; Image fusion; Generative adversarial network
• ISSN: 1566-2535; 1872-6305
• DOI: 10.1016/j.inffus.2022.07.016

This study proposes a unified gradient- and intensity-discriminator generative adversarial network for various image fusion tasks, including infrared and visible image fusion, medical image fusion, multi-focus image fusion, and multi-exposure image fusion. On the one hand, we unify all fusion tasks into discriminating a fused image’s gradient and intensity distributions based on a generative adversarial network. The generator adopts a dual-encoder–single-decoder framework to extract source image features by using different encoder paths. A dual-discriminator is employed to distinguish the gradient and intensity, ensuring that the generated image contains the desired geometric structure and conspicuous information. The dual adversarial game can tackle the generative adversarial network’s mode collapse problem. On the other hand, we define a loss function based on the gradient and intensity that can be adapted to various fusion tasks by using varying relevant parameters with the source images. Qualitative and quantitative experiments on publicly available datasets demonstrate our method’s superiority over state-of-the-art methods.
•We propose a unified end-to-end fusion method for different fusion tasks.•It uses a generative adversarial network with gradient and intensity discriminators.•It uses the gradient and intensity of source images adequately with decision blocks.•It uses gradient and intensity to depict the essential structure and apparent info.•Our results on various fusion tasks contain the desired structure and apparent info.