Perceptual evaluation of different image fusion schemes

• Published in: Displays Vol. 24; no. 1; pp. 25 - 37
• Main Authors: Toet, Alexander, Franken, Eric M
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier B.V 01-02-2003; Elsevier
• Subjects: Applied sciences; Display; Electronics; Exact sciences and technology; Image fusion; Image processing; Imaging devices; Information, signal and communications theory; Infrared; Intensified imagery; Scene recognition; Semiconductor electronics. Microelectronics. Optoelectronics. Solid state devices; Signal processing; Situational awareness; Telecommunications and information theory; Situational awareness; Infrared; Intensified imagery; Scene recognition; Image fusion; Performance evaluation; Image recognition; Infrared camera; Night; Image processing; Merging; Awareness; Vision system; Visual perception; Image intensifier; CCD camera; Color vision; Infrared imaging; Target detection; Image registration; Object recognition; Experimental result; Gray scale; Scene analysis; Thermal image
• ISSN: 0141-9382; 1872-7387
• DOI: 10.1016/S0141-9382(02)00069-0

Human scene recognition performance was tested with images of night-time outdoor scenes. The scenes were registered both with a dual band (visual and near infrared) image intensified low-light CCD camera (DII) and with a thermal middle wavelength band (3–5
μm) infrared (IR) camera. Fused imagery was produced through a grayscale pyramid image merging scheme, in combination with two different colour mappings. Observer performance was tested for each of the (individual and fused) image modalities. The results show that DII imagery contributes most to global scene recognition (situational awareness), whereas IR imagery serves best for the detection and recognition of targets like humans and vehicles. Grayscale fused imagery yields appreciable performance levels in most conditions. With an appropriate colour mapping, colour fused imagery yields the best overall scene recognition performance. However, an inappropriate colour mapping significantly decreases observer performance compared to grayscale image fusion. The deployment of a DII system in addition to a 3–5
μm IR system through image fusion can increase the performance of human observers when the colour mapping relates to the nature of the visual task and the conditions (scene content) at hand.