Enhancing Color Images of Extremely Low Light Scenes Based on RGB/NIR Images Acquisition With Different Exposure Times

• Published in: IEEE transactions on image processing Vol. 24; no. 11; pp. 3586 - 3597
• Main Authors: Sugimura, Daisuke, Mikami, Takuya, Yamashita, Hiroki, Hamamoto, Takayuki
• Format: Journal Article
• Language: English
• Published: United States IEEE 01-11-2015; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Color; Colored noise; Correlation; Image color analysis; Image sequences; Imaging; Low light scene; near infrared image; Noise; Product introduction; Reconstruction; Restoration; Sharpening and deblurring; Smoothing; Video production; low light scene; reconstruction; sharpening and deblurring; smoothing; Restoration; near infrared image
• DOI: 10.1109/TIP.2015.2448356
• EISSN: 1941-0042
• ISSN: 1057-7149

We propose a novel method to synthesize a noise- and blur-free color image sequence using near-infrared (NIR) images captured in extremely low light conditions. In extremely low light scenes, heavy noise and motion blur are simultaneously produced in the captured images. Our goal is to enhance the color image sequence of an extremely low light scene. In this paper, we augment the imaging system as well as enhancing the image synthesis scheme. We propose a novel imaging system that can simultaneously capture the red, green, blue (RGB) and the NIR images with different exposure times. An RGB image is taken with a long exposure time to acquire sufficient color information and mitigates the effects of heavy noise. By contrast, the NIR images are captured with a short exposure time to measure the structure of the scenes. Our imaging system using different exposure times allows us to ensure sufficient information to reconstruct a clear color image sequence. Using the captured image pairs, we reconstruct a latent color image sequence using an adaptive smoothness condition based on gradient and color correlations. Our experiments using both synthetic images and real image sequences show that our method outperforms other state-of-the-art methods.