Computational Cameras: Convergence of Optics and Processing

• Published in: IEEE transactions on image processing Vol. 20; no. 12; pp. 3322 - 3340
• Main Authors: Changyin Zhou, Nayar, S. K.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01-12-2011; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Artificial intelligence; Cameras; Coding; Coding, codes; Computation; Computer science; control theory; systems; Computer vision; Design engineering; Detectors; Devices; Exact sciences and technology; Image coding; Image processing; Imaging; Information, signal and communications theory; Lenses; Optical devices; optics; Pattern recognition. Digital image processing. Computational geometry; Sensors; Signal and communications theory; Signal processing; Telecommunications and information theory; Computer vision; Useful information; Image processing; Coding; Imaging; Imager; Illumination; Visual information; Image sensor; optics
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2011.2171700

A computational camera uses a combination of optics and processing to produce images that cannot be captured with traditional cameras. In the last decade, computational imaging has emerged as a vibrant field of research. A wide variety of computational cameras has been demonstrated to encode more useful visual information in the captured images, as compared with conventional cameras. In this paper, we survey computational cameras from two perspectives. First, we present a taxonomy of computational camera designs according to the coding approaches, including object side coding, pupil plane coding, sensor side coding, illumination coding, camera arrays and clusters, and unconventional imaging systems. Second, we use the abstract notion of light field representation as a general tool to describe computational camera designs, where each camera can be formulated as a projection of a high-dimensional light field to a 2-D image sensor. We show how individual optical devices transform light fields and use these transforms to illustrate how different computational camera designs (collections of optical devices) capture and encode useful visual information.