PANDORA: Polarization-Aided Neural Decomposition Of Radiance

• Main Authors: Dave, Akshat, Zhao, Yongyi, Veeraraghavan, Ashok
• Format: Journal Article
• Language: English
• Published: 25-03-2022
• Subjects: Computer Science - Computer Vision and Pattern Recognition; Computer Science - Graphics
• DOI: 10.48550/arxiv.2203.13458

Reconstructing an object's geometry and appearance from multiple images, also
known as inverse rendering, is a fundamental problem in computer graphics and
vision. Inverse rendering is inherently ill-posed because the captured image is
an intricate function of unknown lighting conditions, material properties and
scene geometry. Recent progress in representing scene properties as
coordinate-based neural networks have facilitated neural inverse rendering
resulting in impressive geometry reconstruction and novel-view synthesis. Our
key insight is that polarization is a useful cue for neural inverse rendering
as polarization strongly depends on surface normals and is distinct for diffuse
and specular reflectance. With the advent of commodity, on-chip, polarization
sensors, capturing polarization has become practical. Thus, we propose PANDORA,
a polarimetric inverse rendering approach based on implicit neural
representations. From multi-view polarization images of an object, PANDORA
jointly extracts the object's 3D geometry, separates the outgoing radiance into
diffuse and specular and estimates the illumination incident on the object. We
show that PANDORA outperforms state-of-the-art radiance decomposition
techniques. PANDORA outputs clean surface reconstructions free from texture
artefacts, models strong specularities accurately and estimates illumination
under practical unstructured scenarios.