Approximate Differentiable Rendering with Algebraic Surfaces

• Main Authors: Keselman, Leonid, Hebert, Martial
• Format: Journal Article
• Language: English
• Published: 21-07-2022
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computer Vision and Pattern Recognition; Computer Science - Graphics
• DOI: 10.48550/arxiv.2207.10606

Differentiable renderers provide a direct mathematical link between an
object's 3D representation and images of that object. In this work, we develop
an approximate differentiable renderer for a compact, interpretable
representation, which we call Fuzzy Metaballs. Our approximate renderer focuses
on rendering shapes via depth maps and silhouettes. It sacrifices fidelity for
utility, producing fast runtimes and high-quality gradient information that can
be used to solve vision tasks. Compared to mesh-based differentiable renderers,
our method has forward passes that are 5x faster and backwards passes that are
30x faster. The depth maps and silhouette images generated by our method are
smooth and defined everywhere. In our evaluation of differentiable renderers
for pose estimation, we show that our method is the only one comparable to
classic techniques. In shape from silhouette, our method performs well using
only gradient descent and a per-pixel loss, without any surrogate losses or
regularization. These reconstructions work well even on natural video sequences
with segmentation artifacts. Project page:
https://leonidk.github.io/fuzzy-metaballs