ObjectCarver: Semi-automatic segmentation, reconstruction and separation of 3D objects
Implicit neural fields have made remarkable progress in reconstructing 3D surfaces from multiple images; however, they encounter challenges when it comes to separating individual objects within a scene. Previous work has attempted to tackle this problem by introducing a framework to train separate s...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Journal Article |
| Language: | English |
| Published: |
26-07-2024
|
| Subjects: | |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Implicit neural fields have made remarkable progress in reconstructing 3D
surfaces from multiple images; however, they encounter challenges when it comes
to separating individual objects within a scene. Previous work has attempted to
tackle this problem by introducing a framework to train separate signed
distance fields (SDFs) simultaneously for each of N objects and using a
regularization term to prevent objects from overlapping. However, all of these
methods require segmentation masks to be provided, which are not always readily
available. We introduce our method, ObjectCarver, to tackle the problem of
object separation from just click input in a single view. Given posed
multi-view images and a set of user-input clicks to prompt segmentation of the
individual objects, our method decomposes the scene into separate objects and
reconstructs a high-quality 3D surface for each one. We introduce a loss
function that prevents floaters and avoids inappropriate carving-out due to
occlusion. In addition, we introduce a novel scene initialization method that
significantly speeds up the process while preserving geometric details compared
to previous approaches. Despite requiring neither ground truth masks nor
monocular cues, our method outperforms baselines both qualitatively and
quantitatively. In addition, we introduce a new benchmark dataset for
evaluation. |
|---|---|
| DOI: | 10.48550/arxiv.2407.19108 |