Deep Semantic Instance Segmentation of Tree-Like Structures Using Synthetic Data

• Published in: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) pp. 1713 - 1722
• Main Authors: Halupka, Kerry, Garnavi, Rahil, Moore, Stephen
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-01-2019
• Subjects: Neural networks; Periodic structures; Semantics; Task analysis; Three-dimensional displays; Training; Two dimensional displays
• DOI: 10.1109/WACV.2019.00187

Tree-like structures, such as blood vessels, often express complexity at very fine scales, requiring high-resolution grids to adequately describe their shape. Such sparse morphology can alternately be represented by locations of centreline points, but learning from this type of data with deep learning is challenging due to it being unordered, and permutation invariant. In this work, we propose a deep neural network that directly consumes unordered points along the centreline of a branching structure, to identify the topology of the represented structure in a single-shot. Key to our approach is the use of a novel multi-task loss function, enabling instance segmentation of arbitrarily complex branching structures. We train the network solely using synthetically generated data, utilizing domain randomization to facilitate the transfer to real 2D and 3D data. Results show that our network can reliably extract meaningful information about branch locations, bifurcations and endpoints, and sets a new benchmark for semantic instance segmentation in branching structures.