SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation

• Published in: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 9811 - 9820
• Main Authors: Liu, Dongfang, Cui, Yiming, Tan, Wenbo, Chen, Yingjie
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-01-2021
• Subjects: Computer architecture; Computer vision; Feature extraction; Predictive models; Robustness; Runtime; Tracking
• ISSN: 2575-7075
• DOI: 10.1109/CVPR46437.2021.00969

Video instance segmentation (VIS) is a new and critical task in computer vision. To date, top-performing VIS methods extend the two-stage Mask R-CNN by adding a tracking branch, leaving plenty of room for improvement. In contrast, we approach the VIS task from a new perspective and propose a one-stage spatial granularity network (SG-Net). Compared to the conventional two-stage methods, SG-Net demonstrates four advantages: 1) Our method has a one-stage compact architecture and each task head (detection, segmentation, and tracking) is crafted interdependently so they can effectively share features and enjoy the joint optimization; 2) Our mask prediction is dynamically performed on the sub-regions of each detected instance, leading to high-quality masks of fine granularity; 3) Each of our task predictions avoids using expensive proposal-based RoI features, resulting in much reduced runtime complexity per instance; 4) Our tracking head models objects' centerness movements for tracking, which effectively enhances the tracking robustness to different object appearances. In evaluation, we present state-of-the-art comparisons on the YouTube-VIS dataset. Extensive experiments demonstrate that our compact one-stage method can achieve improved performance in both accuracy and inference speed. We hope our SG-Net could serve as a strong and flexible base-line for the VIS task. Our code will be available here 1 .