Weakly Supervised Actor-Action Segmentation via Robust Multi-task Ranking

• Published in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1022 - 1031
• Main Authors: Yan Yan, Chenliang Xu, Dawen Cai, Corso, Jason J.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-07-2017
• Subjects: Optimization; Robustness; Semantics; Support vector machines; Training; Videos
• ISSN: 1063-6919
• DOI: 10.1109/CVPR.2017.115

Fine-grained activity understanding in videos has attracted considerable recent attention with a shift from action classification to detailed actor and action understanding that provides compelling results for perceptual needs of cutting-edge autonomous systems. However, current methods for detailed understanding of actor and action have significant limitations: they require large amounts of finely labeled data, and they fail to capture any internal relationship among actors and actions. To address these issues, in this paper, we propose a novel, robust multi-task ranking model for weakly supervised actor-action segmentation where only video-level tags are given for training samples. Our model is able to share useful information among different actors and actions while learning a ranking matrix to select representative supervoxels for actors and actions respectively. Final segmentation results are generated by a conditional random field that considers various ranking scores for different video parts. Extensive experimental results on the Actor-Action Dataset (A2D) demonstrate that the proposed approach outperforms the state-of-the-art weakly supervised methods and performs as well as the top-performing fully supervised method.