Attentive Contexts for Object Detection

• Published in: IEEE transactions on multimedia Vol. 19; no. 5; pp. 944 - 954
• Main Authors: Li, Jianan, Wei, Yunchao, Liang, Xiaodan, Dong, Jian, Xu, Tingfa, Feng, Jiashi, Yan, Shuicheng
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01-05-2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Context; Context modeling; Feature extraction; Neural networks; Object detection; Pipelines; Proposals
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2016.2642789

Modern deep neural network-based object detection methods typically classify candidate proposals using their interior features. However, global and local surrounding contexts that are believed to be valuable for object detection are not fully exploited by existing methods yet. In this work, we take a step towards understanding what is a robust practice to extract and utilize contextual information to facilitate object detection in practice. Specifically, we consider the following two questions: "how to identify useful global contextual information for detecting a certain object?" and "how to exploit local context surrounding a proposal for better inferring its contents?" We provide preliminary answers to these questions through developing a novel attention to context convolution neural network (AC-CNN)-based object detection model. AC-CNN effectively incorporates global and local contextual information into the region-based CNN (e.g., fast R-CNN and faster R-CNN) detection framework and provides better object detection performance. It consists of one attention-based global contextualized (AGC) subnetwork and one multi-scale local contextualized (MLC) subnetwork. To capture global context, the AGC subnetwork recurrently generates an attention map for an input image to highlight useful global contextual locations, through multiple stacked long short-term memory layers. For capturing surrounding local context, the MLC subnetwork exploits both the inside and outside contextual information of each specific proposal at multiple scales. The global and local context are then fused together for making the final decision for detection. Extensive experiments on PASCAL VOC 2007 and VOC 2012 well demonstrate the superiority of the proposed AC-CNN over well-established baselines.