Localizing From Classification: Self-Directed Weakly Supervised Object Localization for Remote Sensing Images

• Published in: IEEE transaction on neural networks and learning systems Vol. PP; pp. 1 - 15
• Main Authors: Bai, Jing, Ren, Junjie, Xiao, Zhu, Chen, Zheng, Gao, Chengxi, Ali, Talal Ahmed Ali, Jiao, Licheng
• Format: Journal Article
• Language: English
• Published: United States IEEE 06-09-2023
• Subjects: Annotations; Class activation map (CAM); deep learning interpretability; Interference; Location awareness; Object detection; object localization; Remote sensing; remote sensing images (RSIs); Task analysis; Visualization; weakly supervised learning (WSL)
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2023.3309889

In recent years, object localization and detection methods in remote sensing images (RSIs) have received increasing attention due to their broad applications. However, most previous fully supervised methods require a large number of time-consuming and labor-intensive instance-level annotations. Compared with those fully supervised methods, weakly supervised object localization (WSOL) aims to recognize object instances using only image-level labels, which greatly saves the labeling costs of RSIs. In this article, we propose a self-directed weakly supervised strategy (SD-WSS) to perform WSOL in RSIs. To specify, we fully exploit and enhance the spatial feature extraction capability of the RSIs' classification model to accurately localize the objects of interest. To alleviate the serious discriminative region problem exhibited by previous WSOL methods, the spatial location information implicit in the classification model is carefully extracted by GradCAM<inline-formula> <tex-math notation="LaTeX">++</tex-math> </inline-formula> to guide the learning procedure. Furthermore, to eliminate the interference from complex backgrounds of RSIs, we design a novel self-directed loss to make the model optimize itself and explicitly tell it where to look. Finally, we review and annotate the existing remote sensing scene classification dataset and create two new WSOL benchmarks in RSIs, named C45V2 and PN2. We conduct extensive experiments to evaluate the proposed method and six mainstream WSOL methods with three backbones on C45V2 and PN2. The results demonstrate that our proposed method achieves better performance when compared with state-of-the-arts.