Bilateral Attention Network for RGB-D Salient Object Detection

• Published in: IEEE transactions on image processing Vol. 30; pp. 1949 - 1961
• Main Authors: Zhang, Zhao, Lin, Zheng, Xu, Jun, Jin, Wen-Da, Lu, Shao-Ping, Fan, Deng-Ping
• Format: Journal Article
• Language: English
• Published: United States IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Ablation; Benchmark testing; Bilateral attention; Fans; Feature extraction; Image color analysis; Modules; Object detection; Object recognition; RGB-D image; Salience; salient object detection; Streaming media; Task analysis
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2021.3049959

RGB-D salient object detection (SOD) aims to segment the most attractive objects in a pair of cross-modal RGB and depth images. Currently, most existing RGB-D SOD methods focus on the foreground region when utilizing the depth images. However, the background also provides important information in traditional SOD methods for promising performance. To better explore salient information in both foreground and background regions, this paper proposes a Bilateral Attention Network (BiANet) for the RGB-D SOD task. Specifically, we introduce a Bilateral Attention Module (BAM) with a complementary attention mechanism: foreground-first (FF) attention and background-first (BF) attention. The FF attention focuses on the foreground region with a gradual refinement style, while the BF one recovers potentially useful salient information in the background region. Benefited from the proposed BAM module, our BiANet can capture more meaningful foreground and background cues, and shift more attention to refining the uncertain details between foreground and background regions. Additionally, we extend our BAM by leveraging the multi-scale techniques for better SOD performance. Extensive experiments on six benchmark datasets demonstrate that our BiANet outperforms other state-of-the-art RGB-D SOD methods in terms of objective metrics and subjective visual comparison. Our BiANet can run up to 80 fps on <inline-formula> <tex-math notation="LaTeX">224\times 224 </tex-math></inline-formula> RGB-D images, with an NVIDIA GeForce RTX 2080Ti GPU. Comprehensive ablation studies also validate our contributions.