Rotated object detection with forward-looking sonar in underwater applications

• Published in: Expert systems with applications Vol. 140; p. 112870
• Main Authors: Neves, Gustavo, Ruiz, Marco, Fontinele, Jefferson, Oliveira, Luciano
• Format: Journal Article
• Language: English
• Published: New York Elsevier Ltd 01-02-2020; Elsevier BV
• Subjects: Artificial neural networks; Autonomous underwater vehicle; Autonomous underwater vehicles; Boxes; Deep learning; Forward-looking sonar; Frames per second; Image detection; Noise levels; Object recognition; Random noise; Regression analysis; Representations; Rotation-invariant recognition; Satellite navigation systems; Shipwreck detection; Sonar; Subsea structure detection; Trajectory planning; Underwater inspection; Deep learning; Subsea structure detection; Shipwreck detection; Forward-looking sonar; Rotation-invariant recognition; Autonomous underwater vehicle
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2019.112870

•A novel data set containing underwater sonar images.•Two novel deep learning architectures to detect underwater objects.•Three novel representations for bounding box regression.
Autonomous underwater vehicles (AUVs) are often used to inspect the condition of submerged structures in oil and gas fields. Because the use of global positioning systems to aid AUV navigation is not feasible, object detection is an alternative method of supporting underwater inspection missions by detecting landmarks. Objects are detected not only to plan the trajectory of the AUVs, but their inspection can be the ultimate goal of the mission. In both cases, detecting an object’s distance and orientation with respect to the AUV provides clues for the vehicle’s navigation. Accordingly, we introduce a novel multi-object detection system that outputs object position and rotation from sonar images to support AUV navigation. To achieve this aim, two novel convolutional neural network-based architectures are proposed to detect and estimate rotated bounding boxes: an end-to-end network (RBoxNet), and a pipeline comprised of two networks (YOLOv2+RBoxDNet). Both proposed networks are structured from one of three novel representations of rotated bounding boxes regressed deep inside. Experimental analyses were performed by comparing several configurations of our proposed methods (by varying the backbone, regression representation, and architecture) with state-of-the-art methods using real sonar images. Results showed that RBoxNet presents the optimum trade-off between accuracy and speed, reaching an averaged mAP@[.5,.95] of 90.3% at 8.58 frames per second (FPS), while YOLOv2+RBoxDNet is the fastest solution, running at 16.19 FPS but with a lower averaged mAP@[.5,.95] of 77.5%. Both proposed methods are robust to additive Gaussian noise variations, and can detect objects even when the noise level is up to 0.10.