MSDPN: Monocular Depth Prediction with Partial Laser Observation using Multi-stage Neural Networks

• Published in: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) pp. 10750 - 10757
• Main Authors: Lim, Hyungtae, Gil, Hyeonjae, Myung, Hyun
• Format: Conference Proceeding
• Language: English
• Published: IEEE 24-10-2020
• Subjects: Laser radar; Mobile robots; Network architecture; Neural networks; Path planning; Robustness; Two dimensional displays
• ISSN: 2153-0866
• DOI: 10.1109/IROS45743.2020.9340767

In this study, a deep-learning-based multi-stage network architecture called Multi-Stage Depth Prediction Network (MSDPN) is proposed to predict a dense depth map using a 2D LiDAR and a monocular camera. Our proposed network consists of a multi-stage encoder-decoder architecture and Cross Stage Feature Aggregation (CSFA). The proposed multi-stage encoder-decoder architecture alleviates the partial observation problem caused by the characteristics of a 2D LiDAR, and CSFA prevents the multi-stage network from diluting the features and allows the network to learn the interspatial relationship between features better. Previous works use sub-sampled data from the ground truth as an input rather than actual 2D LiDAR data. In contrast, our approach trains the model and conducts experiments with a physically-collected 2D LiDAR dataset. To this end, we acquired our own dataset called KAIST RGBD-scan dataset and validated the effectiveness and the robustness of MSDPN under realistic conditions. As verified experimentally, our network yields promising performance against state-of-the-art methods. Additionally, we analyzed the performance of different input methods and confirmed that the reference depth map is robust in untrained scenarios.