R [Formula Omitted] LIVE: A Robust, Real-Time, LiDAR-Inertial-Visual Tightly-Coupled State Estimator and Mapping

• Published in: IEEE robotics and automation letters Vol. 6; no. 4; p. 7469
• Main Authors: Lin, Jiarong, Zheng, Chunran, Xu, Wei, Zhang, Fu
• Format: Journal Article
• Language: English
• Published: Piscataway The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 01-10-2021
• Subjects: Algorithms; Indoor environments; Inertial sensing devices; Kalman filters; Lidar; Multisensor fusion; Optimization; Real time; Robustness; State estimation
• ISSN: 2377-3766
• DOI: 10.1109/LRA.2021.3095515

In this letter, we propose a robust, real-time tightly-coupled multi-sensor fusion framework, which fuses measurements from LiDAR, inertial sensor, and visual camera to achieve robust and accurate state estimation. Our proposed framework is composed of two parts: the filter-based odometry and factor graph optimization. To guarantee real-time performance, we estimate the state within the framework of error-state iterated Kalman-filter, and further improve the overall precision with our factor graph optimization. Taking advantage of measurements from all individual sensors, our algorithm is robust enough to various visual failure, LiDAR-degenerated scenarios, and is able to run in real time on an on-board computation platform, as shown by extensive experiments conducted in indoor, outdoor, and mixed environments of different scale (see attached video). Moreover, the results show that our proposed framework can improve the accuracy of state-of-the-art LiDAR-inertial or visual-inertial odometry. To share our findings and to make contributions to the community, we open source our codes on our Github.