2D–3D synchronous/asynchronous camera fusion for visual odometry

• Published in: Autonomous robots Vol. 43; no. 1; pp. 21 - 35
• Main Authors: Paudel, Danda Pani, Demonceaux, Cédric, Habed, Adlane, Vasseur, Pascal
• Format: Journal Article
• Language: English
• Published: New York Springer US 01-01-2019; Springer Nature B.V; Springer Verlag
• Subjects: Artificial Intelligence; Computer Imaging; Control; Engineering; Engineering Sciences; Image reconstruction; Mechatronics; Motion simulation; Occlusion; Odometers; Pattern Recognition and Graphics; Robotics; Robotics and Automation; Signal and Image processing; Synchronism; Three dimensional motion; Vision; Visual Odometry; Asynchronous cameras; 2D–3D registration; Structure-from-Motion; 2D-3D registration; 2D; REGISTRATION
• ISSN: 0929-5593; 1573-7527
• DOI: 10.1007/s10514-018-9698-5

We propose a robust and direct 2D–3D registration method for camera synchronization. Once the cameras are synchronized—or for synchronous setups—we also propose a visual odometry framework that benefits from both 2D and 3D acquisitions. Our method does not require a precise set of 2D-to-3D correspondences, handles occlusions and works when the scene is only partially known. It is carried out through a 2D–3D based initial motion estimation followed by a constrained nonlinear optimization for motion refinement. The problems of occlusion and that of missing scene parts are handled by comparing the image-based reconstruction and 3D sensor measurements. The results of our experiments demonstrate that the proposed framework allows to obtain a good initial motion estimate and a significant improvement through refinement.