HW/SW co-design of a visual SLAM application

• Published in: Journal of real-time image processing Vol. 17; no. 3; pp. 667 - 689
• Main Authors: Piat, Jonathan, Fillatreau, Philippe, Tortei, Daniel, Brenot, Francois, Devy, Michel
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01-06-2020; Springer Nature B.V; Springer Verlag
• Subjects: Accelerators; Advanced driver assistance systems; Cameras; Co-design; Computer Graphics; Computer Science; Computer Vision and Pattern Recognition; Data processing; Design standards; Embedded systems; Hardware; Hardware Architecture; Image processing; Image Processing and Computer Vision; Lane changing; Localization; Multimedia Information Systems; Obstacle avoidance; Pattern Recognition; Perceptions; Robotics; Robots; Sensors; Signal,Image and Speech Processing; Simultaneous localization and mapping; Software; Special Issue Paper; System on chip; Vehicles; Machine-vision; Co-design; SLAM; ADAS; FPGA; FPGA architecture; SLAM Simultaneous Localisation and Mapping
• ISSN: 1861-8200; 1861-8219
• DOI: 10.1007/s11554-018-0836-2

Vision-based advanced driver assistance systems (ADAS), appeared in the 2000s, are increasingly integrated on-board mass-produced vehicles, as off-the-shelf low-cost cameras are now available. But ADAS implement most of the time-specific and basic functionalities such as lane departure or control of the distance to other vehicles. Integrating accurate localization and mapping functionalities meeting the constraints of ADAS (high-throughput, low-consumption, and small-design footprint) would pave the way towards obstacle detection, identification and tracking on-board vehicles at potential high speed. While the SLAM problem has been widely addressed by the robotics community, very few embedded operational implementations can be found, and they do not meet the ADAS-related constraints. In this paper, we implement the first 3D monocular EKF-SLAM chain on a heterogeneous architecture, on a single System on Chip (SoC), meeting these constraints. In order to do so, we picked up a standard co-design method (Shaout et al. Specification and modeling of hw/sw co-design for heterogeneous embedded systems,
2009
) and adapted it to the implementation of potentially any of such complex processing chains. The refined method encompasses a hardware-in-the-loop approach allowing to progressively integrate hardware accelerators on the basis of a systematic rule. We also have designed original hardware accelerators for all the image processing functions involved, and for some algebraic operations involved in the filtering process.