Decision Making for Autonomous Driving Stack: Shortening the Gap from Simulation to Real-World Implementations

• Published in: 2024 IEEE Intelligent Vehicles Symposium (IV) pp. 3107 - 3113
• Main Authors: Gutierrez-Moreno, Rodrigo, Barea, Rafael, Lopez-Guillen, Elena, Arango, Felipe, Revenga, Pedro, Bergasa, Luis M.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 02-06-2024
• Subjects: Decision making; Deep reinforcement learning; Digital twins; Focusing; Markov decision processes; Training; Urban areas
• ISSN: 2642-7214
• DOI: 10.1109/IV55156.2024.10588560

This paper introduces a novel methodology for implementing a practical Decision Making module within an Autonomous Driving Stack, focusing on merge scenarios in urban environments. Our approach leverages Deep Reinforcement Learning and Curriculum Learning, structured into three stages: initial training in a lightweight simulator (SUMO), refinement in a high-fidelity simulation (CARLA) through a Digital Twin, and final validation in real-world scenarios with Parallel Execution. We propose a Partially Observable Markov Decision Process framework and employ the Trust Region Policy Optimization algorithm to train our agent. Our method significantly narrows the gap between simulated training and real-world application, offering a cost-effective and flexible solution for Autonomous Driving development. The paper details the experimental setup and outcomes in each stage, demonstrating the effectiveness of the proposed methodology.