End-to-end approach for autonomous driving: a supervised learning method using computer vision algorithms for dataset creation

• Published in: Algorithms Vol. 16; no. 9; pp. 1 - 18
• Main Authors: Ribeiro, Inês A., Ribeiro, Tiago, Lopes, Gil, Ribeiro, A. Fernando
• Format: Journal Article
• Language: English
• Published: Basel Multidisciplinary Digital Publishing Institute (MDPI) 01-09-2023; MDPI AG
• Subjects: AI in engineering; Algorithms; Artificial intelligence; Automation; Autonomous driving; Autonomous vehicles; Cameras; Competition; Computer simulation; Computer vision; Control algorithms; Datasets; Deep learning; Driverless cars; Driving; Machine learning; Machine vision; Methods; Middleware; Neural network; Neural networks; Roads & highways; Robotics; Sensors; Simulation; Steering; Supervised learning
• ISSN: 1999-4893; 1999-4893
• DOI: 10.3390/a16090411

This paper presents a solution for an autonomously driven vehicle (a robotic car) based on artificial intelligence using a supervised learning method. A scaled-down robotic car containing only one camera as a sensor was developed to participate in the RoboCup Portuguese Open Autonomous Driving League competition. This study is based solely on the development of this robotic car, and the results presented are only from this competition. Teams usually solve the competition problem by relying on computer vision algorithms, and no research could be found on neural network model-based assistance for vehicle control. This technique is commonly used in general autonomous driving, and the amount of research is increasing. To train a neural network, a large number of labelled images is necessary; however, these are difficult to obtain. In order to address this problem, a graphical simulator was used with an environment containing the track and the robot/car to extract images for the dataset. A classical computer vision algorithm developed by the authors processes the image data to extract relevant information about the environment and uses it to determine the optimal direction for the vehicle to follow on the track, which is then associated with the respective image-grab. Several trainings were carried out with the created dataset to reach the final neural network model; tests were performed within a simulator, and the effectiveness of the proposed approach was additionally demonstrated through experimental results in two real robotics cars, which performed better than expected. This system proved to be very successful in steering the robotic car on a road-like track, and the agent’s performance increased with the use of supervised learning methods. With computer vision algorithms, the system performed an average of 23 complete laps around the track before going off-track, whereas with assistance from the neural network model the system never went off the track.
This work has been supported by COMPETE: POCI01-0145-FEDER-007043 and FCT—Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020. In addition, this work has been funded through a doctoral scholarship from the Portuguese Foundation for Science and Technology (Fundação para a Ciência e a Tecnologia), grant number SFRH/BD/06944/2020, with funds from the Portuguese Ministry of Science, Technology, and Higher Education and the European Social Fund through the Programa Operacional do Capital Humano (POCH).