A Survey of Deep Learning Applications to Autonomous Vehicle Control

• Published in: IEEE transactions on intelligent transportation systems Vol. 22; no. 2; pp. 712 - 733
• Main Authors: Kuutti, Sampo, Bowden, Richard, Jin, Yaochu, Barber, Phil, Fallah, Saber
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-02-2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: advanced driver assistance; Autonomous vehicles; computer vision; Control systems design; Deep learning; intelligent control; Intelligent transportation systems; Linear control; Machine learning; Neural networks; Nonlinear analysis; Nonlinear control; Object recognition; Perception; Reinforcement learning; Semantic segmentation; Sensors; Task analysis; Teaching methods; Training
• ISSN: 1524-9050; 1558-0016
• DOI: 10.1109/TITS.2019.2962338

Designing a controller for autonomous vehicles capable of providing adequate performance in all driving scenarios is challenging due to the highly complex environment and inability to test the system in the wide variety of scenarios which it may encounter after deployment. However, deep learning methods have shown great promise in not only providing excellent performance for complex and non-linear control problems, but also in generalising previously learned rules to new scenarios. For these reasons, the use of deep learning for vehicle control is becoming increasingly popular. Although important advancements have been achieved in this field, these works have not been fully summarised. This paper surveys a wide range of research works reported in the literature which aim to control a vehicle through deep learning methods. Although there exists overlap between control and perception, the focus of this paper is on vehicle control, rather than the wider perception problem which includes tasks such as semantic segmentation and object detection. The paper identifies the strengths and limitations of available deep learning methods through comparative analysis and discusses the research challenges in terms of computation, architecture selection, goal specification, generalisation, verification and validation, as well as safety. Overall, this survey brings timely and topical information to a rapidly evolving field relevant to intelligent transportation systems.