Detection of Stop Sign Violations From Dashcam Data

• Published in: IEEE transactions on intelligent transportation systems Vol. 23; no. 6; pp. 5411 - 5420
• Main Authors: Bravi, Luca, Kubin, Luca, Caprasecca, Stefano, de Andrade, Douglas Coimbra, Simoncini, Matteo, Taccari, Leonardo, Sambo, Francesco
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-06-2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Benchmark testing; Classifiers; Computing time; convolutional neural networks; dashcam; Detectors; Feature extraction; Feature maps; Global Positioning System; Global positioning systems; GPS; Image classification; Inertial platforms; Machine learning; Pipelines; Sensors; Spatial data; Stop sign violations; Vehicles; Video; Videos; Violations; Weather
• ISSN: 1524-9050; 1558-0016
• DOI: 10.1109/TITS.2021.3053648

In this article we present a novel machine learning pipeline for automatic detection of stop sign violations from dashcam videos, Inertial Measurement Units (IMU) and Global Positioning System (GPS) data. We developed a two-step approach, including a detector (Stop Sign Detector) capable of identifying stop signs presence, position, and size within video frames, followed by a classifier (Stop Violation Classifier) that assesses the presence of violations along with a severity score. The Stop Sign Detector is a deep convolutional neural network (CNN) for image classification, which leverages the information contained in its deeper layer feature maps in order to extract estimates of position and size of the detected stop signs. The Stop Violation Classifier fuses the information provided by the Stop Sign Detector with IMU/GPS data to assess the presence and severity of a stop sign violation. The proposed approach has been tested on several thousands of real-world videos, recorded from US vehicles, in all kinds of weather conditions, times of the day and environments. Our method achieves an area under the precision-recall curve of 94% with a required computational time of 2.4 seconds to process a 16-second video entirely on CPU.