Challenges in Using Neural Networks in Safety-Critical Applications

• Published in: 2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC) pp. 1 - 7
• Main Authors: Forsberg, Hakan, Linden, Joakim, Hjorth, Johan, Manefjord, Torbjorn, Daneshtalab, Masoud
• Format: Conference Proceeding
• Language: English
• Published: IEEE 11-10-2020
• Subjects: Artificial neural networks; avionics; Biological neural networks; deep neural networks; machine learning; Monitoring; Safety; safety-critical; Sensors; Software; Uncertainty
• DOI: 10.1109/DASC50938.2020.9256519
• EISSN: 2155-7209

In this paper, we discuss challenges when using neural networks (NNs) in safety-critical applications. We address the challenges one by one, with aviation safety in mind. We then introduce a possible implementation to overcome the challenges. Only a small portion of the solution has been implemented physically and much work is considered as future work. Our current understanding is that a real implementation in a safety-critical system would be extremely difficult. Firstly, to design the intended function of the NN, and secondly, designing monitors needed to achieve a deterministic and fail-safe behavior of the system. We conclude that only the most valuable implementations of NNs should be considered as meaningful to implement in safety-critical systems.