Testing DNN-Based Path Planning Algorithms by Metamorphic Testing

Deep Neural Networks (DNNs) are increasingly applied to solve path planning problems in recent years. However, unexpected or incorrect behaviors of DNNs greatly threaten the reliability of DNN-based path planning algorithms. Therefore, the reliability should be evaluated through the software testing...

Full description

Saved in:

Bibliographic Details
Published in:	2020 7th International Conference on Dependable Systems and Their Applications (DSA) pp. 515 - 526
Main Authors:	Lv, Shuxiao, Yin, Beibei
Format:	Conference Proceeding
Language:	English
Published:	IEEE 01-11-2020
Subjects:	Business deep learning testing DNN-based path planning algorithm metamorphic testing neuron coverage Path planning Software algorithms Software reliability Software systems Testing Training
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	Deep Neural Networks (DNNs) are increasingly applied to solve path planning problems in recent years. However, unexpected or incorrect behaviors of DNNs greatly threaten the reliability of DNN-based path planning algorithms. Therefore, the reliability should be evaluated through the software testing process. The quality of the training dataset is of great importance to the pre-trained DNN models. The pretrained model may still lack generality by using a randomly generated and insufficient training dataset. And DNN-based system testing is faced with Oracle problems. Because Metamorphic Testing (MT) has been shown considerable effectiveness in alleviating the absence of oracle problems. To increase the reliability of DNN-based path planning algorithms, in this paper, we present a test technique specialized for DNN-based path planning algorithms based on metamorphic testing. We present a framework for systematically designing sixteen metamorphic relations (MRs) by combining input transformations and output relations. And experiments are carried out on an actually released business software system, which demonstrates that our method is effective. The results show that our approach can effectively improve the diversity of test data, the accuracy of the DNN model, and the reliability of the software.
DOI:	10.1109/DSA51864.2020.00088