Use of Deep Learning Model with Attention Mechanism for Software Fault Prediction

• Published in: 2021 8th International Conference on Dependable Systems and Their Applications (DSA) pp. 161 - 171
• Main Authors: Yu, Ting-Yan, Huang, Chin-Yu, Fang, Neil C.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-08-2021
• Subjects: Attention mechanism; Convolutional Neural Network; Deep learning; Defect prediction; Feature extraction; Predictive models; Reliability engineering; Self-Attention mechanism; Semantics; Software; Software engineering; Training
• ISSN: 2767-6684
• DOI: 10.1109/DSA52907.2021.00025

Software defect prediction is a skill in software engineering that can increase program reliability. In the past, most defect prediction studies have been based on size and complexity metrics. In recent years, machine learning based predictive studies have been conducted. To build an accurate prediction model, choosing effective features remains critical. In this paper, we constructed a deep learning model called Defect Prediction via Self-Attention mechanism (DPSAM) to extract semantic features and predict defects automatically. We transferred programs into abstract syntax trees (ASTs) and encoded them into token vectors. With input features, we trained a self-attention mechanism to extract semantic features of programs and predict defects. We evaluated performance on 7 open source projects. In Within-Project Defect Prediction (WPDP), DPSAM achieved 16.8% and 14.4% performance improvement compared to state-of-the-art deep belief network (DBN)-based method and defect prediction via convolutional neural network (DP-CNN)-based method in F1 score, respectively. Besides, in Cross-Project Defect Prediction (CPDP), DPSAM achieve 23% and 60% performance improvement in F1 score compared to DBN-based method and DP-CNN-based method.