Software Reliability Analysis of NASA Space Flight Software: A Practical Experience

• Published in: 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS) pp. 386 - 397
• Main Authors: Sukhwani, Harish, Alonso, Javier, Trivedi, Kishor S., Mcginnis, Issac
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-01-2016
• Subjects: Defect Reports; Flight Software; Hardware; Incremental Development; NASA; Software; Software reliability; Software Reliability Growth Models; Space missions; Testing
• DOI: 10.1109/QRS.2016.50

In this paper, we present the software reliability analysis of the flight software of a recently launched space mission. For our analysis, we use the defect reports collected during the flight software development. We find that this software was developed in multiple releases, each release spanning across all software life-cycle phases. We also find that the software releases were developed and tested for four different hardware platforms, spanning from off-the-shelf or emulation hardware to actual flight hardware. For releases that exhibit reliability growth or decay, we fit Software Reliability Growth Models (SRGM), otherwise we fit a distribution function. We find that most releases exhibit reliability growth, with Log-Logistic (NHPP) and S-Shaped (NHPP) as the best-fit SRGMs. For the releases that experience reliability decay, we investigate the causes for the same. We find that such releases were the first software releases to be tested on a new hardware platform, and hence they encountered major hardware integration issues. Also such releases seem to have been developed under time pressure in order to start testing on the new hardware platform sooner. Such releases exhibit poor reliability growth, and hence exhibit high predicted failure rate. Other problems include hardware specification changes and delivery delays from vendors. Thus, our analysis provides critical insights and inputs to the management to improve the software development process. As NASA has moved towards a product line engineering for its flight software development, software for future space missions will be developed in a similar manner and hence the analysis results for this mission can be considered as a baseline for future flight software missions.