A systematic literature review on the industrial use of software process simulation

• Published in: The Journal of systems and software Vol. 97; pp. 65 - 85
• Main Authors: Ali, Nauman Bin, Petersen, Kai, Wohlin, Claes
• Format: Journal Article
• Language: English
• Published: New York Elsevier Inc 01-11-2014; Elsevier Sequoia S.A
• Subjects: Evidence based software engineering; Literature reviews; Simulation; Software engineering; Software process simulation; Software quality; Studies; Systematic literature review; Systematic review; Evidence based software engineering; Software process simulation; Systematic literature review
• ISSN: 0164-1212; 1873-1228
• DOI: 10.1016/j.jss.2014.06.059

•A literature review of software process simulation modelling was performed.•None of the claimed benefits could be substantiated with evidence.•No evidence exists for the claimed impact and adoption in the industrial practice.•Cost of simulation-based studies suggests that it is an expensive undertaking.
Software process simulation modelling (SPSM) captures the dynamic behaviour and uncertainty in the software process. Existing literature has conflicting claims about its practical usefulness: SPSM is useful and has an industrial impact; SPSM is useful and has no industrial impact yet; SPSM is not useful and has little potential for industry.
To assess the conflicting standpoints on the usefulness of SPSM.
A systematic literature review was performed to identify, assess and aggregate empirical evidence on the usefulness of SPSM.
In the primary studies, to date, the persistent trend is that of proof-of-concept applications of software process simulation for various purposes (e.g. estimation, training, process improvement, etc.). They score poorly on the stated quality criteria. Also only a few studies report some initial evaluation of the simulation models for the intended purposes.
There is a lack of conclusive evidence to substantiate the claimed usefulness of SPSM for any of the intended purposes. A few studies that report the cost of applying simulation do not support the claim that it is an inexpensive method. Furthermore, there is a paramount need for improvement in conducting and reporting simulation studies with an emphasis on evaluation against the intended purpose.