Sufficient mutation operators for measuring test effectiveness

• Published in: 2008 ACM/IEEE 30th International Conference on Software Engineering Vol. 2008; no. 24; pp. 351 - 360
• Main Authors: Namin, A.S., Andrews, J., Murdoch, D.
• Format: Conference Proceeding Journal Article
• Language: English
• Published: IEEE 18-05-2008
• Subjects: Application software; Automatic testing; Computer science; Costs; Genetic mutations; mutation analysis; Predictive models; Software measurement; Software testing; Statistical analysis; testing effectiveness; Time measurement
• ISBN: 1424444861; 9781424444861; 1605580791; 9781605580791
• ISSN: 0270-5257; 1558-1225
• DOI: 10.1145/1368088.1368136

Mutants are automatically-generated, possibly faulty variants of programs. The mutation adequacy ratio of a test suite is the ratio of non-equivalent mutants it is able to identify to the total number of non-equivalent mutants. This ratio can be used as a measure of test effectiveness. However, it can be expensive to calculate, due to the large number of different mutation operators that have been proposed for generating the mutants. In this paper, we address the problem of finding a small set of mutation operators which is still sufficient for measuring test effectiveness. We do this by defining a statistical analysis procedure that allows us to identify such a set, together with an associated linear model that predicts mutation adequacy with high accuracy. We confirm the validity of our procedure through cross-validation and the application of other, alternative statistical analyses.