Test case prioritization: a family of empirical studies

• Published in: IEEE transactions on software engineering Vol. 28; no. 2; pp. 159 - 182
• Main Authors: Elbaum, S., Malishevsky, A.G., Rothermel, G.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-02-2002; IEEE Computer Society
• Subjects: Alliances; Case studies; Computer aided software engineering; Computer programs; Computer science; Cost control; Cost engineering; Costs; Debugging; Empirical analysis; Experiments; Fault detection; Faults; Feedback; Instruments; Radio access networks; Regression; Regression analysis; Software; Software engineering; Software measurement; Software testing; Studies; System testing; Systems design; Testing; Tradeoffs
• ISSN: 0098-5589; 1939-3520
• DOI: 10.1109/32.988497

To reduce the cost of regression testing, software testers may prioritize their test cases so that those which are more important, by some measure, are run earlier in the regression testing process. One potential goal of such prioritization is to increase a test suite's rate of fault detection. Previous work reported results of studies that showed that prioritization techniques can significantly improve rate of fault detection. Those studies, however, raised several additional questions: 1) Can prioritization techniques be effective when targeted at specific modified versions; 2) what trade-offs exist between fine granularity and coarse granularity prioritization techniques; 3) can the incorporation of measures of fault proneness into prioritization techniques improve their effectiveness? To address these questions, we have performed several new studies in which we empirically compared prioritization techniques using both controlled experiments and case studies.