A Static Approach to Prioritizing JUnit Test Cases

• Published in: IEEE transactions on software engineering Vol. 38; no. 6; pp. 1258 - 1275
• Main Authors: Hong Mei, Dan Hao, Lingming Zhang, Lu Zhang, Ji Zhou, Rothermel, G.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-11-2012; IEEE Computer Society
• Subjects: call graph; Computer programming; Effectiveness studies; Graphs; Java; JUnit; Priorities; Regression analysis; regression testing; Scheduling; Software testing; test case prioritization; Test systems
• ISSN: 0098-5589; 1939-3520
• DOI: 10.1109/TSE.2011.106

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most of these techniques require data on dynamic execution in the form of code coverage information for test cases. However, the collection of dynamic code coverage information on test cases has several associated drawbacks including cost increases and reduction in prioritization precision. In this paper, we propose an approach to prioritizing test cases in the absence of coverage information that operates on Java programs tested under the JUnit framework-an increasingly popular class of systems. Our approach, JUnit test case Prioritization Techniques operating in the Absence of coverage information (JUPTA), analyzes the static call graphs of JUnit test cases and the program under test to estimate the ability of each test case to achieve code coverage, and then schedules the order of these test cases based on those estimates. To evaluate the effectiveness of JUPTA, we conducted an empirical study on 19 versions of four Java programs ranging from 2K-80K lines of code, and compared several variants of JUPTA with three control techniques, and several other existing dynamic coverage-based test case prioritization techniques, assessing the abilities of the techniques to increase the rate of fault detection of test suites. Our results show that the test suites constructed by JUPTA are more effective than those in random and untreated test orders in terms of fault-detection effectiveness. Although the test suites constructed by dynamic coverage-based techniques retain fault-detection effectiveness advantages, the fault-detection effectiveness of the test suites constructed by JUPTA is close to that of the test suites constructed by those techniques, and the fault-detection effectiveness of the test suites constructed by some of JUPTA's variants is better than that of the test suites constructed by several of those techniques.