Multi-objective test report prioritization using image understanding

• Published in: 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE) pp. 202 - 213
• Main Authors: Yang Feng, Jones, James A., Zhenyu Chen, Chunrong Fang
• Format: Conference Proceeding
• Language: English
• Published: ACM 01-09-2016
• Subjects: Computer bugs; Crowdsourced Testing; Image color analysis; Image Understanding; Inspection; Mobile applications; Mobile communication; Multi-Objective Optimization; Software; Test Report Prioritization; Testing
• DOI: 10.1145/2970276.2970367

In crowdsourced software testing, inspecting the large number of test reports is an overwhelming but inevitable software maintenance task. In recent years, to alleviate this task, many text-based test-report classification and prioritization techniques have been proposed. However in the mobile testing domain, test reports often consist of more screenshots and shorter descriptive text, and thus text-based techniques may be ineffective or inapplicable. The shortage and ambiguity of natural-language text information and the well defined screenshots of activity views within mobile applications motivate our novel technique based on using image understanding for multi-objective test-report prioritization. In this paper, by taking the similarity of screenshots into consideration, we present a multi-objective optimization-based prioritization technique to assist inspections of crowdsourced test reports. In our technique, we employ the Spatial Pyramid Matching (SPM) technique to measure the similarity of the screenshots, and apply the natural-language processing technique to measure the distance between the text of test reports. Furthermore, to validate our technique, an experiment with more than 600 test reports and 2500 images is conducted. The experimental results show that image-understanding techniques can provide benefit to test-report prioritization for most applications.