Practical, Automated Scenario-Based Mobile App Testing

• Published in: IEEE transactions on software engineering Vol. 50; no. 7; pp. 1949 - 1966
• Main Authors: Yu, Shengcheng, Fang, Chunrong, Du, Mingzhe, Ding, Zimin, Chen, Zhenyu, Su, Zhendong
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-07-2024; IEEE Computer Society
• Subjects: Applications programs; Automation; Business; Electrocardiography; event knowledge graph; Graphical user interfaces; image understanding; Knowledge graphs; Knowledge representation; Logic; Mobile app testing; Mobile applications; Mobile computing; Quality assurance; Quality control; scenario-based testing; Scripts; Testing
• ISSN: 0098-5589; 1939-3520
• DOI: 10.1109/TSE.2024.3414672

The importance of mobile application (app) quality assurance is increasing with the rapid development of the mobile Internet. Automated test generation approaches, as a dominant direction of app quality assurance, follow specific models or strategies, targeting at optimizing the code coverage. Such approaches lead to a huge gap between testing execution and app business logic. Test scripts developed by human testers consider business logic by focusing on testing scenarios. Due to the GUI-intensive feature of mobile apps, human testers always understand app GUI to organize test scripts for scenarios. This inspires us to utilize domain knowledge from app GUI understanding for scenario-based test generation. In this paper, we propose a novel approach, ScenTest , for scenario-based mobile app testing with event knowledge graph (EKG) via GUI image understanding. ScenTest tries to start automated testing by imitating human practices and integrating domain knowledge into scenario-based mobile app testing, realizing fully automated testing on target testing scenarios for the first time. ScenTest extracts four kinds of entities and five kinds of corresponding relationships from crowdsourced test reports, where the test events and app GUI information are presented, and constructs the EKGs for specific scenarios. Then, ScenTest conducts test generation for specific scenarios on different apps with the guidance of EKG with the combination consideration of app current state and testing context. We conduct an evaluation on ScenTest on different aspects. The results show that the test generation of ScenTest on the basis of EKG is effective, and ScenTest reveals 150+ distinct real-world bugs in specific scenarios compared with representative baselines.