A Pilot Study of Testing Infrastructure as Code for Cloud Systems

• Published in: 2023 30th Asia-Pacific Software Engineering Conference (APSEC) pp. 584 - 588
• Main Authors: Suwanachote, Nabhan, Pornmaneerattanatri, Soratouch, Kashiwa, Yutaro, Ichikawa, Kohei, Leelaprute, Pattara, Rungsawang, Arnon, Manaskasemsak, Bundit, Iida, Hajimu
• Format: Conference Proceeding
• Language: English
• Published: IEEE 04-12-2023
• Subjects: Cloud System; Codes; Computer bugs; Infrastructure as Code; Java; Servers; Software Testing; Source coding; Testing
• ISSN: 2640-0715
• DOI: 10.1109/APSEC60848.2023.00075

Infrastructure as Code (IaC) has become the de-facto standard method for managing cloud resources. Just like general source code (e.g., Java, etc.), infrastructure code also has numerous bugs so it needs to be tested. While several testing frameworks for IaC for cloud systems have been developed in practice, researchers have paid little attention to their testing. This study presents an empirical investigation of the use of tests for IaC for cloud systems. Our empirical results show that (i) 55.2% of the repositories using Terratest have at least one server infrastructure test; (ii) developers often maintain server infrastructure tests (1.7%-11.3% commits out of all the commits); (iii) many repositories have tests for system functionality (28%), deployment (20%), and configuration (17%).