Debugging Static Analysis

• Published in: IEEE transactions on software engineering Vol. 46; no. 7; pp. 697 - 709
• Main Authors: Do, Lisa Nguyen Quang, Kruger, Stefan, Hill, Patrick, Ali, Karim, Bodden, Eric
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-07-2020; IEEE Computer Society
• Subjects: Computer bugs; Data flow analysis; Debugging; development tools; Encoding; graphical environments; integrated environments; program analysis; Static analysis; Static code analysis; Testing and debugging; usability testing; Writing
• ISSN: 0098-5589; 1939-3520
• DOI: 10.1109/TSE.2018.2868349

Static analysis is increasingly used by companies and individual code developers to detect and fix bugs and security vulnerabilities. As programs grow more complex, the analyses have to support new code concepts, frameworks and libraries. However, static-analysis code itself is also prone to bugs. While more complex analyses are written and used in production systems every day, the cost of debugging and fixing them also increases tremendously. To understand the difficulties of debugging static analysis, we surveyed 115 static-analysis writers. From their responses, we determined the core requirements to build a debugger for static analyses, which revolve around two main issues: abstracting from both the analysis code and the code it analyses at the same time, and tracking the analysis internal state throughout both code bases. Most tools used by our survey participants lack the capabilities to address both issues. Focusing on those requirements, we introduce Visuflow , a debugging environment for static data-flow analysis. Visuflow features graph visualizations and custom breakpoints that enable users to view the state of an analysis at any time. In a user study on 20 static-analysis writers, Visuflow helped identify 25 and fix 50 percent more errors in the analysis code compared to the standard Eclipse debugging environment.