Improving AFL++ CmpLog: Tackling the bottlenecks

• Main Authors: Wiebing, Sander, Rooijakkers, Thomas, Tesink, Sebastiaan
• Format: Journal Article
• Language: English
• Published: 15-11-2022
• Subjects: Computer Science - Cryptography and Security
• DOI: 10.48550/arxiv.2211.08357

The performance of the AFL++ CmpLog feature varies considerably for specific
programs under test (PUTs). In this paper it is demonstrated that the main
cause of the poor performance is low seed entropy, and a lack of deduplication
of magic bytes candidates. An improvement is proposed by mapping comparisons to
input bytes, in order to track which comparisons are controlled by what input
bytes. This mapping is then used to fuzz only the comparison values that are
magic byte candidates for that input part. Second, a caching mechanism is
introduced to reduce the number of redundant executions. The evaluation of the
improved versions shows a significant coverage gain compared to the original
AFL++ implementation of CmpLog for all PUTs, without breaking functionality.
The proposed solution in this paper provides a solid basis for a redesign of
CmpLog.