JEMMA: An Extensible Java Dataset for ML4Code Applications

• Main Authors: Karmakar, Anjan, Allamanis, Miltiadis, Robbes, Romain
• Format: Journal Article
• Language: English
• Published: 18-12-2022
• Subjects: Computer Science - Learning; Computer Science - Software Engineering
• DOI: 10.48550/arxiv.2212.09132

Machine Learning for Source Code (ML4Code) is an active research field in
which extensive experimentation is needed to discover how to best use source
code's richly structured information. With this in mind, we introduce JEMMA, an
Extensible Java Dataset for ML4Code Applications, which is a large-scale,
diverse, and high-quality dataset targeted at ML4Code. Our goal with JEMMA is
to lower the barrier to entry in ML4Code by providing the building blocks to
experiment with source code models and tasks. JEMMA comes with a considerable
amount of pre-processed information such as metadata, representations (e.g.,
code tokens, ASTs, graphs), and several properties (e.g., metrics, static
analysis results) for 50,000 Java projects from the 50KC dataset, with over 1.2
million classes and over 8 million methods. JEMMA is also extensible allowing
users to add new properties and representations to the dataset, and evaluate
tasks on them. Thus, JEMMA becomes a workbench that researchers can use to
experiment with novel representations and tasks operating on source code. To
demonstrate the utility of the dataset, we also report results from two
empirical studies on our data, ultimately showing that significant work lies
ahead in the design of context-aware source code models that can reason over a
broader network of source code entities in a software project, the very task
that JEMMA is designed to help with.