Recommending Comprehensive Solutions for Programming Tasks by Mining Crowd Knowledge

• Published in: 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC) pp. 358 - 368
• Main Authors: Silva, Rodrigo F.G., Roy, Chanchal K., Rahman, Mohammad Masudur, Schneider, Kevin A., Paixao, Klerisson, de Almeida Maia, Marcelo
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-05-2019
• Subjects: Documentation; Indexes; Java; Mining Crowd Knowledge; Natural languages; Programming profession; Stack Overflow; Task analysis; Word Embedding
• ISSN: 2643-7171
• DOI: 10.1109/ICPC.2019.00054

Developers often search for relevant code examples on the web for their programming tasks. Unfortunately, they face two major problems. First, the search is impaired due to a lexical gap between their query (task description) and the information associated with the solution. Second, the retrieved solution may not be comprehensive, i.e., the code segment might miss a succinct explanation. These problems make the developers browse dozens of documents in order to synthesize an appropriate solution. To address these two problems, we propose CROKAGE (Crowd Knowledge Answer Generator), a tool that takes the description of a programming task (the query) and provides a comprehensive solution for the task. Our solutions contain not only relevant code examples but also their succinct explanations. Our proposed approach expands the task description with relevant API classes from Stack Overflow Q&A threads and then mitigates the lexical gap problems. Furthermore, we perform natural language processing on the top quality answers and then return such programming solutions containing code examples and code explanations unlike earlier studies. We evaluate our approach using 97 programming queries, of which 50% was used for training and 50% was used for testing, and show that it outperforms six baselines including the state-of-art by a statistically significant margin. Furthermore, our evaluation with 29 developers using 24 tasks (queries) confirms the superiority of CROKAGE over the state-of-art tool in terms of relevance of the suggested code examples, benefit of the code explanations and the overall solution quality (code + explanation).