Towards Explainable Prediction Feedback Messages Using BERT

• Published in: International journal of artificial intelligence in education Vol. 34; no. 3; pp. 1046 - 1071
• Main Authors: Cavalcanti, Anderson Pinheiro, Mello, Rafael Ferreira, Gašević, Dragan, Freitas, Fred
• Format: Journal Article
• Language: English
• Published: New York Springer New York 01-09-2024; Springer Nature B.V
• Subjects: Academic Achievement; Artificial Intelligence; Best practice; Classification; Computer Science; Computers and Education; Content analysis; Distance learning; Educational Technology; Educational Theories; Electronic Learning; Essays; Explainable artificial intelligence; Feedback; Formative evaluation; Learner Engagement; Learning Processes; Learning Strategies; Messages; Neural networks; Portuguese; Student Improvement; Students; Teacher Role; Teachers; User Interfaces and Human Computer Interaction; Explainable artificial intelligence; BERT; Feedback; Online learning
• ISSN: 1560-4292; 1560-4306
• DOI: 10.1007/s40593-023-00375-w

Educational feedback is a crucial factor in the student’s learning journey, as through it, students are able to identify their areas of deficiencies and improve self-regulation. However, the literature shows that this is an area of great dissatisfaction, especially in higher education. Providing effective feedback becomes an increasingly challenging task as the number of students increases. Therefore, this article explores the use of automated content analysis to examine instructor feedback based on reputable models from the literature that provide best practices and classify feedback at different levels. For this, this article proposes using the transformer model BERT to classify feedback messages. The proposed method outperforms previous works by up to 35.71% in terms of Cohen’s kappa. Finally, this study adopted an explainable artificial intelligence to provide insights into the most predictive features for each classifier analyzed.