Do Teachers Find Dashboards Trustworthy, Actionable and Useful? A Vignette Study Using a Logs and Audio Dashboard

• Published in: Technology, knowledge and learning Vol. 27; no. 3; pp. 971 - 989
• Main Authors: Kasepalu, Reet, Chejara, Pankaj, Prieto, Luis P., Ley, Tobias
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01-09-2022; Springer; Springer Nature B.V
• Subjects: Audio data; Collaboration; Collaborative work; Cooperative Learning; Creativity and Arts Education; Dashboards; Data Use; Education; Educational Technology; Information Management; Integrated Learning Systems; Learning; Learning Activities; Learning Analytics; Learning and Instruction; Mathematical analysis; Monitoring; Online Systems; Original Research; Science Education; Secondary School Teachers; Secondary Schools; Students; Teacher Attitudes; Teachers; Teaching Methods; Technology Uses in Education; Vignettes; Visualization; Teaching augmentation; Collaborative analytics; Learning analytics; Actionability; Teacher support; Design-based research; Monitoring students
• ISSN: 2211-1662; 2211-1670
• DOI: 10.1007/s10758-021-09522-5

Monitoring and guiding multiple groups of students in face-to-face collaborative work is a demanding task which could possibly be alleviated with the use of a technological assistant in the form of learning analytics. However, it is still unclear whether teachers would indeed trust, understand, and use such analytics in their classroom practice and how they would interact with such an assistant. The present research aimed to find out what the perception of in-service secondary school teachers is when provided with a dashboard based on audio and digital trace data when monitoring a collaborative learning activity. In a vignette study, we presented twenty-one in-service teachers with videos from an authentic collaborative activity, together with visualizations of simple collaboration analytics of those activities. The teachers perceived the dashboards as providers of useful information for their everyday work. In addition to assisting in monitoring collaboration, the involved teachers imagined using it for picking out students in need, getting information about the individual contribution of each collaborator, or even as a basis for assessment. Our results highlight the need for guiding dashboards as only providing new information to teachers did not compel them to intervene and additionally, a guiding dashboard could possibly help less experienced teachers with data-informed assessment.