From Lab to Production: Lessons Learnt and Real-Life Challenges of an Early Student-Dropout Prevention System

This paper presents the work done to support student dropout risk prevention in a real online e-learning environment: A Spanish distance university with thousands of undergraduate students. The main goal is to prevent students from abandoning the university by means of retention actions focused on t...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on learning technologies Vol. 12; no. 2; pp. 264 - 277
Main Authors:	Ortigosa, Alvaro, Carro, Rosa M., Bravo-Agapito, Javier, Lizcano, David, Alcolea, Juan Jesus, Blanco, Oscar
Format:	Journal Article
Language:	English
Published:	Piscataway IEEE 01-04-2019 Institute of Electrical and Electronics Engineers, Inc The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Academic Persistence Alarm systems Algorithms At Risk Students Colleges & universities Cost Effectiveness Data models Distance Education Distance learning Dropout Prevention e-learning Early warning systems Educational data mining Educational Environment Electronic learning Foreign Countries Prediction Prediction algorithms prediction methods Prediction models Predictive models Production Program Effectiveness Recording Retention School Holding Power student dropout Students Undergraduate Students warning systems Spain
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	This paper presents the work done to support student dropout risk prevention in a real online e-learning environment: A Spanish distance university with thousands of undergraduate students. The main goal is to prevent students from abandoning the university by means of retention actions focused on the most at-risk students, trying to maximize the effectiveness of institutional efforts in this direction. With this purpose, we generated predictive models based on the C5.0 algorithm using data from more than 11,000 students collected along five years. Then, we developed SPA (Sistema de Predicciæn de Abandono, dropout prediction system in Spanish), an early warning system that uses these models to generate static early dropout-risk predictions and dynamic periodically updated ones. It also supports the recording of the resulting retention-oriented interventions for further analysis. SPA is in production since 2017 and is currently in its fourth semester of continuous use. It has calculated more than 117,000 risk scores to predict the dropout risk of more than 5700 students. About 13,000 retention actions have been recorded. The white-box predictive models used in production provided reasonably good results, very close to those obtained in the laboratory. On the way from research to production, we faced several challenges that needed to be effectively addressed in order to be successful. In this paper, we share the challenges faced and the lessons learnt during this process. We hope this helps those who wish to cross the road from predictive modeling with potential value to the exploitation of complete dropout prevention systems that provide sustained value in real production scenarios.
ISSN:	1939-1382 1939-1382 2372-0050
DOI:	10.1109/TLT.2019.2911608