The Limited Value of Machine Learning Approach to Improving Predictive Performance: The Ministry of Justice Case Assessment Tool

• Published in: Psychology, public policy, and law Vol. 30; no. 2; pp. 171 - 178
• Main Authors: Mori, Takemi, Harada, Kyoko, Yamaki, Mayuko, Yamaguchi, Masatoshi
• Format: Journal Article
• Language: English
• Published: American Psychological Association 01-05-2024
• Subjects: Decision Tree Algorithms; Female; Human; Justice; Machine Learning; Machine Learning Algorithms; Male; Predictability (Measurement); Predictive Validity; Recidivism; Risk Assessment; predictive validity; risk assessment; recidivism; machine learning; youth who offend
• ISSN: 1076-8971; 1939-1528
• DOI: 10.1037/law0000421

This study aims to improve the predictive performance of existing risk assessment tools and the predictive validity of the original Ministry of Justice Case Assessment Tool (MJCA) concerning recidivism rates using machine learning (ML) and examine whether the tool's predictive performance can be improved. With follow-up data on 5,942 individuals in Japanese Juvenile Assessment Centers, the study uses ML methods, such as the K-nearest neighbor algorithm, support vector machine, random forest, gradient boosting tree, and multilayer perceptron, to improve the MJCA's prediction power. The results show that the predictive validity of the original MJCA significantly improves for three of the six ML methods; gradient boosting tree, random forest, and multilayer perceptron have the highest predictive validity. The area under the receiver operating characteristic curve (AUC) for the gradient boosting tree is 0.75, significantly higher than the AUC of the original MJCA (0.67). We concluded that ML can improve the predictive validity of recidivism rates. Among the ML techniques, decision tree algorithms were better at predicting recidivism. The improvement, as with other findings, is less pronounced than the enormous impact that recent artificial intelligence methods have had on information processing. However, it is significant because recidivism risk assessment is important in determining the treatment for individuals who offend. ML is beneficial for risk assessment and must be used with a focus on these issues.