Handling missing values in machine learning to predict patient-specific risk of adverse cardiac events: Insights from REFINE SPECT registry

• Published in: Computers in biology and medicine Vol. 145; p. 105449
• Main Authors: Rios, Richard, Miller, Robert J.H., Manral, Nipun, Sharir, Tali, Einstein, Andrew J., Fish, Mathews B., Ruddy, Terrence D., Kaufmann, Philipp A., Sinusas, Albert J., Miller, Edward J., Bateman, Timothy M., Dorbala, Sharmila, Di Carli, Marcelo, Van Kriekinge, Serge D., Kavanagh, Paul B., Parekh, Tejas, Liang, Joanna X., Dey, Damini, Berman, Daniel S., Slomka, Piotr J.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01-06-2022; Elsevier Limited
• Subjects: Accuracy; Angina pectoris; Animal models; Blood pressure; Cardiac stress tests; Cardiovascular disease; Clinical implementation; Clinical medicine; Clusters; Continuity (mathematics); Coronary vessels; Diagnosis; Electrocardiography; Heart attacks; Humans; Laboratories; Learning algorithms; Machine Learning; Medical prognosis; Methods; Missing values; Model accuracy; Mortality; Myocardial perfusion imaging; Myocardial Perfusion Imaging - methods; Patients; Performance evaluation; Performance prediction; Perfusion; Population; Predictions; Prognosis; Questionnaires; Registries; Regression; Review boards; Risk; Single photon emission computed tomography; Software; Tomography, Emission-Computed, Single-Photon - methods; Variables; SBP; Myocardial perfusion imaging; Prognosis; CAD; MPI; ECG; AUC; DBP; MACE; Missing values; Machine learning; Clinical implementation; ML
• ISSN: 0010-4825; 1879-0534
• DOI: 10.1016/j.compbiomed.2022.105449

Machine learning (ML) models can improve prediction of major adverse cardiovascular events (MACE), but in clinical practice some values may be missing. We evaluated the influence of missing values in ML models for patient-specific prediction of MACE risk.
We included 20,179 patients from the multicenter REFINE SPECT registry with MACE follow-up data. We evaluated seven methods for handling missing values: 1) removal of variables with missing values (ML-Remove), 2) imputation with median and unique category for continuous and categorical variables, respectively (ML-Traditional), 3) unique category for missing variables (ML-Unique), 4) cluster-based imputation (ML-Cluster), 5) regression-based imputation (ML-Regression), 6) missRanger imputation (ML-MR), and 7) multiple imputation (ML-MICE). We trained ML models with full data and simulated missing values in testing patients. Prediction performance was evaluated using area under the receiver-operating characteristic curve (AUC) and compared with a model without missing values (ML-All), expert visual diagnosis and total perfusion deficit (TPD).
During mean follow-up of 4.7 ± 1.5 years, 3,541 patients experienced at least one MACE (3.7% annualized risk). ML-All (reference model-no missing values) had AUC 0.799 for MACE risk prediction. All seven models with missing values had lower AUC (ML-Remove: 0.778, ML-MICE: 0.774, ML-Cluster: 0.771, ML-Traditional: 0.771, ML-Regression: 0.770, ML-MR: 0.766, and ML-Unique: 0.766; p < 0.01 for ML-Remove vs remaining methods). Stress TPD (AUC 0.698) and visual diagnosis (0.681) had the lowest AUCs.
Missing values reduce the accuracy of ML models when predicting MACE risk. Removing variables with missing values and retraining the model may yield superior patient-level prediction performance.
•Removing variables with missing values and retraining was the most accurate method.•Multiple imputation had the highest accuracy of the imputation methods.•Having a selection of reduced ML models may be a practical clinical solution.