Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre study

• Published in: The clinical respiratory journal Vol. 17; no. 1; pp. 40 - 49
• Main Authors: Engoren, Milo, Pancaro, Carlo, Yeldo, Nicholas S., Kerzabi, Lotfi S., Douville, Nicholas
• Format: Journal Article
• Language: English
• Published: England John Wiley & Sons, Inc 01-01-2023; John Wiley and Sons Inc; Wiley
• Subjects: Adult; COVID; COVID-19; COVID-19 - epidemiology; COVID-19 - therapy; death, clinical decision support, clinical prediction models; Humans; Laboratories; Logistic Models; logistic regression; mechanical ventilation; Original; Regression analysis; Respiration, Artificial; Retrospective Studies; SARS-CoV-2; Ventilation; COVID; death, clinical decision support, clinical prediction models; mechanical ventilation; logistic regression
• ISSN: 1752-6981; 1752-699X
• DOI: 10.1111/crj.13560

Introduction
COVID‐19 virus has undergone mutations, and the introduction of vaccines and effective treatments have changed its clinical severity. We hypothesized that models that evolve may better predict invasive mechanical ventilation or death than do static models.
Methods
This retrospective study of adult patients with COVID‐19 from six Michigan hospitals analysed 20 demographic, comorbid, vital sign and laboratory factors, one derived factor and nine factors representing changes in vital signs or laboratory values with time for their ability to predict death or invasive mechanical ventilation within the next 4, 8 or 24 h. Static logistic regression was constructed on the initial 300 patients and tested on the remaining 6741 patients. Rolling logistic regression was similarly constructed on the initial 300 patients, but then new patients were added, and older patients removed. Each new construction model was subsequently tested on the next patient. Static and rolling models were compared with receiver operator characteristic and precision‐recall curves.
Results
Of the 7041 patients, 534 (7.6%) required invasive mechanical ventilation or died within 14 days of arrival. Rolling models improved discrimination (0.865 ± 0.010, 0.856 ± 0.007 and 0.843 ± 0.005 for the 4, 8 and 24‐h models, respectively; all p < 0.001 compared with the static logistic regressions with 0.827 ± 0.011, 0.794 ± 0.012 and 0.735 ± 0.012, respectively). Similarly, the areas under the precision‐recall curves improved from 0.006, 0.010 and 0.021 with the static models to 0.030, 0.045 and 0.076 for the 4‐, 8‐ and 24‐h rolling models, respectively, all p < 0.001.
Conclusion
Rolling models with contemporaneous data maintained better metrics of performance than static models, which used older data.
When a disease or treatment is rapidly changing, traditional (static) predictive models may lose their accuracy. We show that in 7041 patients of whom 534 (7.6%) required invasive mechanical ventilation or died within 14 days that rolling logistic regression models improved both discrimination and the areas under the precision‐recall curves—all p < 0.001.