Large-Scale Study on AI’s Impact on Identifying Chest Radiographs with No Actionable Disease in Outpatient Imaging

• Published in: Academic radiology
• Main Authors: Mansoor, Awais, Schmuecking, Ingo, Ghesu, Florin C., Georgescu, Bogdan, Grbic, Sasa, Vishwanath, R.S., Farri, Oladimeji, Ghosh, Rikhiya, Vunikili, Ramya, Zimmermann, Mathis, Sutcliffe, James, Mendelsohn, Steven L., Comaniciu, Dorin, Gefter, Warren B.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 12-07-2024
• Subjects: NAD; PA; FCOS; PAD; AD; NLP; PPV; AI; CXR; GT; AP
• ISSN: 1076-6332; 1878-4046; 1878-4046
• DOI: 10.1016/j.acra.2024.06.031

Given the high volume of chest radiographs, radiologists frequently encounter heavy workloads. In outpatient imaging, a substantial portion of chest radiographs show no actionable findings. Automatically identifying these cases could improve efficiency by facilitating shorter reading workflows.
A large-scale study to assess the performance of AI on identifying chest radiographs with no actionable disease (NAD) in an outpatient imaging population using comprehensive, objective, and reproducible criteria for NAD.
The independent validation study includes 15000 patients with chest radiographs in posterior-anterior (PA) and lateral projections from an outpatient imaging center in the United States. Ground truth was established by reviewing CXR reports and classifying cases as NAD or actionable disease (AD). The NAD definition includes completely normal chest radiographs and radiographs with well-defined non-actionable findings. The AI NAD Analyzer11For research purposes only. Not for clinical use. Future commercial availability cannot be guaranteed. (trained with 100 million multimodal images and fine-tuned on 1.3 million radiographs) utilizes a tandem system with image-level rule in and compartment-level rule out to provide case level output as NAD or potential actionable disease (PAD).
A total of 14057 cases met our eligibility criteria (age 56 ± 16.1 years, 55% women and 45% men). The prevalence of NAD cases in the study population was 70.7%. The AI NAD Analyzer correctly classified NAD cases with a sensitivity of 29.1% and a yield of 20.6%. The specificity was 98.9% which corresponds to a miss rate of 0.3% of cases. Significant findings were missed in 0.06% of cases, while no cases with critical findings were missed by AI.
In an outpatient population, AI can identify 20% of chest radiographs as NAD with a very low rate of missed findings. These cases could potentially be read using a streamlined protocol, thus improving efficiency and consequently reducing daily workload for radiologists.