Evaluation of the impact of artificial intelligence-assisted image interpretation on the diagnostic performance of clinicians in identifying pneumothoraces on plain chest X-ray: a multi-case multi-reader study

Artificial intelligence (AI)-assisted image interpretation is a fast-developing area of clinical innovation. Most research to date has focused on the performance of AI-assisted algorithms in comparison with that of radiologists rather than evaluating the algorithms' impact on the clinicians who...

Full description

Saved in:

Bibliographic Details
Published in:	Emergency medicine journal : EMJ Vol. 41; no. 10; p. 602
Main Authors:	Novak, Alex, Ather, Sarim, Gill, Avneet, Aylward, Peter, Maskell, Giles, Cowell, Gordon W, Espinosa Morgado, Abdala Trinidad, Duggan, Tom, Keevill, Melissa, Gamble, Olivia, Akrama, Osama, Belcher, Elizabeth, Taberham, Rhona, Hallifax, Rob, Bahra, Jasdeep, Banerji, Abhishek, Bailey, Jon, James, Antonia, Ansaripour, Ali, Spence, Nathan, Wrightson, John, Jarral, Waqas, Barry, Steven, Bhatti, Saher, Astley, Kerry, Shadmaan, Amied, Ghelman, Sharon, Baenen, Alec, Oke, Jason, Bloomfield, Claire, Johnson, Hilal, Beggs, Mark, Gleeson, Fergus
Format:	Journal Article
Language:	English
Published:	England 15-07-2024
Subjects:	Algorithms Artificial Intelligence Clinical Competence - standards Female Humans Male Pneumothorax - diagnostic imaging Radiography, Thoracic - methods Sensitivity and Specificity emergency department chest pneumothorax
Online Access:	Get more information
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	Artificial intelligence (AI)-assisted image interpretation is a fast-developing area of clinical innovation. Most research to date has focused on the performance of AI-assisted algorithms in comparison with that of radiologists rather than evaluating the algorithms' impact on the clinicians who often undertake initial image interpretation in routine clinical practice. This study assessed the impact of AI-assisted image interpretation on the diagnostic performance of frontline acute care clinicians for the detection of pneumothoraces (PTX). A multicentre blinded multi-case multi-reader study was conducted between October 2021 and January 2022. The online study recruited 18 clinician readers from six different clinical specialties, with differing levels of seniority, across four English hospitals. The study included 395 plain CXR images, 189 positive for PTX and 206 negative. The reference standard was the consensus opinion of two thoracic radiologists with a third acting as arbitrator. General Electric Healthcare Critical Care Suite (GEHC CCS) PTX algorithm was applied to the final dataset. Readers individually interpreted the dataset without AI assistance, recording the presence or absence of a PTX and a confidence rating. Following a 'washout' period, this process was repeated including the AI output. Analysis of the performance of the algorithm for detecting or ruling out a PTX revealed an overall AUROC of 0.939. Overall reader sensitivity increased by 11.4% (95% CI 4.8, 18.0, p=0.002) from 66.8% (95% CI 57.3, 76.2) unaided to 78.1% aided (95% CI 72.2, 84.0, p=0.002), specificity 93.9% (95% CI 90.9, 97.0) without AI to 95.8% (95% CI 93.7, 97.9, p=0.247). The junior reader subgroup showed the largest improvement at 21.7% (95% CI 10.9, 32.6), increasing from 56.0% (95% CI 37.7, 74.3) to 77.7% (95% CI 65.8, 89.7, p<0.01). The study indicates that AI-assisted image interpretation significantly enhances the diagnostic accuracy of clinicians in detecting PTX, particularly benefiting less experienced practitioners. While overall interpretation time remained unchanged, the use of AI improved diagnostic confidence and sensitivity, especially among junior clinicians. These findings underscore the potential of AI to support less skilled clinicians in acute care settings.
ISSN:	1472-0213
DOI:	10.1136/emermed-2023-213620