minoHealth.ai: A Clinical Evaluation Of Deep Learning Systems For the Diagnosis of Pleural Effusion and Cardiomegaly In Ghana, Vietnam and the United States of America

• Main Authors: Akogo, Darlington, Sarkodie, Benjamin Dabo, Samori, Issah Abubakari, Jimah, Bashiru Babatunde, Anim, Dorothea Akosua, Mensah, Yaw Boateng
• Format: Journal Article
• Language: English
• Published: 31-10-2022
• Subjects: Computer Science - Computer Vision and Pattern Recognition
• DOI: 10.48550/arxiv.2211.00644

A rapid and accurate diagnosis of cardiomegaly and pleural effusion is of the
utmost importance to reduce mortality and medical costs. Artificial
Intelligence has shown promise in diagnosing medical conditions. With this
study, we seek to evaluate how well Artificial Intelligence (AI) systems,
developed my minoHealth AI Labs, will perform at diagnosing cardiomegaly and
pleural effusion, using chest x-rays from Ghana, Vietnam and the USA, and how
well AI systems will perform when compared with radiologists working in Ghana.
The evaluation dataset used in this study contained 100 images randomly
selected from three datasets. The Deep Learning models were further tested on a
larger Ghanaian dataset containing five hundred and sixty one (561) samples.
Two AI systems were then evaluated on the evaluation dataset, whilst we also
gave the same chest x-ray images within the evaluation dataset to 4
radiologists, with 5 - 20 years experience, to diagnose independently. For
cardiomegaly, minoHealth-ai systems scored Area under the Receiver operating
characteristic Curve (AUC-ROC) of 0.9 and 0.97 while the AUC-ROC of individual
radiologists ranged from 0.77 to 0.87. For pleural effusion, the minoHealth-ai
systems scored 0.97 and 0.91 whereas individual radiologists scored between
0.75 and 0.86. On both conditions, the best performing AI model outperforms the
best performing radiologist by about 10%. We also evaluate the specificity,
sensitivity, negative predictive value (NPV), and positive predictive value
(PPV) between the minoHealth-ai systems and radiologists.