Limitations when using artificial intelligence services to analyze chest X-rays

• Published in: Digital diagnostics
• Main Authors: Vasilev, Yuriy A., Vladzymyrskyy, Anton V., Arzamasov, Kirill M., Shulkin, Igor M., Astapenko, Elena V., Pestrenin, Lev D.
• Format: Journal Article
• Language: English
• Published: 16-10-2024
• ISSN: 2712-8490; 2712-8962
• DOI: 10.17816/DD626310

Background: One of the first radiology areas in which artificial intelligence began to be used and is still actively used to this day is chest X-ray examination. However, when interpreting these studies using artificial intelligence, radiologists still face a number of limitations on a daily basis that must be taken into account when making a medical opinion and which developers need to pay attention to in order to further improve the algorithms to increase their efficiency. Aims: Identification of limitations in the use of currently available artificial intelligence services for chest X-ray examinations and identification of promising directions for their further development. Materials and methods: A retrospective analysis of 155 cases of disagreement between the results of conclusions of artificial intelligence services and medical opinions when analyzing chest X-ray examinations was carried out. All cases included in the study were obtained from the Unified Radiological Information Service of the Unified Medical Information and Analytical System of Moscow. Results: Among the 155 analyzed cases of disagreement, 48 (31.0%) were false positives and 78 (50.3%) were false negatives. The remaining 29 (18.7%) cases were excluded from further study because they turned out to be true positive (27) or true negative (2). Among the 48 false-positive cases, the majority (93.8%) was due to the fact that the artificial intelligence service mistook normal anatomical structures of the chest (97.8% of cases) or a catheter shadow (2.2% of cases) for pneumothorax. Among false-negative studies, the proportion of missed clinically significant pathologies was 22.0%. Almost half of these cases (44.4%) were associated with missed lung nodes. The most common clinically insignificant pathology was calcifications in the lungs (60.9%). Conclusions: On the part of AI services, there was a tendency towards overdiagnosis. All false-positive cases were associated with erroneous detection of clinically significant pathology: pneumothorax, lung nodules, and pulmonary consolidation. Among false-negative cases, the proportion of missing clinically significant pathology was small and amounted to less than one-fourth.