Learning to detect chest radiographs containing pulmonary lesions using visual attention networks

• Published in: Medical image analysis Vol. 53; pp. 26 - 38
• Main Authors: Pesce, Emanuele, Joseph Withey, Samuel, Ypsilantis, Petros-Pavlos, Bakewell, Robert, Goh, Vicky, Montana, Giovanni
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01-04-2019; Elsevier BV
• Subjects: Algorithms; Annotations; Archives & records; Archiving; Attention; Back propagation; Chest; Communications systems; Datasets as Topic; Deep learning; Diagnosis, Computer-Assisted - methods; Humans; Image classification; Learning algorithms; Lung cancer; Lung Diseases - diagnostic imaging; Lung nodules; Machine learning; Natural language processing; Neural networks; Neural Networks, Computer; Nodules; Object localisation; Position (location); Pulmonary lesions; Radiographic Image Interpretation, Computer-Assisted - methods; Radiographs; Radiography; Radiography, Thoracic; Reinforcement; Training; Visual attention; Visual discrimination learning; Visual perception; X-rays; Object localisation; Deep learning; Visual attention; Lung cancer; X-rays; Image classification
• ISSN: 1361-8415; 1361-8423
• DOI: 10.1016/j.media.2018.12.007

•We propose two novel neural network architectures to detect pulmonary lesions in chest x-rays imagesthat use visual attention mechanisms.•The architectures are designed to learn from a large number of weakly-labelled images (class labels only) and a small number of annotated images (bounding boxes).•The algorithms and trained and tested using a large dataset of over 700k chest x-ray exams extracted from two hospital’s archives and processed using a novel NLP pipeline.•Results are validated on 6,131 manually labelled exams from two independent radiologists.•Our experimental resultsare either in line or better than currently reported performance from other studies.
[Display omitted]
Machine learning approaches hold great potential for the automated detection of lung nodules on chest radiographs, but training algorithms requires very large amounts of manually annotated radiographs, which are difficult to obtain. The increasing availability of PACS (Picture Archiving and Communication System), is laying the technological foundations needed to make available large volumes of clinical data and images from hospital archives. Binary labels indicating whether a radiograph contains a pulmonary lesion can be extracted at scale, using natural language processing algorithms. In this study, we propose two novel neural networks for the detection of chest radiographs containing pulmonary lesions. Both architectures make use of a large number of weakly-labelled images combined with a smaller number of manually annotated x-rays. The annotated lesions are used during training to deliver a type of visual attention feedback informing the networks about their lesion localisation performance. The first architecture extracts saliency maps from high-level convolutional layers and compares the inferred position of a lesion against the true position when this information is available; a localisation error is then back-propagated along with the softmax classification error. The second approach consists of a recurrent attention model that learns to observe a short sequence of smaller image portions through reinforcement learning; the reward function penalises the exploration of areas, within an image, that are unlikely to contain nodules. Using a repository of over 430,000 historical chest radiographs, we present and discuss the proposed methods over related architectures that use either weakly-labelled or annotated images only.