DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning

• Published in: Journal of medical imaging (Bellingham, Wash.) Vol. 5; no. 3; p. 036501
• Main Authors: Yan, Ke, Wang, Xiaosong, Lu, Le, Summers, Ronald M
• Format: Journal Article
• Language: English
• Published: United States Society of Photo-Optical Instrumentation Engineers 01-07-2018
• Subjects: PACS and Imaging Informatics; lesion detection; deep learning; bookmark; medical image dataset; convolutional neural network; picture archiving and communication system
• ISSN: 2329-4302; 2329-4310
• DOI: 10.1117/1.JMI.5.3.036501

Extracting, harvesting, and building large-scale annotated radiological image datasets is a greatly important yet challenging problem. Meanwhile, vast amounts of clinical annotations have been collected and stored in hospitals' picture archiving and communication systems (PACS). These types of annotations, also known as bookmarks in PACS, are usually marked by radiologists during their daily workflow to highlight significant image findings that may serve as reference for later studies. We propose to mine and harvest these abundant retrospective medical data to build a large-scale lesion image dataset. Our process is scalable and requires minimum manual annotation effort. We mine bookmarks in our institute to develop DeepLesion, a dataset with 32,735 lesions in 32,120 CT slices from 10,594 studies of 4,427 unique patients. There are a variety of lesion types in this dataset, such as lung nodules, liver tumors, enlarged lymph nodes, and so on. It has the potential to be used in various medical image applications. Using DeepLesion, we train a universal lesion detector that can find all types of lesions with one unified framework. In this challenging task, the proposed lesion detector achieves a sensitivity of 81.1% with five false positives per image.