Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning

• Published in: IEEE access Vol. 7; pp. 17809 - 17822
• Main Authors: Swati, Zar Nawab Khan, Zhao, Qinghua, Kabir, Muhammad, Ali, Farman, Ali, Zakir, Ahmed, Saeed, Lu, Jianfeng
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Biomedical imaging; block-wise fine-tuning; Brain; Brain cancer; Brain tumor retrieval; closed-form metric learning; convolutional neural networks; Datasets; Deep learning; Feature extraction; Image contrast; Image enhancement; Image management; Image retrieval; Machine learning; Magnetic resonance imaging; Measurement; Medical imaging; Shape; Task analysis; transfer learning; Tumors
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2892455

This paper presents an automatic content-based image retrieval (CBIR) system for brain tumors on T1-weighted contrast-enhanced magnetic resonance images (CE-MRI). The key challenge in CBIR systems for MR images is the semantic gap between the low-level visual information captured by the MRI machine and the high-level information perceived by the human evaluator. The traditional feature extraction methods focus only on low-level or high-level features and use some handcrafted features to reduce this gap. It is necessary to design a feature extraction framework to reduce this gap without using handcrafted features by encoding/combining low-level and high-level features. Deep learning is very powerful for feature representation that can depict low-level and high-level information completely and embed the phase of feature extraction in self-learning. Therefore, we propose a deep convolutional neural network VGG19-based novel feature extraction framework and apply closed-form metric learning to measure the similarity between the query image and database images. Furthermore, we adopt transfer learning and propose a block-wise fine-tuning strategy to enhance the retrieval performance. The extensive experiments are performed on a publicly available CE-MRI dataset that consists of three types of brain tumors (i.e., glioma, meningioma, and pituitary tumor) collected from 233 patients with a total of 3064 images across the axial, coronal, and sagittal views. Our method is more generic, as we do not use any handcrafted features; it requires minimal preprocessing, tested as robust on fivefold cross-validation, can achieve a fivefold mean average precision of 96.13%, and outperforms the state-of-the-art CBIR systems on the CE-MRI dataset.