Variational AdaBoost knowledge distillation for skin lesion classification in dermatology images

Knowledge Distillation has shown promising results for classifying skin lesions in dermatology images. Traditional knowledge distillation typically involves the student model passively mimicking the teacher model's knowledge. We propose utilizing AdaBoost to enable the student to actively mine...

Full description

Saved in:

Bibliographic Details
Published in:	Complex & intelligent systems Vol. 10; no. 5; pp. 6787 - 6804
Main Authors:	Yu, Xiangchun, Xiong, Guoliang, Wu, Jianqing, Zheng, Jian, Liang, Miaomiao, Qiu, Liujin, Yu, Lingjuan, Xu, Qing
Format:	Journal Article
Language:	English
Published:	Cham Springer International Publishing 01-10-2024 Springer Nature B.V Springer
Subjects:	AdaBoost Classification Complexity Computational Intelligence Data Structures and Information Theory Datasets Dermatology Dermatology images Engineering Knowledge Knowledge distillation Knowledge representation Learning Learning disabilities Lesions Machine learning Original Article Skin Skin lesion classification Teachers Variational difficulty mining strategy Skin lesion classification AdaBoost Dermatology images Variational difficulty mining strategy Knowledge distillation
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	Knowledge Distillation has shown promising results for classifying skin lesions in dermatology images. Traditional knowledge distillation typically involves the student model passively mimicking the teacher model's knowledge. We propose utilizing AdaBoost to enable the student to actively mine the teacher's learning representation for skin lesion classification. This paradigm allows the student to determine the “granularity” in mining the teacher's knowledge. As the student's learning process progresses, it can become challenging to pinpoint specific learning difficulties, especially with potential interference from the teacher. To address this issue, we introduce a variational difficulty mining strategy to reduce the impact of such interference. This strategy involves the distillation module capturing more nuanced classification difficulties by extracting information from the node's l th hops. By maximizing the mutual information between the teacher and student, we effectively filter out noise interference from these nuanced difficulties. Our proposed framework, Variational AdaBoost Knowledge Distillation (VAdaKD), allows the student to actively mine and leverage the teacher's knowledge for improved skin lesion classification. Our proposed method performs satisfactorily on three benchmark datasets: the Dermnet dataset, ISIC 2019 dataset, and HAM10000 dataset, respectively. Specifically, our method shows an improvement of 2–3% over the baseline on the Dermnet dataset and outperforms the best results of the other compared methods by 1%. Experimental results and visualization performance indicate that our proposed method effectively captures the learning difficulties and achieves better visualized t-distributed stochastic neighbor embedding classification results. Our code is available at https://github.com/25brilliant/VAdaKD .
ISSN:	2199-4536 2198-6053
DOI:	10.1007/s40747-024-01501-4