Hierarchical reinforcement learning for automatic disease diagnosis

• Published in: Bioinformatics Vol. 38; no. 16; pp. 3995 - 4001
• Main Authors: Zhong, Cheng, Liao, Kangenbei, Chen, Wei, Liu, Qianlong, Peng, Baolin, Huang, Xuanjing, Peng, Jiajie, Wei, Zhongyu
• Format: Journal Article
• Language: English
• Published: England Oxford University Press 10-08-2022
• Subjects: Algorithms; Benchmarking; Deep Learning; Markov Chains
• ISSN: 1367-4803; 1460-2059; 1367-4811
• DOI: 10.1093/bioinformatics/btac408

Abstract
Motivation
Disease diagnosis-oriented dialog system models the interactive consultation procedure as the Markov decision process, and reinforcement learning algorithms are used to solve the problem. Existing approaches usually employ a flat policy structure that treat all symptoms and diseases equally for action making. This strategy works well in a simple scenario when the action space is small; however, its efficiency will be challenged in the real environment. Inspired by the offline consultation process, we propose to integrate a hierarchical policy structure of two levels into the dialog system for policy learning. The high-level policy consists of a master model that is responsible for triggering a low-level model, the low-level policy consists of several symptom checkers and a disease classifier. The proposed policy structure is capable to deal with diagnosis problem including large number of diseases and symptoms.
Results
Experimental results on three real-world datasets and a synthetic dataset demonstrate that our hierarchical framework achieves higher accuracy and symptom recall in disease diagnosis compared with existing systems. We construct a benchmark including datasets and implementation of existing algorithms to encourage follow-up researches.
Availability and implementation
The code and data are available from https://github.com/FudanDISC/DISCOpen-MedBox-DialoDiagnosis
Supplementary information
Supplementary data are available at Bioinformatics online.