Segmented Harmonic Loss: Handling Class-Imbalanced Multi-Label Clinical Data for Medical Coding with Large Language Models

• Main Authors: Ray, Surjya, Mehta, Pratik, Zhang, Hongen, Chaman, Ada, Wang, Jian, Ho, Chung-Jen, Chiou, Michael, Suleman, Tashfeen
• Format: Journal Article
• Language: English
• Published: 06-10-2023
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computation and Language
• DOI: 10.48550/arxiv.2310.04595

The precipitous rise and adoption of Large Language Models (LLMs) have
shattered expectations with the fastest adoption rate of any consumer-facing
technology in history. Healthcare, a field that traditionally uses NLP
techniques, was bound to be affected by this meteoric rise. In this paper, we
gauge the extent of the impact by evaluating the performance of LLMs for the
task of medical coding on real-life noisy data. We conducted several
experiments on MIMIC III and IV datasets with encoder-based LLMs, such as BERT.
Furthermore, we developed Segmented Harmonic Loss, a new loss function to
address the extreme class imbalance that we found to prevail in most medical
data in a multi-label scenario by segmenting and decoupling co-occurring
classes of the dataset with a new segmentation algorithm. We also devised a
technique based on embedding similarity to tackle noisy data. Our experimental
results show that when trained with the proposed loss, the LLMs achieve
significant performance gains even on noisy long-tailed datasets, outperforming
the F1 score of the state-of-the-art by over ten percentage points.