Using histopathology latent diffusion models as privacy-preserving dataset augmenters improves downstream classification performance

• Published in: Computers in biology and medicine Vol. 175; p. 108410
• Main Authors: Niehues, Jan M., Müller-Franzes, Gustav, Schirris, Yoni, Wagner, Sophia Janine, Jendrusch, Michael, Kloor, Matthias, Pearson, Alexander T., Muti, Hannah Sophie, Hewitt, Katherine J., Veldhuizen, Gregory P., Zigutyte, Laura, Truhn, Daniel, Kather, Jakob Nikolas
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01-06-2024; Elsevier Limited
• Subjects: Artificial intelligence; Biomarkers; Cancer; Classification; Classifiers; Colorectal cancer; Colorectal carcinoma; Computational pathology; Data augmentation; Datasets; Deep learning; Diffusion; Diffusion models; Digitization; Generative adversarial networks; Generative models; Histology; Histopathology; Image processing; Image quality; Medical imaging; Memory; Pathology; Patients; Privacy; Training; Diffusion models; Generative adversarial networks; Generative models; Artificial intelligence; Computational pathology; Colorectal cancer
• ISSN: 0010-4825; 1879-0534
• DOI: 10.1016/j.compbiomed.2024.108410

Latent diffusion models (LDMs) have emerged as a state-of-the-art image generation method, outperforming previous Generative Adversarial Networks (GANs) in terms of training stability and image quality. In computational pathology, generative models are valuable for data sharing and data augmentation. However, the impact of LDM-generated images on histopathology tasks compared to traditional GANs has not been systematically studied.
We trained three LDMs and a styleGAN2 model on histology tiles from nine colorectal cancer (CRC) tissue classes. The LDMs include 1) a fine-tuned version of stable diffusion v1.4, 2) a Kullback-Leibler (KL)-autoencoder (KLF8-DM), and 3) a vector quantized (VQ)-autoencoder deploying LDM (VQF8-DM). We assessed image quality through expert ratings, dimensional reduction methods, distribution similarity measures, and their impact on training a multiclass tissue classifier. Additionally, we investigated image memorization in the KLF8-DM and styleGAN2 models.
All models provided a high image quality, with the KLF8-DM achieving the best Frechet Inception Distance (FID) and expert rating scores for complex tissue classes. For simpler classes, the VQF8-DM and styleGAN2 models performed better. Image memorization was negligible for both styleGAN2 and KLF8-DM models. Classifiers trained on a mix of KLF8-DM generated and real images achieved a 4% improvement in overall classification accuracy, highlighting the usefulness of these images for dataset augmentation.
Our systematic study of generative methods showed that KLF8-DM produces the highest quality images with negligible image memorization. The higher classifier performance in the generatively augmented dataset suggests that this augmentation technique can be employed to enhance histopathology classifiers for various tasks.
•Latent Diffusion Models (LDMs) generate realistic images of histopathological tissue.•LDMs beat generative adversarial networks (GANs) across several evaluation metrics.•Our best LDM shows no data memorization.•Deployment of LDMs improves downstream classification tasks.