Beware of diffusion models for synthesizing medical images—a comparison with GANs in terms of memorizing brain MRI and chest x-ray images
(2025) In Machine Learning: Science and Technology 6(1).- Abstract
Diffusion models were initially developed for text-to-image generation and are now being utilized to generate high quality synthetic images. Preceded by generative adversarial networks (GANs), diffusion models have shown impressive results using various evaluation metrics. However, commonly used metrics such as Frechet inception distance and inception score are not suitable for determining whether diffusion models are simply reproducing the training images. Here we train StyleGAN and a diffusion model, using BRATS20, BRATS21 and a chest x-ray (CXR) pneumonia dataset, to synthesize brain MRI and CXR images, and measure the correlation between the synthetic images and all training images. Our results show that diffusion models are more... (More)
Diffusion models were initially developed for text-to-image generation and are now being utilized to generate high quality synthetic images. Preceded by generative adversarial networks (GANs), diffusion models have shown impressive results using various evaluation metrics. However, commonly used metrics such as Frechet inception distance and inception score are not suitable for determining whether diffusion models are simply reproducing the training images. Here we train StyleGAN and a diffusion model, using BRATS20, BRATS21 and a chest x-ray (CXR) pneumonia dataset, to synthesize brain MRI and CXR images, and measure the correlation between the synthetic images and all training images. Our results show that diffusion models are more likely to memorize the training images, compared to StyleGAN, especially for small datasets and when using 2D slices from 3D volumes. Researchers should be careful when using diffusion models (and to some extent GANs) for medical imaging, if the final goal is to share the synthetic images.
(Less)
- author
- Usman Akbar, Muhammad LU ; Wang, Wuhao and Eklund, Anders
- organization
- publishing date
- 2025-03
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- brain MRI, CXR, diffusion model, GANs, generative AI, memorization, synthetic data
- in
- Machine Learning: Science and Technology
- volume
- 6
- issue
- 1
- article number
- 015022
- publisher
- IOP Publishing
- external identifiers
-
- scopus:85217039477
- ISSN
- 2632-2153
- DOI
- 10.1088/2632-2153/ad9a3a
- language
- English
- LU publication?
- yes
- id
- 6122d883-cb2e-4ea6-8d31-be1a84588003
- date added to LUP
- 2025-03-21 16:12:08
- date last changed
- 2025-06-28 00:29:21
@article{6122d883-cb2e-4ea6-8d31-be1a84588003,
abstract = {{<p>Diffusion models were initially developed for text-to-image generation and are now being utilized to generate high quality synthetic images. Preceded by generative adversarial networks (GANs), diffusion models have shown impressive results using various evaluation metrics. However, commonly used metrics such as Frechet inception distance and inception score are not suitable for determining whether diffusion models are simply reproducing the training images. Here we train StyleGAN and a diffusion model, using BRATS20, BRATS21 and a chest x-ray (CXR) pneumonia dataset, to synthesize brain MRI and CXR images, and measure the correlation between the synthetic images and all training images. Our results show that diffusion models are more likely to memorize the training images, compared to StyleGAN, especially for small datasets and when using 2D slices from 3D volumes. Researchers should be careful when using diffusion models (and to some extent GANs) for medical imaging, if the final goal is to share the synthetic images.</p>}},
author = {{Usman Akbar, Muhammad and Wang, Wuhao and Eklund, Anders}},
issn = {{2632-2153}},
keywords = {{brain MRI; CXR; diffusion model; GANs; generative AI; memorization; synthetic data}},
language = {{eng}},
number = {{1}},
publisher = {{IOP Publishing}},
series = {{Machine Learning: Science and Technology}},
title = {{Beware of diffusion models for synthesizing medical images—a comparison with GANs in terms of memorizing brain MRI and chest x-ray images}},
url = {{http://dx.doi.org/10.1088/2632-2153/ad9a3a}},
doi = {{10.1088/2632-2153/ad9a3a}},
volume = {{6}},
year = {{2025}},
}