Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Impact of deep learning model uncertainty on manual corrections to MRI-based auto-segmentation in prostate cancer radiotherapy

Rogowski, Viktor LU ; Svalkvist, Angelica ; Maspero, Matteo ; Janssen, Tomas ; Maruccio, Federica Carmen ; Gorgisyan, Jenny LU orcid ; Scherman, Jonas ; Häggström, Ida ; Wåhlstrand, Victor and Gunnlaugsson, Adalsteinn LU , et al. (2025) In Journal of Applied Clinical Medical Physics 26(9).
Abstract

Background: Deep learning (DL)-based organ segmentation is increasingly used in radiotherapy. While methods exist to generate voxel-wise uncertainty maps from DL-based auto-segmentation models, these maps are rarely presented to clinicians. Purpose: This study aimed to evaluate the impact of DL-generated uncertainty maps on experienced radiation oncologists during the manual correction of DL-based auto-segmentation for prostate radiotherapy. Methods: Two nnUNet DL models were trained with 10-fold cross-validation on a dataset of 434 patient cases undergoing ultra-hypofractionated MRI-only radiotherapy for prostate cancer. The models performed prostate clinical target volume (CTV) and rectum segmentation. Each cross-validation model was... (More)

Background: Deep learning (DL)-based organ segmentation is increasingly used in radiotherapy. While methods exist to generate voxel-wise uncertainty maps from DL-based auto-segmentation models, these maps are rarely presented to clinicians. Purpose: This study aimed to evaluate the impact of DL-generated uncertainty maps on experienced radiation oncologists during the manual correction of DL-based auto-segmentation for prostate radiotherapy. Methods: Two nnUNet DL models were trained with 10-fold cross-validation on a dataset of 434 patient cases undergoing ultra-hypofractionated MRI-only radiotherapy for prostate cancer. The models performed prostate clinical target volume (CTV) and rectum segmentation. Each cross-validation model was evaluated on an independent test set of 35 patient cases. Segmentation uncertainty was calculated voxel-wise as the SoftMax standard deviation (0–0.5, n = 10) and visualized as a fixed scale color-coded map. Four experienced oncologists were asked to:. Step 1: Rate the quality of and confidence in the DL segmentations using a four- and five-point Likert scale, respectively, and edit the segmentations without access to the uncertainty map. Step 2: Repeat step 1 after at least 4 weeks, but this time with the color-coded uncertainty map available. Oncologists were asked to blend the uncertainty map with the DL segmentation and MRI volume. Segmentation edit time was recorded for both steps. In step 2, oncologists also provided free-text feedback on the benefits and drawbacks of using the uncertainty map during segmentation. A histogram analysis was performed to compare the number of voxels edited between step 1 and step 2 for different uncertainty levels (bins with 0.1 intervals). Results: The DL models achieved high-quality segmentations with a mean Dice coefficient per oncologist of 0.97–0.99, calculated between edited and unedited segmentation in step 1 for the prostate CTV and rectum. While the overall quality rating for rectum segmentations decreased slightly on a group level in step 2 compared to step 1, individual responses varied. Some oncologists rated the quality higher for the prostate CTV segmentation with the uncertainty map present, while others rated it lower. Similarly, confidence ratings varied across oncologists for prostate CTV and rectum. Decreased segmentation time was recorded for three oncologists using uncertainty maps, saving 1–2 min per patient case, corresponding to 14%–33% time reduction. Three oncologists found the uncertainty maps helpful, and one reported benefit was the ability to identify regions of interest more quickly. The histogram analysis had fewer voxel edits in regions of low uncertainty in step 2 compared to step 1. Specifically, 50% fewer voxel edits were recorded for the uncertainty region 0.0–0.1, suggesting increased trust in the DL model's prediction in these areas. Conclusions: Presenting DL uncertainty information to experienced radiation oncologists influences their decision-making, quality perception, and confidence in the DL segmentations. Regions with low uncertainty were less likely to be edited, indicating increased reliance on the model's predictions. Additionally, uncertainty maps can improve efficiency by reducing segmentation time. DL-based segmentation uncertainty can be a valuable tool in clinical practice, enhancing the efficiency of radiotherapy planning.

(Less)
Please use this url to cite or link to this publication:
author
; ; ; ; ; ; ; ; and , et al. (More)
; ; ; ; ; ; ; ; ; ; ; ; ; and (Less)
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
deep learning, delineation, radiation therapy, segmentation
in
Journal of Applied Clinical Medical Physics
volume
26
issue
9
article number
e70221
publisher
American College of Medical Physics
external identifiers
  • pmid:40849835
  • scopus:105014102055
ISSN
1526-9914
DOI
10.1002/acm2.70221
language
English
LU publication?
yes
id
76160283-2f16-4d03-9ff9-51f9d45b0acf
date added to LUP
2025-10-16 15:47:19
date last changed
2025-11-13 17:49:00
@article{76160283-2f16-4d03-9ff9-51f9d45b0acf,
  abstract     = {{<p>Background: Deep learning (DL)-based organ segmentation is increasingly used in radiotherapy. While methods exist to generate voxel-wise uncertainty maps from DL-based auto-segmentation models, these maps are rarely presented to clinicians. Purpose: This study aimed to evaluate the impact of DL-generated uncertainty maps on experienced radiation oncologists during the manual correction of DL-based auto-segmentation for prostate radiotherapy. Methods: Two nnUNet DL models were trained with 10-fold cross-validation on a dataset of 434 patient cases undergoing ultra-hypofractionated MRI-only radiotherapy for prostate cancer. The models performed prostate clinical target volume (CTV) and rectum segmentation. Each cross-validation model was evaluated on an independent test set of 35 patient cases. Segmentation uncertainty was calculated voxel-wise as the SoftMax standard deviation (0–0.5, n = 10) and visualized as a fixed scale color-coded map. Four experienced oncologists were asked to:. Step 1: Rate the quality of and confidence in the DL segmentations using a four- and five-point Likert scale, respectively, and edit the segmentations without access to the uncertainty map. Step 2: Repeat step 1 after at least 4 weeks, but this time with the color-coded uncertainty map available. Oncologists were asked to blend the uncertainty map with the DL segmentation and MRI volume. Segmentation edit time was recorded for both steps. In step 2, oncologists also provided free-text feedback on the benefits and drawbacks of using the uncertainty map during segmentation. A histogram analysis was performed to compare the number of voxels edited between step 1 and step 2 for different uncertainty levels (bins with 0.1 intervals). Results: The DL models achieved high-quality segmentations with a mean Dice coefficient per oncologist of 0.97–0.99, calculated between edited and unedited segmentation in step 1 for the prostate CTV and rectum. While the overall quality rating for rectum segmentations decreased slightly on a group level in step 2 compared to step 1, individual responses varied. Some oncologists rated the quality higher for the prostate CTV segmentation with the uncertainty map present, while others rated it lower. Similarly, confidence ratings varied across oncologists for prostate CTV and rectum. Decreased segmentation time was recorded for three oncologists using uncertainty maps, saving 1–2 min per patient case, corresponding to 14%–33% time reduction. Three oncologists found the uncertainty maps helpful, and one reported benefit was the ability to identify regions of interest more quickly. The histogram analysis had fewer voxel edits in regions of low uncertainty in step 2 compared to step 1. Specifically, 50% fewer voxel edits were recorded for the uncertainty region 0.0–0.1, suggesting increased trust in the DL model's prediction in these areas. Conclusions: Presenting DL uncertainty information to experienced radiation oncologists influences their decision-making, quality perception, and confidence in the DL segmentations. Regions with low uncertainty were less likely to be edited, indicating increased reliance on the model's predictions. Additionally, uncertainty maps can improve efficiency by reducing segmentation time. DL-based segmentation uncertainty can be a valuable tool in clinical practice, enhancing the efficiency of radiotherapy planning.</p>}},
  author       = {{Rogowski, Viktor and Svalkvist, Angelica and Maspero, Matteo and Janssen, Tomas and Maruccio, Federica Carmen and Gorgisyan, Jenny and Scherman, Jonas and Häggström, Ida and Wåhlstrand, Victor and Gunnlaugsson, Adalsteinn and Nilsson, Martin P. and Moreau, Mathieu and Vass, Nándor and Pettersson, Niclas and Gustafsson, Christian Jamtheim}},
  issn         = {{1526-9914}},
  keywords     = {{deep learning; delineation; radiation therapy; segmentation}},
  language     = {{eng}},
  number       = {{9}},
  publisher    = {{American College of Medical Physics}},
  series       = {{Journal of Applied Clinical Medical Physics}},
  title        = {{Impact of deep learning model uncertainty on manual corrections to MRI-based auto-segmentation in prostate cancer radiotherapy}},
  url          = {{http://dx.doi.org/10.1002/acm2.70221}},
  doi          = {{10.1002/acm2.70221}},
  volume       = {{26}},
  year         = {{2025}},
}