Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Evalutation of two commercial deep learning OAR segmentation models for prostate cancer treatment

Gorgisyan, Jenny LU ; Bengtsson, Ida ; Lempart, Michael LU ; Lerner, Minna LU ; Wieslander, Elinore LU ; Alkner, Sara LU and Jamtheim Gustafsson, Christian LU (2022) ESTRO 2022
Abstract

Purpose or Objective

To evaluate two commercial, CE labeled deep learning-based models for automatic organs at risk segmentation on planning CT images for prostate cancer radiotherapy. Model evaluation was focused on assessing both geometrical metrics and evaluating a potential time saving.
Material and Methods

The evaluated models consisted of RayStation 10B Deep Learning Segmentation (RaySearch Laboratories AB, Stockholm, Sweden) and MVision AI Segmentation Service (MVision, Helsinki, Finland) and were applied to CT images for a dataset of 54 male pelvis patients. The RaySearch model was re-trained with 44 clinic specific patients (Skåne University Hospital, Lund, Sweden) for the femoral head structures to... (More)

Purpose or Objective

To evaluate two commercial, CE labeled deep learning-based models for automatic organs at risk segmentation on planning CT images for prostate cancer radiotherapy. Model evaluation was focused on assessing both geometrical metrics and evaluating a potential time saving.
Material and Methods

The evaluated models consisted of RayStation 10B Deep Learning Segmentation (RaySearch Laboratories AB, Stockholm, Sweden) and MVision AI Segmentation Service (MVision, Helsinki, Finland) and were applied to CT images for a dataset of 54 male pelvis patients. The RaySearch model was re-trained with 44 clinic specific patients (Skåne University Hospital, Lund, Sweden) for the femoral head structures to adjust the model to our specific delineation guidelines. The model was evaluated on 10 patients from the same clinic. Dice similarity coefficient (DSC) and Hausdorff distance (95th percentile) was computed for model evaluation, using an in-house developed Python script. The average time for manual and AI model delineations was recorded.
Results

Average DSC scores and Hausdorff distances for all patients and both models are presented in Figure 1 and Table 1, respectively. The femoral head segmentations in the re-trained RaySearch model had increased overlap with our clinical data, with a DSC (mean±1 STD) for the right femoral head of 0.55±0.06 (n=53) increasing to 0.91±0.02 (n=10) and mean Hausdorff (mm) decreasing from 55±7 (n=53) to 4±1 (n=10) (similar results for the left femoral head). The deviation in femoral head compared to the RaySearch and MVision original models occurred due to a difference in the femoral head segmentation guideline in the clinic specific data, see Figure 2. Time recording of manual delineation was 13 minutes compared to 0.5 minutes (RaySearch) and 1.4 minutes (MVision) for the AI models, manual correction not included.

Conclusion

Both AI models demonstrate good segmentation performance for bladder and rectum. Clinic specific training data (or data that complies to the clinic specific delineation guideline) might be necessary to achieve segmentation results in accordance to the clinical specific standard for some anatomical structures, such as the femoral heads in our case. The time saving was around 90%, not including manual correction. (Less)
Please use this url to cite or link to this publication:
author
; ; ; ; ; and
organization
publishing date
type
Contribution to conference
publication status
published
subject
conference name
ESTRO 2022
conference location
Köpenhamn, Denmark
conference dates
2022-05-07 - 2022-05-10
project
Evaluation of AI-based vendor solutions for automatic organ-at-risk and target segmentation for head-and-neck, brain and prostate radiation therapy treatment planning
language
English
LU publication?
yes
id
826ceeaa-cdff-49bd-a661-60033a1ad028
alternative location
https://www.estro.org/Congresses/ESTRO-2022/661/radiomics-modellingandstatisticalmethods/11383/evaluationoftwocommercialdeeplearningoarsegmentati
date added to LUP
2022-11-15 17:52:15
date last changed
2022-11-16 09:49:58
@misc{826ceeaa-cdff-49bd-a661-60033a1ad028,
  abstract     = {{<br/>Purpose or Objective<br/><br/>To evaluate two commercial, CE labeled deep learning-based models for automatic organs at risk segmentation on planning CT images for prostate cancer radiotherapy. Model evaluation was focused on assessing both geometrical metrics and evaluating a potential time saving.<br/>Material and Methods<br/><br/>The evaluated models consisted of RayStation 10B Deep Learning Segmentation (RaySearch Laboratories AB, Stockholm, Sweden) and MVision AI Segmentation Service (MVision, Helsinki, Finland) and were applied to CT images for a dataset of 54 male pelvis patients. The RaySearch model was re-trained with 44 clinic specific patients (Skåne University Hospital, Lund, Sweden) for the femoral head structures to adjust the model to our specific delineation guidelines. The model was evaluated on 10 patients from the same clinic. Dice similarity coefficient (DSC) and Hausdorff distance (95th percentile) was computed for model evaluation, using an in-house developed Python script. The average time for manual and AI model delineations was recorded.<br/>Results<br/><br/>Average DSC scores and Hausdorff distances for all patients and both models are presented in Figure 1 and Table 1, respectively. The femoral head segmentations in the re-trained RaySearch model had increased overlap with our clinical data, with a DSC (mean±1 STD) for the right femoral head of 0.55±0.06 (n=53) increasing to 0.91±0.02 (n=10) and mean Hausdorff (mm) decreasing from 55±7 (n=53) to 4±1 (n=10) (similar results for the left femoral head). The deviation in femoral head compared to the RaySearch and MVision original models occurred due to a difference in the femoral head segmentation guideline in the clinic specific data, see Figure 2. Time recording of manual delineation was 13 minutes compared to 0.5 minutes (RaySearch) and 1.4 minutes (MVision) for the AI models, manual correction not included.<br/><br/>Conclusion<br/><br/>Both AI models demonstrate good segmentation performance for bladder and rectum. Clinic specific training data (or data that complies to the clinic specific delineation guideline) might be necessary to achieve segmentation results in accordance to the clinical specific standard for some anatomical structures, such as the femoral heads in our case. The time saving was around 90%, not including manual correction.}},
  author       = {{Gorgisyan, Jenny and Bengtsson, Ida and Lempart, Michael and Lerner, Minna and Wieslander, Elinore and Alkner, Sara and Jamtheim Gustafsson, Christian}},
  language     = {{eng}},
  title        = {{Evalutation of two commercial deep learning OAR segmentation models for prostate cancer treatment}},
  url          = {{https://www.estro.org/Congresses/ESTRO-2022/661/radiomics-modellingandstatisticalmethods/11383/evaluationoftwocommercialdeeplearningoarsegmentati}},
  year         = {{2022}},
}