Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Artificial intelligence-supported segmentation of cardiac anatomy in open-heart surgery videos

Stenmark, Maj LU orcid ; Önerud, Julia ; Anvariazar, Shahriar and Tran, Phan Kiet LU (2026) In Journal of Robotic Surgery 20(1).
Abstract

Accurate identification of anatomical structures is essential for safe congenital cardiac surgery and for the development of assistive robotic systems. However, scalable annotation of open-heart surgical video remains a major challenge due to dynamic tissue motion, occlusion, and anatomical variability. The aim of this study was to develop and evaluate a human-in-the-loop segmentation and tracking pipeline for congenital open-heart surgery videos. A dataset of 72 annotated video clips comprising 27,461 frames was created from routine recordings of congenital cardiac surgery. Independent tracking evaluation was performed on 6 video clips (2,400 frames) from three surgical cases that were not used for training or validation. A hybrid... (More)

Accurate identification of anatomical structures is essential for safe congenital cardiac surgery and for the development of assistive robotic systems. However, scalable annotation of open-heart surgical video remains a major challenge due to dynamic tissue motion, occlusion, and anatomical variability. The aim of this study was to develop and evaluate a human-in-the-loop segmentation and tracking pipeline for congenital open-heart surgery videos. A dataset of 72 annotated video clips comprising 27,461 frames was created from routine recordings of congenital cardiac surgery. Independent tracking evaluation was performed on 6 video clips (2,400 frames) from three surgical cases that were not used for training or validation. A hybrid framework combining fine-tuned Segment Anything Model 2 (SAM 2) for propagation-based tracking and YOLOv11 for detection-based monitoring was implemented and evaluated. Fine-tuning improved temporal tracking performance compared with the pretrained model, increasing the mean IoU from 82.1% to 92.2% and the proportion of frames with IoU ≥ 90% from 48.2% to 72.3% on the independent evaluation dataset. A graphical user interface enabled efficient dataset creation, reducing annotation time by approximately 40-fold compared with manual tracing. This study demonstrates that domain-adapted foundation-model tracking can provide robust anatomical tracking in a dynamic open surgical environment and offers a scalable framework for future assistive and robotic cardiac applications.

(Less)
Please use this url to cite or link to this publication:
author
; ; and
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
Anatomical segmentation, Artificial intelligence, Open-heart surgery, SAM 2, Surgical decision-making, YOLO
in
Journal of Robotic Surgery
volume
20
issue
1
article number
801
publisher
Springer Nature
external identifiers
  • pmid:42557484
  • scopus:105046539431
ISSN
1863-2483
DOI
10.1007/s11701-026-03775-x
language
English
LU publication?
yes
id
a860891a-0246-4a3d-9ec8-8cbb3219af67
date added to LUP
2026-09-28 16:05:44
date last changed
2026-10-06 14:57:56
@article{a860891a-0246-4a3d-9ec8-8cbb3219af67,
  abstract     = {{<p>Accurate identification of anatomical structures is essential for safe congenital cardiac surgery and for the development of assistive robotic systems. However, scalable annotation of open-heart surgical video remains a major challenge due to dynamic tissue motion, occlusion, and anatomical variability. The aim of this study was to develop and evaluate a human-in-the-loop segmentation and tracking pipeline for congenital open-heart surgery videos. A dataset of 72 annotated video clips comprising 27,461 frames was created from routine recordings of congenital cardiac surgery. Independent tracking evaluation was performed on 6 video clips (2,400 frames) from three surgical cases that were not used for training or validation. A hybrid framework combining fine-tuned Segment Anything Model 2 (SAM 2) for propagation-based tracking and YOLOv11 for detection-based monitoring was implemented and evaluated. Fine-tuning improved temporal tracking performance compared with the pretrained model, increasing the mean IoU from 82.1% to 92.2% and the proportion of frames with IoU ≥ 90% from 48.2% to 72.3% on the independent evaluation dataset. A graphical user interface enabled efficient dataset creation, reducing annotation time by approximately 40-fold compared with manual tracing. This study demonstrates that domain-adapted foundation-model tracking can provide robust anatomical tracking in a dynamic open surgical environment and offers a scalable framework for future assistive and robotic cardiac applications.</p>}},
  author       = {{Stenmark, Maj and Önerud, Julia and Anvariazar, Shahriar and Tran, Phan Kiet}},
  issn         = {{1863-2483}},
  keywords     = {{Anatomical segmentation; Artificial intelligence; Open-heart surgery; SAM 2; Surgical decision-making; YOLO}},
  language     = {{eng}},
  number       = {{1}},
  publisher    = {{Springer Nature}},
  series       = {{Journal of Robotic Surgery}},
  title        = {{Artificial intelligence-supported segmentation of cardiac anatomy in open-heart surgery videos}},
  url          = {{http://dx.doi.org/10.1007/s11701-026-03775-x}},
  doi          = {{10.1007/s11701-026-03775-x}},
  volume       = {{20}},
  year         = {{2026}},
}