Skip to main content

LUP Student Papers

LUND UNIVERSITY LIBRARIES

Machine Learning Based Video Editing Toolbox for Automatic Summary of Medical Videos

Oswald, Olle LU and Nilén, Emil LU (2019) In Master's Theses in Mathematical Sciences FMAM05 20191
Mathematics (Faculty of Engineering)
Abstract
Recording of medical procedures makes it possible for medical staff to review
their work, learn from the mistakes and produce material for education. Medical videos tend to be long, which has an impact on usability and raises issues concerning memory storage.

The aim of this master thesis is to make the material more user-friendly with
an intelligent toolbox based on machine learning and image analysis techniques. The tools divide the video into chapters with a k-means++ clustering technique, detect when an X-ray source is active with ROI based processing, track camera movements by comparing frames with the optical flow algorithm by Gunnar Farnebäck and identify when medical instruments are present using an artificial neural network.... (More)
Recording of medical procedures makes it possible for medical staff to review
their work, learn from the mistakes and produce material for education. Medical videos tend to be long, which has an impact on usability and raises issues concerning memory storage.

The aim of this master thesis is to make the material more user-friendly with
an intelligent toolbox based on machine learning and image analysis techniques. The tools divide the video into chapters with a k-means++ clustering technique, detect when an X-ray source is active with ROI based processing, track camera movements by comparing frames with the optical flow algorithm by Gunnar Farnebäck and identify when medical instruments are present using an artificial neural network. Based on the information from these tools a combined timeline
and an automatic summary is created.

The results indicate that the chapter tool is especially promising when the
videos include pre and post procedure sections, since these are easier to separate. The ROI based tool detects all the frames with an active X-ray. The
neural network performs well on classifying frames containing an instrument,
but requires annotated data. The majority of camera movements are found,
but the algorithm sometimes fails to detect zoom in the video.

This thesis is intended as a proof of concept of the potential in automatic processing of medical videos. The tools can create reference points to important sequences. More data and evaluation of the tools are necessary for the further development of an automatic summary system. (Less)
Popular Abstract (Swedish)
Inspelningar av medicinska operationer möjliggör utvärdering av kirurgers arbete och kan användas som utbildningsmaterial. Medicinska videor tenderar dock att vara långa och kräver manuell redigering för att förbättra användbarheten. Idag finns det företag som erbjuder att med sin medicinska expertis sammanfatta operationsvideo manuellt.

Målet med det här examensarbetet är att skapa en uppsättning av intelligenta verktyg som kan användas för att göra videomaterialet mer lätthanterligt genom att automatiskt sammanfatta det. Verktygen, som bygger på maskininlärningstekniker och bildanalys, kan automatiskt dela in en medicinsk video i kapitel, identifiera när röntgen används under ingreppet, hitta medicinska instrument samt upptäcka när... (More)
Inspelningar av medicinska operationer möjliggör utvärdering av kirurgers arbete och kan användas som utbildningsmaterial. Medicinska videor tenderar dock att vara långa och kräver manuell redigering för att förbättra användbarheten. Idag finns det företag som erbjuder att med sin medicinska expertis sammanfatta operationsvideo manuellt.

Målet med det här examensarbetet är att skapa en uppsättning av intelligenta verktyg som kan användas för att göra videomaterialet mer lätthanterligt genom att automatiskt sammanfatta det. Verktygen, som bygger på maskininlärningstekniker och bildanalys, kan automatiskt dela in en medicinsk video i kapitel, identifiera när röntgen används under ingreppet, hitta medicinska instrument samt upptäcka när kameran rör på sig i inspelningen. Baserat på denna information skapas en sammanfattning automatiskt.

Kapitelindelningen och identifieringen av medicinska instrument gav de mest lovande resultaten. En fördel med kapitelindelningen är att den bygger på så kallad “unsupervised machine learning”, vilket innebär att den inte behöver någon manuellt bearbetad data. Verktyget kan därför göras generellt för många typer av operationer. Detektion av vissa instrument kan indikera början och slutet på kritiska faser under en operation, men verktyget behöver skräddarsys beroende på vilka instrument man är intresserad av att hitta.

Examensarbetet är ett proof of concept för att visa potentialen i att automatisera bearbetningen av medicinsk video. För att vidareutveckla projektet behövs det mer data och djupare utvärdering av verktygen. En djupare utvärdering skulle exempelvis kunna vara att sätta samman en panel som diskuterar vad som är relevant att se i en sammanfattning. (Less)
Please use this url to cite or link to this publication:
author
Oswald, Olle LU and Nilén, Emil LU
supervisor
organization
course
FMAM05 20191
year
type
H2 - Master's Degree (Two Years)
subject
keywords
machine learning, image analysis, video summary, medical video, artificial neural networks
publication/series
Master's Theses in Mathematical Sciences
report number
LUFTMA-3383-2019
ISSN
1404-6342
other publication id
2019:E25
language
English
id
8977838
date added to LUP
2019-07-12 11:35:01
date last changed
2019-07-12 11:35:01
@misc{8977838,
  abstract     = {{Recording of medical procedures makes it possible for medical staff to review
their work, learn from the mistakes and produce material for education. Medical videos tend to be long, which has an impact on usability and raises issues concerning memory storage.

The aim of this master thesis is to make the material more user-friendly with
an intelligent toolbox based on machine learning and image analysis techniques. The tools divide the video into chapters with a k-means++ clustering technique, detect when an X-ray source is active with ROI based processing, track camera movements by comparing frames with the optical flow algorithm by Gunnar Farnebäck and identify when medical instruments are present using an artificial neural network. Based on the information from these tools a combined timeline
and an automatic summary is created.

The results indicate that the chapter tool is especially promising when the
videos include pre and post procedure sections, since these are easier to separate. The ROI based tool detects all the frames with an active X-ray. The
neural network performs well on classifying frames containing an instrument,
but requires annotated data. The majority of camera movements are found,
but the algorithm sometimes fails to detect zoom in the video.

This thesis is intended as a proof of concept of the potential in automatic processing of medical videos. The tools can create reference points to important sequences. More data and evaluation of the tools are necessary for the further development of an automatic summary system.}},
  author       = {{Oswald, Olle and Nilén, Emil}},
  issn         = {{1404-6342}},
  language     = {{eng}},
  note         = {{Student Paper}},
  series       = {{Master's Theses in Mathematical Sciences}},
  title        = {{Machine Learning Based Video Editing Toolbox for Automatic Summary of Medical Videos}},
  year         = {{2019}},
}