Machine Learning Based Video Editing Toolbox for Automatic Summary of Medical Videos
(2019) In Master's Theses in Mathematical Sciences FMAM05 20191Mathematics (Faculty of Engineering)
- Abstract
- Recording of medical procedures makes it possible for medical staff to review
their work, learn from the mistakes and produce material for education. Medical videos tend to be long, which has an impact on usability and raises issues concerning memory storage.
The aim of this master thesis is to make the material more user-friendly with
an intelligent toolbox based on machine learning and image analysis techniques. The tools divide the video into chapters with a k-means++ clustering technique, detect when an X-ray source is active with ROI based processing, track camera movements by comparing frames with the optical flow algorithm by Gunnar Farnebäck and identify when medical instruments are present using an artificial neural network.... (More) - Recording of medical procedures makes it possible for medical staff to review
their work, learn from the mistakes and produce material for education. Medical videos tend to be long, which has an impact on usability and raises issues concerning memory storage.
The aim of this master thesis is to make the material more user-friendly with
an intelligent toolbox based on machine learning and image analysis techniques. The tools divide the video into chapters with a k-means++ clustering technique, detect when an X-ray source is active with ROI based processing, track camera movements by comparing frames with the optical flow algorithm by Gunnar Farnebäck and identify when medical instruments are present using an artificial neural network. Based on the information from these tools a combined timeline
and an automatic summary is created.
The results indicate that the chapter tool is especially promising when the
videos include pre and post procedure sections, since these are easier to separate. The ROI based tool detects all the frames with an active X-ray. The
neural network performs well on classifying frames containing an instrument,
but requires annotated data. The majority of camera movements are found,
but the algorithm sometimes fails to detect zoom in the video.
This thesis is intended as a proof of concept of the potential in automatic processing of medical videos. The tools can create reference points to important sequences. More data and evaluation of the tools are necessary for the further development of an automatic summary system. (Less) - Popular Abstract (Swedish)
- Inspelningar av medicinska operationer möjliggör utvärdering av kirurgers arbete och kan användas som utbildningsmaterial. Medicinska videor tenderar dock att vara långa och kräver manuell redigering för att förbättra användbarheten. Idag finns det företag som erbjuder att med sin medicinska expertis sammanfatta operationsvideo manuellt.
Målet med det här examensarbetet är att skapa en uppsättning av intelligenta verktyg som kan användas för att göra videomaterialet mer lätthanterligt genom att automatiskt sammanfatta det. Verktygen, som bygger på maskininlärningstekniker och bildanalys, kan automatiskt dela in en medicinsk video i kapitel, identifiera när röntgen används under ingreppet, hitta medicinska instrument samt upptäcka när... (More) - Inspelningar av medicinska operationer möjliggör utvärdering av kirurgers arbete och kan användas som utbildningsmaterial. Medicinska videor tenderar dock att vara långa och kräver manuell redigering för att förbättra användbarheten. Idag finns det företag som erbjuder att med sin medicinska expertis sammanfatta operationsvideo manuellt.
Målet med det här examensarbetet är att skapa en uppsättning av intelligenta verktyg som kan användas för att göra videomaterialet mer lätthanterligt genom att automatiskt sammanfatta det. Verktygen, som bygger på maskininlärningstekniker och bildanalys, kan automatiskt dela in en medicinsk video i kapitel, identifiera när röntgen används under ingreppet, hitta medicinska instrument samt upptäcka när kameran rör på sig i inspelningen. Baserat på denna information skapas en sammanfattning automatiskt.
Kapitelindelningen och identifieringen av medicinska instrument gav de mest lovande resultaten. En fördel med kapitelindelningen är att den bygger på så kallad “unsupervised machine learning”, vilket innebär att den inte behöver någon manuellt bearbetad data. Verktyget kan därför göras generellt för många typer av operationer. Detektion av vissa instrument kan indikera början och slutet på kritiska faser under en operation, men verktyget behöver skräddarsys beroende på vilka instrument man är intresserad av att hitta.
Examensarbetet är ett proof of concept för att visa potentialen i att automatisera bearbetningen av medicinsk video. För att vidareutveckla projektet behövs det mer data och djupare utvärdering av verktygen. En djupare utvärdering skulle exempelvis kunna vara att sätta samman en panel som diskuterar vad som är relevant att se i en sammanfattning. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/8977838
- author
- Oswald, Olle LU and Nilén, Emil LU
- supervisor
- organization
- course
- FMAM05 20191
- year
- 2019
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- machine learning, image analysis, video summary, medical video, artificial neural networks
- publication/series
- Master's Theses in Mathematical Sciences
- report number
- LUFTMA-3383-2019
- ISSN
- 1404-6342
- other publication id
- 2019:E25
- language
- English
- id
- 8977838
- date added to LUP
- 2019-07-12 11:35:01
- date last changed
- 2019-07-12 11:35:01
@misc{8977838, abstract = {{Recording of medical procedures makes it possible for medical staff to review their work, learn from the mistakes and produce material for education. Medical videos tend to be long, which has an impact on usability and raises issues concerning memory storage. The aim of this master thesis is to make the material more user-friendly with an intelligent toolbox based on machine learning and image analysis techniques. The tools divide the video into chapters with a k-means++ clustering technique, detect when an X-ray source is active with ROI based processing, track camera movements by comparing frames with the optical flow algorithm by Gunnar Farnebäck and identify when medical instruments are present using an artificial neural network. Based on the information from these tools a combined timeline and an automatic summary is created. The results indicate that the chapter tool is especially promising when the videos include pre and post procedure sections, since these are easier to separate. The ROI based tool detects all the frames with an active X-ray. The neural network performs well on classifying frames containing an instrument, but requires annotated data. The majority of camera movements are found, but the algorithm sometimes fails to detect zoom in the video. This thesis is intended as a proof of concept of the potential in automatic processing of medical videos. The tools can create reference points to important sequences. More data and evaluation of the tools are necessary for the further development of an automatic summary system.}}, author = {{Oswald, Olle and Nilén, Emil}}, issn = {{1404-6342}}, language = {{eng}}, note = {{Student Paper}}, series = {{Master's Theses in Mathematical Sciences}}, title = {{Machine Learning Based Video Editing Toolbox for Automatic Summary of Medical Videos}}, year = {{2019}}, }