Segmentation and Prediction of Mutation Status of Malignant Melanoma Whole-slide Images using Deep Learning

Abstract

Malignant melanoma is an aggressive type of skin cancer. Gene mutations can make the disease progress faster, but specialised treatment exists. Today, gene mutations are detected with DNA-analysis which is costly and time-consuming. The aim of our thesis is to investigate whether deep learning can be used to differentiate whole-slide images of tumours with different gene mutations. This was done in two steps, first whole-slide images were segmented based on tissue types, and then classification of gene mutations was done.

The tissue segmentation was done using the deep convolutional network Inception v3, modified to a four class output. Image tiles of the size 244 x 244 pixels were used to train and evaluate the network, with F1-score 0.84 on tumour tissue.

Two different methods to predict mutation status were tested. First, image features extracted from the segmentation network were fed into binary classifiers to separate images of tumours with and without NRAS mutation. Due to unsatisfactory results, another method was tested. A new Inception v3 network was trained to distinguish between NRAS and BRAF mutated tumours. Data from the public database The Cancer Genome Atlas was used for training and evaluation. Further testing was done on two independent test sets. Only tiles with 90% or higher probability of being tumour according to the segmentation network were used. The classification network was tested tilewise (AUC 0.53-0.66) and patientwise with AUC-values around 0.60 for all datasets.

The results indicate that it is possible to separate tissue images based on gene mutations. We believe that deep learning networks like these have great potential of being integrated into diagnostics of malignant melanoma. This could lead to faster and more accessible gene mutation diagnostics around the world.

Abstract (Swedish)

Malignt melanom är en aggressiv form av hudcancer. Genmutationer kan påskynda sjukdomsförloppet och spridningen av tumörer, men specialanpassad behandling finns att tillgå. Idag används DNA-analys för att upptäcka genmutationer, vilket är kostsamt och tidskrävande. Syftet med vårt examensarbete är att undersöka om djupinlärning (deep learning) kan användas för att hitta genmutationer från vävnadsbilder på malignt melanom. Detta har vi gjort i två steg, först genom att hitta tumörrik vävnad i mikroskopbilder, och sedan utföra klassificering av mutationer på dessa regioner.

Segmentering av olika vävnadstyper gjordes med hjälp av det djupa neurala nätverket Inception v3. Bildurklipp av storleken 244 x 244 pixlar användes för att träna och testa nätverket med F1-score 0,84 på tumörvävnad.

För att utföra klassificering av genmutationer testades två metoder. Först testade vi på att skilja på vävnadsbilder med och utan NRAS-mutationer med hjälp av s.k. features, numeriska värden som hämtats ut från segmenteringsnätverket. Försöket gav inte tillfredsställande resultat och därför tränades istället ett nytt Inception v3-nätverk till att göra klassificering av tumörbilder med NRAS- och BRAF-mutationer. Nätverket tränades på bilder från databasen The Cancer Genome Atlas och testades på ytterligare två separata dataset. Endast urklipp med mer än 90% sannolikhet att vara tumörvävnad enligt segmenteringsnätverket användes. Klassificeringen testades både urklippsvis (AUC 0,53-0,66) och patientvis med AUC-värden runt 0,60 för samtliga dataset.

Resultaten visar på att det är möjligt att skilja på bilder på tumörvävnad med olika genmutationer. Vi tror att liknande djupa neurala nätverk har stor potential att integreras i diagnostiken av malignt melanom. Det skulle kunna innebära snabbare och mer tillgänglig diagnostik av genmutationer.

Popular Abstract

Deep Learning - the key to revolutionise specialised skin cancer treatment?

Malignant melanoma is an aggressive type of skin cancer that develops from moles. Gene mutations can make the disease progress faster, but if the mutations are detected, it is possible to specialise the treatment. Using deep learning as a complement in diagnostics is state of the art in many medical fields. It is a type of artificial intelligence that can detect patterns that are invisible for the human eye. In our thesis we have shown that it is possible to use deep learning to predict the mutation status of melanoma using microscopy images. With further development, this method could possible replace advanced, expensive and time consuming lab analyses. The technique could contribute to more rapid and accessible diagnostics around the world.

Malignant melanoma is increasing at a high pace all over the world. With the exception of lung cancer in women, it is the cancer type that is increasing the most in prevalence. Specialised treatment is an important step of defeating cancer. Gene mutations in malignant melanoma enhance tumour growth which makes the disease progress faster. The two most common mutations are present in 40% and 20% of the cases, respectively. Since specialised treatment exists, detection of these mutations is crucial. Today, this is done with costly and time-consuming DNA analysis. However, recent studies show that deep learning can be used to detect the mutation status from tissue images alone. For a better chance at saving a patient's life, early detection and comprehensive patient investigation play vital roles. It is common to visually inspect cancer tissue in a microscope to mark out the tumour areas. However, this is a tedious task performed manually by a specialist.

Deep learning is a subfield of artificial intelligence and it can be used to automatically mark the different tissue types, without the need for human participation. In our thesis, we have trained a deep learning network that can identify four tissue types in melanoma biopsies which can assist in the segmentation procedure and save a great amount of time for the specialist. The segmentation network was trained and evaluated with a dataset from Skåne University hospital in Lund and its performance was visually evaluated on an independent dataset from the public database The Cancer Genome Atlas.

The segmentation network was used to find tumour-rich areas in the tissue and another deep learning network was trained to classify the mutation status. Even though further improvement is needed, the deep learning models developed in this thesis show high potential of being an integrated part of an automatic diagnostic tool. This tool would not only increase the speed but also make the melanoma diagnosis more accessible across the world since it only needs microscopy images and a computer.

• author: Johansson, Elin and Månefjord, Fanny
• supervisor: Melinda Rezeli; Ida Arvidsson; Jonatan Eriksson
• organization: Department of Biomedical Engineering
• course: BMEM01 20211
• year: 2021
• type: H2 - Master's Degree (Two Years)
• subject: Technology and Engineering
• keywords: deep learning, image analysis, malignant melanoma, tissue segmentation, mutation classification, Inception v3
• language: English
• additional info: 2021-05
• id: 9050824
• date added to LUP: 2021-06-14 14:09:08
• date last changed: 2021-06-14 14:09:08