Virtual H&E Staining Using PLS Microscopy and Neural Networks
(2024) In Master’s Theses in Mathematical Sciences FMAM05 20232Mathematics (Faculty of Engineering)
- Abstract
- Histopathological examination, crucial in diagnosing diseases such as cancer, traditionally relies on time- and resource-consuming, poorly standardized chemical staining for tissue visualization. This thesis presents a novel digital alternative using generative neural networks and a point light source (PLS) microscope to transform unstained skin tissue images into their stained counterparts. This approach utilizes PLS-microscopy’s unique illumination angles, providing more structural information about a sample and thereby enhancing a neural network’s ability to produce accurate, virtually stained images.
Two matched datasets, each containing paired unstained and chemically stained tissue images, were used for supervised training of... (More) - Histopathological examination, crucial in diagnosing diseases such as cancer, traditionally relies on time- and resource-consuming, poorly standardized chemical staining for tissue visualization. This thesis presents a novel digital alternative using generative neural networks and a point light source (PLS) microscope to transform unstained skin tissue images into their stained counterparts. This approach utilizes PLS-microscopy’s unique illumination angles, providing more structural information about a sample and thereby enhancing a neural network’s ability to produce accurate, virtually stained images.
Two matched datasets, each containing paired unstained and chemically stained tissue images, were used for supervised training of several networks. One dataset comprised healthy tissue, while the other, in addition to healthy tissue, included basal and squa- mous cell carcinomas. Given the limited scope of this master’s thesis, which constrained data acquisition, these datasets were relatively small, potentially impacting the general- izability of the model. The project explored the virtual staining capabilities of UNet and DenseUNet architectures, focusing on network depth and input channels. Variations in activation functions, upsampling blocks, and attention gates were tested, alongside the development of Relativistic Generative Adversarial Network (RGAN) models.
Quantitative evaluation using standard metrics and qualitative assessment by pathologists and other medical professionals demonstrated the potential of PLS microscopy in virtual staining. The final model, based on RGAN, achieved superior staining accuracy with a structural similarity (SSIM) score of 0.799, significantly outperforming traditional bright field imaging (SSIM 0.631). However, the limited diversity and size of the datasets may have inflated these scores and highlight the need for caution in interpreting the results. The pathologists and medical professionals found virtually stained images indistinguishable from their chemically stained counterparts, with average stain quality ratings of 6.40 out of 10 for virtual images, which did not differ significantly from the rating of 6.41 for chemically stained ones. The pathologists and medical professionals were also able to classify 95.83% of all images as healthy or containing cancerous tissue correctly.
In conclusion, virtual staining using PLS-microscopy holds considerable promise, offering a more standardized and sustainable approach compared to chemical staining. This method has the potential to speed up diagnosis and facilitate further analysis using image analysis algorithms. Future research could expand this technique beyond skin tissues, enhancing its applicability across a broader range of histopathological examinations. (Less) - Popular Abstract
- In 2023, only 38% of all Swedes diagnosed with cancer began their treatment in time. One reason for this is the time consuming and poorly standardized chemical staining processes. Staining is needed to visualize structures of tissue under a microscope, in order to arrive at a diagnosis. By replacing the chemical staining process with neural networks, a more efficient and sustainable workflow which produces synthetically stained images, referred to as virtual staining, can be achieved. Our thesis presents a novel virtual staining approach using images taken with a PLS microscope as input to the neural networks to try to answer the questions: Can virtual staining performance be improved using PLS images? Do the virtually stained images... (More)
- In 2023, only 38% of all Swedes diagnosed with cancer began their treatment in time. One reason for this is the time consuming and poorly standardized chemical staining processes. Staining is needed to visualize structures of tissue under a microscope, in order to arrive at a diagnosis. By replacing the chemical staining process with neural networks, a more efficient and sustainable workflow which produces synthetically stained images, referred to as virtual staining, can be achieved. Our thesis presents a novel virtual staining approach using images taken with a PLS microscope as input to the neural networks to try to answer the questions: Can virtual staining performance be improved using PLS images? Do the virtually stained images retain pathologically relevant information? (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9149003
- author
- Vizins, Sally LU and Råhnängen, Hanna LU
- supervisor
- organization
- course
- FMAM05 20232
- year
- 2024
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- Deep learning, Virtual staining, Skin tissue, Hematoxylin & Eosin, H&E, Pathology, Carcinoma, Point light source illumination, Neural Networks, GANs, Generative adversarial networks, CNNs, Convolutional neural networks, Relativistic generative adversarial network, Unet, Digital microscopy, Attention-Unet, Dense-Unet
- publication/series
- Master’s Theses in Mathematical Sciences
- report number
- LUTFMA-3522-2024
- ISSN
- 1404-6342
- other publication id
- 2024:E1
- language
- English
- id
- 9149003
- date added to LUP
- 2024-03-12 11:30:36
- date last changed
- 2024-03-13 09:11:19
@misc{9149003,
abstract = {{Histopathological examination, crucial in diagnosing diseases such as cancer, traditionally relies on time- and resource-consuming, poorly standardized chemical staining for tissue visualization. This thesis presents a novel digital alternative using generative neural networks and a point light source (PLS) microscope to transform unstained skin tissue images into their stained counterparts. This approach utilizes PLS-microscopy’s unique illumination angles, providing more structural information about a sample and thereby enhancing a neural network’s ability to produce accurate, virtually stained images.
Two matched datasets, each containing paired unstained and chemically stained tissue images, were used for supervised training of several networks. One dataset comprised healthy tissue, while the other, in addition to healthy tissue, included basal and squa- mous cell carcinomas. Given the limited scope of this master’s thesis, which constrained data acquisition, these datasets were relatively small, potentially impacting the general- izability of the model. The project explored the virtual staining capabilities of UNet and DenseUNet architectures, focusing on network depth and input channels. Variations in activation functions, upsampling blocks, and attention gates were tested, alongside the development of Relativistic Generative Adversarial Network (RGAN) models.
Quantitative evaluation using standard metrics and qualitative assessment by pathologists and other medical professionals demonstrated the potential of PLS microscopy in virtual staining. The final model, based on RGAN, achieved superior staining accuracy with a structural similarity (SSIM) score of 0.799, significantly outperforming traditional bright field imaging (SSIM 0.631). However, the limited diversity and size of the datasets may have inflated these scores and highlight the need for caution in interpreting the results. The pathologists and medical professionals found virtually stained images indistinguishable from their chemically stained counterparts, with average stain quality ratings of 6.40 out of 10 for virtual images, which did not differ significantly from the rating of 6.41 for chemically stained ones. The pathologists and medical professionals were also able to classify 95.83% of all images as healthy or containing cancerous tissue correctly.
In conclusion, virtual staining using PLS-microscopy holds considerable promise, offering a more standardized and sustainable approach compared to chemical staining. This method has the potential to speed up diagnosis and facilitate further analysis using image analysis algorithms. Future research could expand this technique beyond skin tissues, enhancing its applicability across a broader range of histopathological examinations.}},
author = {{Vizins, Sally and Råhnängen, Hanna}},
issn = {{1404-6342}},
language = {{eng}},
note = {{Student Paper}},
series = {{Master’s Theses in Mathematical Sciences}},
title = {{Virtual H&E Staining Using PLS Microscopy and Neural Networks}},
year = {{2024}},
}