Test automation with grad-CAM Heatmaps - A future pipe segment in MLOps for Vision AI?
(2021) 14th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2021 p.175-181- Abstract
Machine Learning (ML) is a fundamental part of modern perception systems. In the last decade, the performance of computer vision using trained deep neural networks has outperformed previous approaches based on careful feature engineering. However, the opaqueness of large ML models is a substantial impediment for critical applications such as in the automotive context. As a remedy, Gradient-weighted Class Activation Mapping (Grad-CAM) has been proposed to provide visual explanations of model internals. In this paper, we demonstrate how Grad-CAM heatmaps can be used to increase the explainability of an image recognition model trained for a pedestrian underpass. We argue how the heatmaps support compliance to the EU's seven key... (More)
Machine Learning (ML) is a fundamental part of modern perception systems. In the last decade, the performance of computer vision using trained deep neural networks has outperformed previous approaches based on careful feature engineering. However, the opaqueness of large ML models is a substantial impediment for critical applications such as in the automotive context. As a remedy, Gradient-weighted Class Activation Mapping (Grad-CAM) has been proposed to provide visual explanations of model internals. In this paper, we demonstrate how Grad-CAM heatmaps can be used to increase the explainability of an image recognition model trained for a pedestrian underpass. We argue how the heatmaps support compliance to the EU's seven key requirements for Trustworthy AI. Finally, we propose adding automated heatmap analysis as a pipe segment in an MLOps pipeline. We believe that such a building block can be used to automatically detect if a trained ML-model is activated based on invalid pixels in test images, suggesting biased models.
(Less)
- author
- Borg, Markus LU ; Jabangwe, Ronald ; Åberg, Simon ; Ekblom, Arvid ; Hedlund, Ludwig and Lidfeldt, August
- organization
- publishing date
- 2021
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- keywords
- Grad-CAM, Image recognition, Machine learning testing, Neural networks, Test automation
- host publication
- Proceedings - 2021 IEEE 14th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2021
- article number
- 9440142
- pages
- 7 pages
- publisher
- IEEE - Institute of Electrical and Electronics Engineers Inc.
- conference name
- 14th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2021
- conference location
- Virtual, Porto de Galinhas, Brazil
- conference dates
- 2021-04-12 - 2021-04-16
- external identifiers
-
- scopus:85108025865
- ISBN
- 9781665444569
- DOI
- 10.1109/ICSTW52544.2021.00039
- language
- English
- LU publication?
- yes
- id
- ac8a6636-2565-4842-8c56-142c5ab4dac4
- date added to LUP
- 2021-07-16 11:48:13
- date last changed
- 2022-04-27 02:52:58
@inproceedings{ac8a6636-2565-4842-8c56-142c5ab4dac4,
abstract = {{<p>Machine Learning (ML) is a fundamental part of modern perception systems. In the last decade, the performance of computer vision using trained deep neural networks has outperformed previous approaches based on careful feature engineering. However, the opaqueness of large ML models is a substantial impediment for critical applications such as in the automotive context. As a remedy, Gradient-weighted Class Activation Mapping (Grad-CAM) has been proposed to provide visual explanations of model internals. In this paper, we demonstrate how Grad-CAM heatmaps can be used to increase the explainability of an image recognition model trained for a pedestrian underpass. We argue how the heatmaps support compliance to the EU's seven key requirements for Trustworthy AI. Finally, we propose adding automated heatmap analysis as a pipe segment in an MLOps pipeline. We believe that such a building block can be used to automatically detect if a trained ML-model is activated based on invalid pixels in test images, suggesting biased models. </p>}},
author = {{Borg, Markus and Jabangwe, Ronald and Åberg, Simon and Ekblom, Arvid and Hedlund, Ludwig and Lidfeldt, August}},
booktitle = {{Proceedings - 2021 IEEE 14th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2021}},
isbn = {{9781665444569}},
keywords = {{Grad-CAM; Image recognition; Machine learning testing; Neural networks; Test automation}},
language = {{eng}},
pages = {{175--181}},
publisher = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
title = {{Test automation with grad-CAM Heatmaps - A future pipe segment in MLOps for Vision AI?}},
url = {{http://dx.doi.org/10.1109/ICSTW52544.2021.00039}},
doi = {{10.1109/ICSTW52544.2021.00039}},
year = {{2021}},
}