Advanced

Relevance in the eye of the beholder

Lie, Christian LU (2019) MMKM10 20182
Product Development
Abstract
This thesis examines a method for how humans can assess quality of classifications by image based neural network. Through examples visualisations of layerwise relevance propagation (Bach et al. 2015) test participants are tasked with learning to differentiate correctly and incorrectly classified images. To test their learning, they are tasked with evaluating visualisations based on whether they believe the underlying classification to be correct or not. The test was conducted for 20 test participants who after 10 learning examples each were tasked with evaluating the same 72 visualisations. Three different types of visualisations are developed according to different mathematical principles.
The results show that the test was often... (More)
This thesis examines a method for how humans can assess quality of classifications by image based neural network. Through examples visualisations of layerwise relevance propagation (Bach et al. 2015) test participants are tasked with learning to differentiate correctly and incorrectly classified images. To test their learning, they are tasked with evaluating visualisations based on whether they believe the underlying classification to be correct or not. The test was conducted for 20 test participants who after 10 learning examples each were tasked with evaluating the same 72 visualisations. Three different types of visualisations are developed according to different mathematical principles.
The results show that the test was often confusing for the test participants indicating both that better preparation and training of test participants might be appropriate but also that the subject is inaccessible to many. There are also small beneficial effects which are expected to increase with better training and preparation. (Less)
Please use this url to cite or link to this publication:
author
Lie, Christian LU
supervisor
organization
alternative title
Diagnosing classifications based on visualised layerwise relevance propagation.
course
MMKM10 20182
year
type
H2 - Master's Degree (Two Years)
subject
keywords
Neural network, Machine Learning, Visualisation, User testing, Layerwise relevance propagation, LRP
language
English
id
8969821
date added to LUP
2019-02-11 13:08:56
date last changed
2019-02-11 13:08:56
@misc{8969821,
  abstract     = {This thesis examines a method for how humans can assess quality of classifications by image based neural network. Through examples visualisations of layerwise relevance propagation (Bach et al. 2015) test participants are tasked with learning to differentiate correctly and incorrectly classified images. To test their learning, they are tasked with evaluating visualisations based on whether they believe the underlying classification to be correct or not. The test was conducted for 20 test participants who after 10 learning examples each were tasked with evaluating the same 72 visualisations. Three different types of visualisations are developed according to different mathematical principles. 
The results show that the test was often confusing for the test participants indicating both that better preparation and training of test participants might be appropriate but also that the subject is inaccessible to many. There are also small beneficial effects which are expected to increase with better training and preparation.},
  author       = {Lie, Christian},
  keyword      = {Neural network,Machine Learning,Visualisation,User testing,Layerwise relevance propagation,LRP},
  language     = {eng},
  note         = {Student Paper},
  title        = {Relevance in the eye of the beholder},
  year         = {2019},
}