Subjective Image Quality Evaluation Using the Softcopy Quality Ruler Method
(2014) In Master's Theses in Mathematical Sciences FMA820 20132Mathematics (Faculty of Engineering)
- Abstract
- Image quality is in essence based on subjective experience, and the involvement of human subjects is therefore necessary in one way or the other in order to make reliable assessments of it. This topic is of interest for Axis Communications AB -- the market leader in surveillance cameras.
In this thesis, the softcopy quality ruler method described in ISO 20462 is evaluated through a study of one specific image quality problem; human preference in the noise reduction -- texture loss space. The method involves quality or attribute judgment by comparison of test images to a series of ordered, univariate (variation in one attribute only) reference images, called ''ruler images''. Sharpness is used as reference attribute for the ruler images.... (More) - Image quality is in essence based on subjective experience, and the involvement of human subjects is therefore necessary in one way or the other in order to make reliable assessments of it. This topic is of interest for Axis Communications AB -- the market leader in surveillance cameras.
In this thesis, the softcopy quality ruler method described in ISO 20462 is evaluated through a study of one specific image quality problem; human preference in the noise reduction -- texture loss space. The method involves quality or attribute judgment by comparison of test images to a series of ordered, univariate (variation in one attribute only) reference images, called ''ruler images''. Sharpness is used as reference attribute for the ruler images.
For the purpose of the thesis project, a working environment for performing subjective studies was set up. A total of 47 persons (observers) were invited to judge a set of test images varying in noise level and amount of noise reduction as well as scene content.
From the data, confidence intervals were calculated using the non-parametric bootstrap method as well as using the normal distribution. The two methods gave very similar results, and thus it could be confirmed that the data is well described by a normal distribution.
The results of the study indicate that it is possible to determine an optimum level of noise reduction for a given noise level.
For increasing noise levels, the variances of the observer judgments increase, while they decrease for increasing noise reduction levels. This difference may be explained by making the observation that increasing noise reduction levels result in increasing texture blur, which appears more similar to sharpness loss in comparison with noise. It is reasonable to assume that the uncertainty in observer judgments should be lower when comparing images with similar degradations. Therefore, it might be argued that the variability of the judgments depends to a large extent on the perceived similarities between the ruler and test images, in terms of the attribute(s) considered. If the ruler images appear similar to the test images, the variances of the judgments will be lower than for less similar ruler images and test images, and also more strongly dependent on the number of observers. In order to reduce the number of observers when testing some particular image quality attribute(s), care should be taken to use ruler images varying in an attribute similar in appearance to the test images.
Also noticeable are discrepancies between experienced (having experience in judging or evaluating images) and unexperienced observers, as concluded by a Welch t-test. When judging image quality, experienced observers at Axis, such as imaging engineers, camera designers etc., seem more tolerant to noise than unexperienced observers. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/4498793
- author
- Persson, Maria LU
- supervisor
- organization
- course
- FMA820 20132
- year
- 2014
- type
- H2 - Master's Degree (Two Years)
- subject
- publication/series
- Master's Theses in Mathematical Sciences
- report number
- LUTFMA-3263-2014
- ISSN
- 1404-6342
- other publication id
- 2014:E36
- language
- English
- id
- 4498793
- date added to LUP
- 2014-12-15 13:22:00
- date last changed
- 2014-12-15 13:22:00
@misc{4498793,
abstract = {{Image quality is in essence based on subjective experience, and the involvement of human subjects is therefore necessary in one way or the other in order to make reliable assessments of it. This topic is of interest for Axis Communications AB -- the market leader in surveillance cameras.
In this thesis, the softcopy quality ruler method described in ISO 20462 is evaluated through a study of one specific image quality problem; human preference in the noise reduction -- texture loss space. The method involves quality or attribute judgment by comparison of test images to a series of ordered, univariate (variation in one attribute only) reference images, called ''ruler images''. Sharpness is used as reference attribute for the ruler images.
For the purpose of the thesis project, a working environment for performing subjective studies was set up. A total of 47 persons (observers) were invited to judge a set of test images varying in noise level and amount of noise reduction as well as scene content.
From the data, confidence intervals were calculated using the non-parametric bootstrap method as well as using the normal distribution. The two methods gave very similar results, and thus it could be confirmed that the data is well described by a normal distribution.
The results of the study indicate that it is possible to determine an optimum level of noise reduction for a given noise level.
For increasing noise levels, the variances of the observer judgments increase, while they decrease for increasing noise reduction levels. This difference may be explained by making the observation that increasing noise reduction levels result in increasing texture blur, which appears more similar to sharpness loss in comparison with noise. It is reasonable to assume that the uncertainty in observer judgments should be lower when comparing images with similar degradations. Therefore, it might be argued that the variability of the judgments depends to a large extent on the perceived similarities between the ruler and test images, in terms of the attribute(s) considered. If the ruler images appear similar to the test images, the variances of the judgments will be lower than for less similar ruler images and test images, and also more strongly dependent on the number of observers. In order to reduce the number of observers when testing some particular image quality attribute(s), care should be taken to use ruler images varying in an attribute similar in appearance to the test images.
Also noticeable are discrepancies between experienced (having experience in judging or evaluating images) and unexperienced observers, as concluded by a Welch t-test. When judging image quality, experienced observers at Axis, such as imaging engineers, camera designers etc., seem more tolerant to noise than unexperienced observers.}},
author = {{Persson, Maria}},
issn = {{1404-6342}},
language = {{eng}},
note = {{Student Paper}},
series = {{Master's Theses in Mathematical Sciences}},
title = {{Subjective Image Quality Evaluation Using the Softcopy Quality Ruler Method}},
year = {{2014}},
}