Skip to main content

LUP Student Papers

LUND UNIVERSITY LIBRARIES

Test Selection Based on Historical Test Data

Dunn Ekelund, Edward LU (2015) In LU-CS-EX 2015-20 EDA920 20151
Department of Computer Science
Abstract
This thesis is the culmination of an action research project conducted at Axis Communications, which tries to enable test selection based on historical test data. In order to realize this, a prototype program -- the Difference Engine -- was created, which can parse and analyze historical test data drawn from a large database. The analysis produces a weighted correlation between code packages and test packages, and this correlation can then be used to select only a recommended subset of the full regression test suite in order to minimize the time spent running tests which are not related to the code that was changed.

The evaluation results of the Difference Engine shows that its algorithm for finding correlations is robust, and running... (More)
This thesis is the culmination of an action research project conducted at Axis Communications, which tries to enable test selection based on historical test data. In order to realize this, a prototype program -- the Difference Engine -- was created, which can parse and analyze historical test data drawn from a large database. The analysis produces a weighted correlation between code packages and test packages, and this correlation can then be used to select only a recommended subset of the full regression test suite in order to minimize the time spent running tests which are not related to the code that was changed.

The evaluation results of the Difference Engine shows that its algorithm for finding correlations is robust, and running only the recommended tests is more effective than running the full regression test suite by a significant factor. On average it successfully identifies nearly all relevant tests (median success rate is 100%), and it discards 14 times more irrelevant tests given the code that has been changed compared to the default selection strategy of running all tests. Furthermore, the average size of the recommended subset is only 5% of the size of the full regression test suite, meaning that the time spent running tests can be decreased by a factor of 20. (Less)
Please use this url to cite or link to this publication:
author
Dunn Ekelund, Edward LU
supervisor
organization
course
EDA920 20151
year
type
H3 - Professional qualifications (4 Years - )
subject
keywords
regression testing, test selection, test prioritization, RTS
publication/series
LU-CS-EX 2015-20
report number
LU-CS-EX 2015-20
ISSN
1650-2884
language
English
id
7358122
date added to LUP
2015-06-16 11:19:15
date last changed
2015-06-18 14:04:28
@misc{7358122,
  abstract     = {{This thesis is the culmination of an action research project conducted at Axis Communications, which tries to enable test selection based on historical test data. In order to realize this, a prototype program -- the Difference Engine -- was created, which can parse and analyze historical test data drawn from a large database. The analysis produces a weighted correlation between code packages and test packages, and this correlation can then be used to select only a recommended subset of the full regression test suite in order to minimize the time spent running tests which are not related to the code that was changed.

The evaluation results of the Difference Engine shows that its algorithm for finding correlations is robust, and running only the recommended tests is more effective than running the full regression test suite by a significant factor. On average it successfully identifies nearly all relevant tests (median success rate is 100%), and it discards 14 times more irrelevant tests given the code that has been changed compared to the default selection strategy of running all tests. Furthermore, the average size of the recommended subset is only 5% of the size of the full regression test suite, meaning that the time spent running tests can be decreased by a factor of 20.}},
  author       = {{Dunn Ekelund, Edward}},
  issn         = {{1650-2884}},
  language     = {{eng}},
  note         = {{Student Paper}},
  series       = {{LU-CS-EX 2015-20}},
  title        = {{Test Selection Based on Historical Test Data}},
  year         = {{2015}},
}