Advanced

Efficient Regression Testing Based on Test History: An Industrial Evaluation

Ekelund, Edward Dunn and Engström, Emelie LU (2015) International Conference on Software Maintenance and Evolution In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)
Abstract
Due to changes in the development practices at Axis Communications, towards continuous integration, faster regression testing feedback is needed. The current automated regression test suite takes approximately seven hours to run which prevents developers from integrating code changes several times a day as preferred. Therefore we want to implement a highly selective yet accurate regression testing strategy. Traditional code coverage based techniques are not applicable due to the size and complexity of the software under test. Instead we decided to select tests based on regression test history. We developed a tool, the Difference Engine, which parses and analyzes results from previous test runs and outputs regression test recommendations.... (More)
Due to changes in the development practices at Axis Communications, towards continuous integration, faster regression testing feedback is needed. The current automated regression test suite takes approximately seven hours to run which prevents developers from integrating code changes several times a day as preferred. Therefore we want to implement a highly selective yet accurate regression testing strategy. Traditional code coverage based techniques are not applicable due to the size and complexity of the software under test. Instead we decided to select tests based on regression test history. We developed a tool, the Difference Engine, which parses and analyzes results from previous test runs and outputs regression test recommendations. The Difference Engine correlates code and test cases at package level and recommends test cases that are strongly correlated to recently changed packages. We evaluated the technique with respect to correctness, precision, recall and efficiency. Our results are promising. On average the tool manages to identify 80% of the relevant tests while recommending only 4% of the test cases in the full regression test suite. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
in
2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)
pages
9 pages
publisher
IEEE--Institute of Electrical and Electronics Engineers Inc.
conference name
International Conference on Software Maintenance and Evolution
external identifiers
  • Scopus:84961644861
DOI
10.1109/ICSM.2015.7332496
language
English
LU publication?
yes
id
ce239e5f-6811-4e9b-93ac-3d7a32b0eef6 (old id 7989669)
date added to LUP
2015-09-24 09:45:12
date last changed
2016-10-13 04:37:48
@misc{ce239e5f-6811-4e9b-93ac-3d7a32b0eef6,
  abstract     = {Due to changes in the development practices at Axis Communications, towards continuous integration, faster regression testing feedback is needed. The current automated regression test suite takes approximately seven hours to run which prevents developers from integrating code changes several times a day as preferred. Therefore we want to implement a highly selective yet accurate regression testing strategy. Traditional code coverage based techniques are not applicable due to the size and complexity of the software under test. Instead we decided to select tests based on regression test history. We developed a tool, the Difference Engine, which parses and analyzes results from previous test runs and outputs regression test recommendations. The Difference Engine correlates code and test cases at package level and recommends test cases that are strongly correlated to recently changed packages. We evaluated the technique with respect to correctness, precision, recall and efficiency. Our results are promising. On average the tool manages to identify 80% of the relevant tests while recommending only 4% of the test cases in the full regression test suite.},
  author       = {Ekelund, Edward Dunn and Engström, Emelie},
  language     = {eng},
  pages        = {9},
  publisher    = {ARRAY(0x9abc5e8)},
  series       = {2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)},
  title        = {Efficient Regression Testing Based on Test History: An Industrial Evaluation},
  url          = {http://dx.doi.org/10.1109/ICSM.2015.7332496},
  year         = {2015},
}