Advanced

Supporting Decisions on Regression test Scoping in a Software Product Line Context - from Evidence to Practice

Engström, Emelie LU (2013)
Abstract
Large software organizations with a product line development approach face many challenges regarding testing. Test managers need to make tradeoffs along three dimensions of repeated testing (abstraction level, time and product space) and consider a range of goals and constraints. In order to keep pace with the decreased development time for new products, which is enabled by the systematic reuse, selective testing of product variants is necessary. A common industrial practice is to base test scoping decisions on practitioners’ expertise and experience. However, as software systems grow in size and complexity, the combinatorial explosion of test possibilities makes it infeasible to assess the reasonableness of the decisions without tool... (More)
Large software organizations with a product line development approach face many challenges regarding testing. Test managers need to make tradeoffs along three dimensions of repeated testing (abstraction level, time and product space) and consider a range of goals and constraints. In order to keep pace with the decreased development time for new products, which is enabled by the systematic reuse, selective testing of product variants is necessary. A common industrial practice is to base test scoping decisions on practitioners’ expertise and experience. However, as software systems grow in size and complexity, the combinatorial explosion of test possibilities makes it infeasible to assess the reasonableness of the decisions without tool support.

Research on regression test selection propose several systematic strategies for setting a proper test scope when verifying changes of previously tested code. The goal of our research is to improve test management and reduce the amount of redundant testing in the product line context by applying regression test selection strategies. However, despite extensive research on regression testing, gained knowledge has not impacted on industry practices. Thus a secondary goal of our research is to bridge the gap between research and practice in the field of regression testing. Test planning support, like any decision support system, must operate in a complex context and need to be pragmatic, adapted to the context and evolve incrementally within the context.

This thesis explores state of art and state of practice of regression testing and software product line testing, and proposes and evaluates a visualization strategy to support regression test scoping in the product line context. Two extensive systematic literature reviews are conducted as well as four empirical studies in close cooperation with industry. Through visualization of relevant information at a proper

level of detail, test management in general may be supported. A visual analytics tool for test management would also provide a framework which enables research based and context specific regression testing improvements. (Less)
Please use this url to cite or link to this publication:
author
supervisor
opponent
  • Smolander, Kari, Lappeenranta University of Technology, Finland
organization
publishing date
type
Thesis
publication status
published
subject
pages
228 pages
defense location
Lecture hall E:1406, E-building, Ole Römers väg 3, Lund University Faculty of Engineering
defense date
2013-04-05 13:15
ISSN
1404-1219
ISBN
978-91-980754-1-0
language
English
LU publication?
yes
id
da1dba41-530f-4eb7-877d-908c5e8c8521 (old id 3563108)
date added to LUP
2013-03-13 14:07:45
date last changed
2016-09-19 08:45:01
@misc{da1dba41-530f-4eb7-877d-908c5e8c8521,
  abstract     = {Large software organizations with a product line development approach face many challenges regarding testing. Test managers need to make tradeoffs along three dimensions of repeated testing (abstraction level, time and product space) and consider a range of goals and constraints. In order to keep pace with the decreased development time for new products, which is enabled by the systematic reuse, selective testing of product variants is necessary. A common industrial practice is to base test scoping decisions on practitioners’ expertise and experience. However, as software systems grow in size and complexity, the combinatorial explosion of test possibilities makes it infeasible to assess the reasonableness of the decisions without tool support.<br/><br>
Research on regression test selection propose several systematic strategies for setting a proper test scope when verifying changes of previously tested code. The goal of our research is to improve test management and reduce the amount of redundant testing in the product line context by applying regression test selection strategies. However, despite extensive research on regression testing, gained knowledge has not impacted on industry practices. Thus a secondary goal of our research is to bridge the gap between research and practice in the field of regression testing. Test planning support, like any decision support system, must operate in a complex context and need to be pragmatic, adapted to the context and evolve incrementally within the context.<br/><br>
This thesis explores state of art and state of practice of regression testing and software product line testing, and proposes and evaluates a visualization strategy to support regression test scoping in the product line context. Two extensive systematic literature reviews are conducted as well as four empirical studies in close cooperation with industry. Through visualization of relevant information at a proper<br/><br>
level of detail, test management in general may be supported. A visual analytics tool for test management would also provide a framework which enables research based and context specific regression testing improvements.},
  author       = {Engström, Emelie},
  isbn         = {978-91-980754-1-0},
  issn         = {1404-1219},
  language     = {eng},
  pages        = {228},
  title        = {Supporting Decisions on Regression test Scoping in a Software Product Line Context - from Evidence to Practice},
  year         = {2013},
}