Advanced

Test Overlay in an Emerging Software Product Line – An Industrial Case Study

Engström, Emelie LU and Runeson, Per LU (2013) In Information and Software Technology 55(3). p.581-594
Abstract
Context: In large software organizations with a product line development approach, system test planning and scope selection is a complex task. Due to repeated testing: across different testing levels, over time (test for regression) as well as of different variants, the risk of redundant testing is large as well as the risk of overlooking important tests, hidden by the huge amount of possible tests. Aims: This study assesses the amount and type of overlaid manual test- ing across feature, integration and system test in such context, it explores the causes of potential redundancy and elaborates on how to provide decision sup- port in terms of visualization for the purpose of avoiding redundancy. Method: An in-depth case study was launched... (More)
Context: In large software organizations with a product line development approach, system test planning and scope selection is a complex task. Due to repeated testing: across different testing levels, over time (test for regression) as well as of different variants, the risk of redundant testing is large as well as the risk of overlooking important tests, hidden by the huge amount of possible tests. Aims: This study assesses the amount and type of overlaid manual test- ing across feature, integration and system test in such context, it explores the causes of potential redundancy and elaborates on how to provide decision sup- port in terms of visualization for the purpose of avoiding redundancy. Method: An in-depth case study was launched including both qualitative and quantitative observations. Results: A high degree of test overlay is identified originating from distributed test responsibilities, poor documentation and structure of test cases, parallel work and insufficient delta analysis. The amount of test overlay depends on which level of abstraction is studied. Conclusions: Avoiding re- dundancy requires tool support, e.g. visualization of test design coverage, test execution progress, priorities of coverage items as well as visualized priorities of variants to support test case selection. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
in
Information and Software Technology
volume
55
issue
3
pages
581 - 594
publisher
Elsevier
external identifiers
  • wos:000315369200007
  • scopus:84872969667
ISSN
0950-5849
DOI
10.1016/j.infsof.2012.04.009
language
English
LU publication?
yes
id
a9ca8886-5654-4126-89bf-c974fb62af01 (old id 2518836)
date added to LUP
2012-05-02 14:31:59
date last changed
2019-03-19 01:56:36
@article{a9ca8886-5654-4126-89bf-c974fb62af01,
  abstract     = {Context: In large software organizations with a product line development approach, system test planning and scope selection is a complex task. Due to repeated testing: across different testing levels, over time (test for regression) as well as of different variants, the risk of redundant testing is large as well as the risk of overlooking important tests, hidden by the huge amount of possible tests. Aims: This study assesses the amount and type of overlaid manual test- ing across feature, integration and system test in such context, it explores the causes of potential redundancy and elaborates on how to provide decision sup- port in terms of visualization for the purpose of avoiding redundancy. Method: An in-depth case study was launched including both qualitative and quantitative observations. Results: A high degree of test overlay is identified originating from distributed test responsibilities, poor documentation and structure of test cases, parallel work and insufficient delta analysis. The amount of test overlay depends on which level of abstraction is studied. Conclusions: Avoiding re- dundancy requires tool support, e.g. visualization of test design coverage, test execution progress, priorities of coverage items as well as visualized priorities of variants to support test case selection.},
  author       = {Engström, Emelie and Runeson, Per},
  issn         = {0950-5849},
  language     = {eng},
  number       = {3},
  pages        = {581--594},
  publisher    = {Elsevier},
  series       = {Information and Software Technology},
  title        = {Test Overlay in an Emerging Software Product Line – An Industrial Case Study},
  url          = {http://dx.doi.org/10.1016/j.infsof.2012.04.009},
  volume       = {55},
  year         = {2013},
}