Advanced

Estimating the number of components with defects post-release that showed no defects in testing

Stringfellow, C; Andrews, A; Wohlin, Claes LU and Petersson, H (2002) In Software Testing, Verification & Reliability 12(2). p.93-122
Abstract
Components that have defects after release, but not during testing, are very undesirable as they point to 'holes' in the testing process. Either new components were not tested enough, or old ones were broken during enhancements and defects slipped through testing undetected. The latter is particularly pernicious, since customers are less forgiving when existing functionality is no longer working than when a new feature is not working quite properly. Rather than using capture-recapture models and curve-fitting methods to estimate the number of remaining defects after inspection, these methods are adapted to estimate the number of components with post-release defects that have no defects in testing. A simple experience-based method is used... (More)
Components that have defects after release, but not during testing, are very undesirable as they point to 'holes' in the testing process. Either new components were not tested enough, or old ones were broken during enhancements and defects slipped through testing undetected. The latter is particularly pernicious, since customers are less forgiving when existing functionality is no longer working than when a new feature is not working quite properly. Rather than using capture-recapture models and curve-fitting methods to estimate the number of remaining defects after inspection, these methods are adapted to estimate the number of components with post-release defects that have no defects in testing. A simple experience-based method is used as a basis for comparison. The estimates can then be used to make decisions on whether or not to stop testing and release software. While most investigations so far have been experimental or have used virtual inspections to do a statistical validation, the investigation presented in this paper is a case study. This case study evaluates how well the capture-recapture, curve-fitting and experience-based methods work in practice. The results show that the methods work quite well. A further benefit of these techniques is that they can be applied to new systems for which no historical data are available and to releases that are very different from each other. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
release decisions, defect estimation, software quality, fault-prone models, capture-recapture methods, curve-fitting methods
in
Software Testing, Verification & Reliability
volume
12
issue
2
pages
93 - 122
publisher
John Wiley & Sons
external identifiers
  • wos:000176633800004
  • scopus:0036609784
ISSN
0960-0833
DOI
10.1002/stvr.235
language
English
LU publication?
yes
id
65a50fa1-ee0c-432e-a45f-08e19a785753 (old id 334003)
date added to LUP
2007-11-09 12:39:01
date last changed
2017-01-01 04:47:13
@article{65a50fa1-ee0c-432e-a45f-08e19a785753,
  abstract     = {Components that have defects after release, but not during testing, are very undesirable as they point to 'holes' in the testing process. Either new components were not tested enough, or old ones were broken during enhancements and defects slipped through testing undetected. The latter is particularly pernicious, since customers are less forgiving when existing functionality is no longer working than when a new feature is not working quite properly. Rather than using capture-recapture models and curve-fitting methods to estimate the number of remaining defects after inspection, these methods are adapted to estimate the number of components with post-release defects that have no defects in testing. A simple experience-based method is used as a basis for comparison. The estimates can then be used to make decisions on whether or not to stop testing and release software. While most investigations so far have been experimental or have used virtual inspections to do a statistical validation, the investigation presented in this paper is a case study. This case study evaluates how well the capture-recapture, curve-fitting and experience-based methods work in practice. The results show that the methods work quite well. A further benefit of these techniques is that they can be applied to new systems for which no historical data are available and to releases that are very different from each other.},
  author       = {Stringfellow, C and Andrews, A and Wohlin, Claes and Petersson, H},
  issn         = {0960-0833},
  keyword      = {release decisions,defect estimation,software quality,fault-prone models,capture-recapture methods,curve-fitting methods},
  language     = {eng},
  number       = {2},
  pages        = {93--122},
  publisher    = {John Wiley & Sons},
  series       = {Software Testing, Verification & Reliability},
  title        = {Estimating the number of components with defects post-release that showed no defects in testing},
  url          = {http://dx.doi.org/10.1002/stvr.235},
  volume       = {12},
  year         = {2002},
}