Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

On Rapid Releases and Software Testing

Mäntylä, Mika LU ; Khomh, Foutse ; Adams, Bram ; Engström, Emelie LU orcid and Kai, Petersen (2013) 29th IEEE International Conference on Maintenance (ICSM), 2013 p.20-29
Abstract
Large open and closed source organizations like Google, Facebook and Mozilla are migrating their products towards rapid releases. While this allows faster time-to-market and user feedback, it also implies less time for testing and bug fixing. Since initial research results indeed show that rapid releases fix proportionally less reported bugs than traditional releases, this paper investigates the changes in software testing effort after moving to rapid releases. We analyze the results of 312,502 execution runs of the 1,547 mostly manual system level test cases of Mozilla Fire fox from 2006 to 2012 (5 major traditional and 9 major rapid releases), and triangulated our findings with a Mozilla QA engineer. In rapid releases, testing has a... (More)
Large open and closed source organizations like Google, Facebook and Mozilla are migrating their products towards rapid releases. While this allows faster time-to-market and user feedback, it also implies less time for testing and bug fixing. Since initial research results indeed show that rapid releases fix proportionally less reported bugs than traditional releases, this paper investigates the changes in software testing effort after moving to rapid releases. We analyze the results of 312,502 execution runs of the 1,547 mostly manual system level test cases of Mozilla Fire fox from 2006 to 2012 (5 major traditional and 9 major rapid releases), and triangulated our findings with a Mozilla QA engineer. In rapid releases, testing has a narrower scope that enables deeper investigation of the features and regressions with the highest risk, while traditional releases run the whole test suite. Furthermore, rapid releases make it more difficult to build a large testing community, forcing Mozilla to increase contractor resources in order to sustain testing for rapid releases. (Less)
Please use this url to cite or link to this publication:
author
; ; ; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
Mozilla Software testing agile releases bugs builds opensource release model
host publication
[Host publication title missing]
editor
Lisa, O'Conner
pages
20 - 29
publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
conference name
29th IEEE International Conference on Maintenance (ICSM), 2013
conference location
Eindhoven, Netherlands
conference dates
2013-09-22 - 2013-09-28
external identifiers
  • scopus:84891723874
  • wos:000332836100003
ISSN
1063-6773
DOI
10.1109/ICSM.2013.13
language
English
LU publication?
yes
id
daff54e9-3518-4d24-b882-8e656fc9ba62 (old id 4195378)
date added to LUP
2016-04-01 12:59:57
date last changed
2022-01-27 08:46:01
@inproceedings{daff54e9-3518-4d24-b882-8e656fc9ba62,
  abstract     = {{Large open and closed source organizations like Google, Facebook and Mozilla are migrating their products towards rapid releases. While this allows faster time-to-market and user feedback, it also implies less time for testing and bug fixing. Since initial research results indeed show that rapid releases fix proportionally less reported bugs than traditional releases, this paper investigates the changes in software testing effort after moving to rapid releases. We analyze the results of 312,502 execution runs of the 1,547 mostly manual system level test cases of Mozilla Fire fox from 2006 to 2012 (5 major traditional and 9 major rapid releases), and triangulated our findings with a Mozilla QA engineer. In rapid releases, testing has a narrower scope that enables deeper investigation of the features and regressions with the highest risk, while traditional releases run the whole test suite. Furthermore, rapid releases make it more difficult to build a large testing community, forcing Mozilla to increase contractor resources in order to sustain testing for rapid releases.}},
  author       = {{Mäntylä, Mika and Khomh, Foutse and Adams, Bram and Engström, Emelie and Kai, Petersen}},
  booktitle    = {{[Host publication title missing]}},
  editor       = {{Lisa, O'Conner}},
  issn         = {{1063-6773}},
  keywords     = {{Mozilla Software testing agile releases bugs builds opensource release model}},
  language     = {{eng}},
  pages        = {{20--29}},
  publisher    = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
  title        = {{On Rapid Releases and Software Testing}},
  url          = {{http://dx.doi.org/10.1109/ICSM.2013.13}},
  doi          = {{10.1109/ICSM.2013.13}},
  year         = {{2013}},
}