Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review

Unterkalmsteiner, Michael ; Gorschek, Tony ; Islam, A.K.M. Moinul ; Cheng, Chow Kian ; Permadi, Rahadian Bayu and Feldt, Robert (2012) In IEEE Transactions on Software Engineering 38(2). p.398-424
Abstract
BACKGROUND—Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE—This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD—The systematic literature review includes 148 papers published between 1991 and 2008. The selected papers were classified according to SPI initiative, applied evaluation strategies, and measurement perspectives. Potential confounding factors interfering with the evaluation of the improvement effort were assessed. RESULTS—Seven distinct evaluation strategies were identified, wherein the most common... (More)
BACKGROUND—Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE—This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD—The systematic literature review includes 148 papers published between 1991 and 2008. The selected papers were classified according to SPI initiative, applied evaluation strategies, and measurement perspectives. Potential confounding factors interfering with the evaluation of the improvement effort were assessed. RESULTS—Seven distinct evaluation strategies were identified, wherein the most common one, “Pre-Post Comparison,” was applied in 49 percent of the inspected papers. Quality was the most measured attribute (62 percent), followed by Cost (41 percent), and Schedule (18 percent). Looking at measurement perspectives, “Project” represents the majority with 66 percent. CONCLUSION—The evaluation validity of SPI initiatives is challenged by the scarce consideration of potential confounding factors, particularly given that “Pre-Post Comparison” was identified as the most common evaluation strategy, and the inaccurate descriptions of the evaluation context. Measurements to assess the short and mid-term impact of SPI initiatives prevail, whereas long-term measurements in terms of customer satisfaction and return on investment tend to be less used. (Less)
Please use this url to cite or link to this publication:
author
; ; ; ; and
publishing date
type
Contribution to journal
publication status
published
subject
in
IEEE Transactions on Software Engineering
volume
38
issue
2
pages
398 - 424
publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
external identifiers
  • scopus:84859747286
ISSN
0098-5589
DOI
10.1109/TSE.2011.26
project
Embedded Applications Software Engineering
language
English
LU publication?
no
additional info
Software Eng. Res. Lab., Blekinge Inst. of Technol., Karlskrona, Sweden
id
eaf802fe-d32e-41c8-aab2-44503fff7844 (old id 2438945)
date added to LUP
2016-04-01 14:28:45
date last changed
2023-04-06 23:06:09
@article{eaf802fe-d32e-41c8-aab2-44503fff7844,
  abstract     = {{BACKGROUND—Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE—This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD—The systematic literature review includes 148 papers published between 1991 and 2008. The selected papers were classified according to SPI initiative, applied evaluation strategies, and measurement perspectives. Potential confounding factors interfering with the evaluation of the improvement effort were assessed. RESULTS—Seven distinct evaluation strategies were identified, wherein the most common one, “Pre-Post Comparison,” was applied in 49 percent of the inspected papers. Quality was the most measured attribute (62 percent), followed by Cost (41 percent), and Schedule (18 percent). Looking at measurement perspectives, “Project” represents the majority with 66 percent. CONCLUSION—The evaluation validity of SPI initiatives is challenged by the scarce consideration of potential confounding factors, particularly given that “Pre-Post Comparison” was identified as the most common evaluation strategy, and the inaccurate descriptions of the evaluation context. Measurements to assess the short and mid-term impact of SPI initiatives prevail, whereas long-term measurements in terms of customer satisfaction and return on investment tend to be less used.}},
  author       = {{Unterkalmsteiner, Michael and Gorschek, Tony and Islam, A.K.M. Moinul and Cheng, Chow Kian and Permadi, Rahadian Bayu and Feldt, Robert}},
  issn         = {{0098-5589}},
  language     = {{eng}},
  number       = {{2}},
  pages        = {{398--424}},
  publisher    = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
  series       = {{IEEE Transactions on Software Engineering}},
  title        = {{Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review}},
  url          = {{http://dx.doi.org/10.1109/TSE.2011.26}},
  doi          = {{10.1109/TSE.2011.26}},
  volume       = {{38}},
  year         = {{2012}},
}