Advanced

On the Reliability of Mapping Studies in Software Engineering

Wohlin, Claes; Runeson, Per LU ; da Mota Silveira Neto, Paulo Anselmo; Engström, Emelie LU ; do Carmo Machado, Ivan and Santana de Almeida, Eduardo (2013) In Journal of Systems and Software 86(10). p.2594-2610
Abstract
Background: Systematic literature reviews and systematic mapping studies are becoming increasingly common in software engineering, and hence it becomes even more important to better understand the reliability of such studies.

Objective: This paper presents a study of two systematic mapping studies to evaluate the reliability of mapping studies and point out some challenges related to this type of study in software engineering.

Method: The research is based on an in-depth case study of two published mapping studies on software product line testing.

Results: We found that despite the fact that the two studies are addressing the same topic, there are quite a number of differences when it comes to papers included... (More)
Background: Systematic literature reviews and systematic mapping studies are becoming increasingly common in software engineering, and hence it becomes even more important to better understand the reliability of such studies.

Objective: This paper presents a study of two systematic mapping studies to evaluate the reliability of mapping studies and point out some challenges related to this type of study in software engineering.

Method: The research is based on an in-depth case study of two published mapping studies on software product line testing.

Results: We found that despite the fact that the two studies are addressing the same topic, there are quite a number of differences when it comes to papers included and in terms of classification of the papers included in the two mapping studies.

Conclusions: From this we conclude that although mapping studies are important, their reliability cannot simply be taken for granted. Based on the findings we also provide four conjectures that further research has to address to make secondary studies (systematic mapping studies and systematic literature reviews) even more valuable to both researchers and practitioners. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
Systematic mapping study, systematic map, software product lines, systematic reviews, systematic literature review, review of reviews, meta-study, software testing
in
Journal of Systems and Software
volume
86
issue
10
pages
2594 - 2610
publisher
Elsevier
external identifiers
  • wos:000324564100009
  • scopus:84882695716
ISSN
0164-1212
DOI
10.1016/j.jss.2013.04.076
language
English
LU publication?
yes
id
42e4a11b-786a-462c-9ac3-f0297a380275 (old id 3971732)
date added to LUP
2013-08-14 10:47:29
date last changed
2018-06-24 04:07:19
@article{42e4a11b-786a-462c-9ac3-f0297a380275,
  abstract     = {Background: Systematic literature reviews and systematic mapping studies are becoming increasingly common in software engineering, and hence it becomes even more important to better understand the reliability of such studies. <br/><br>
Objective: This paper presents a study of two systematic mapping studies to evaluate the reliability of mapping studies and point out some challenges related to this type of study in software engineering. <br/><br>
Method: The research is based on an in-depth case study of two published mapping studies on software product line testing. <br/><br>
Results: We found that despite the fact that the two studies are addressing the same topic, there are quite a number of differences when it comes to papers included and in terms of classification of the papers included in the two mapping studies. <br/><br>
Conclusions: From this we conclude that although mapping studies are important, their reliability cannot simply be taken for granted. Based on the findings we also provide four conjectures that further research has to address to make secondary studies (systematic mapping studies and systematic literature reviews) even more valuable to both researchers and practitioners.},
  author       = {Wohlin, Claes and Runeson, Per and da Mota Silveira Neto, Paulo Anselmo and Engström, Emelie and do Carmo Machado, Ivan and Santana de Almeida, Eduardo},
  issn         = {0164-1212},
  keyword      = {Systematic mapping study,systematic map,software product lines,systematic reviews,systematic literature review,review of reviews,meta-study,software testing},
  language     = {eng},
  number       = {10},
  pages        = {2594--2610},
  publisher    = {Elsevier},
  series       = {Journal of Systems and Software},
  title        = {On the Reliability of Mapping Studies in Software Engineering},
  url          = {http://dx.doi.org/10.1016/j.jss.2013.04.076},
  volume       = {86},
  year         = {2013},
}