Advanced

Experience from replicating empirical studies on prediction models

Ohlsson, Magnus C and Runeson, Per LU (2002) Proceedings Eighth IEEE Symposium on Software Metrics In Proceedings Eighth IEEE Symposium on Software Metrics p.217-226
Abstract
When conducting empirical studies, replications are important contributors to investigating the generality of the studies. By replicating a study in another context, we investigate what impact the specific environment has, related to the effect of the studied object. In this paper, we define different levels of replication to characterise the similarities and differences between an original study and a replication, with particular focus on prediction models for the identification of fault-prone software components. Further, we derive a set of issues and concerns which are important in order to enable replication of an empirical study and to enable practitioners to use the results. To illustrate the importance of the issues raised, a... (More)
When conducting empirical studies, replications are important contributors to investigating the generality of the studies. By replicating a study in another context, we investigate what impact the specific environment has, related to the effect of the studied object. In this paper, we define different levels of replication to characterise the similarities and differences between an original study and a replication, with particular focus on prediction models for the identification of fault-prone software components. Further, we derive a set of issues and concerns which are important in order to enable replication of an empirical study and to enable practitioners to use the results. To illustrate the importance of the issues raised, a replication case study is presented in the domain of prediction models for fault-prone software components. It is concluded that the results are very divergent, depending on how different parameters are chosen, which demonstrates the need for well-documented empirical studies to enable replication and use (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
documentation, parameter selection method, case study, prediction models, empirical study replications, fault-prone software component identification, study generality, environment impact
in
Proceedings Eighth IEEE Symposium on Software Metrics
pages
217 - 226
publisher
IEEE--Institute of Electrical and Electronics Engineers Inc.
conference name
Proceedings Eighth IEEE Symposium on Software Metrics
external identifiers
  • wos:000176683300021
  • scopus:84948456456
ISBN
0-7695-1339-5
DOI
10.1109/METRIC.2002.1011340
language
English
LU publication?
yes
id
0ba88e06-5cc8-438c-af96-340f25ab018b (old id 610818)
date added to LUP
2007-11-28 12:53:51
date last changed
2017-03-19 04:25:43
@inproceedings{0ba88e06-5cc8-438c-af96-340f25ab018b,
  abstract     = {When conducting empirical studies, replications are important contributors to investigating the generality of the studies. By replicating a study in another context, we investigate what impact the specific environment has, related to the effect of the studied object. In this paper, we define different levels of replication to characterise the similarities and differences between an original study and a replication, with particular focus on prediction models for the identification of fault-prone software components. Further, we derive a set of issues and concerns which are important in order to enable replication of an empirical study and to enable practitioners to use the results. To illustrate the importance of the issues raised, a replication case study is presented in the domain of prediction models for fault-prone software components. It is concluded that the results are very divergent, depending on how different parameters are chosen, which demonstrates the need for well-documented empirical studies to enable replication and use},
  author       = {Ohlsson, Magnus C and Runeson, Per},
  booktitle    = {Proceedings Eighth IEEE Symposium on Software Metrics},
  isbn         = {0-7695-1339-5},
  keyword      = {documentation,parameter selection method,case study,prediction models,empirical study replications,fault-prone software component identification,study generality,environment impact},
  language     = {eng},
  pages        = {217--226},
  publisher    = {IEEE--Institute of Electrical and Electronics Engineers Inc.},
  title        = {Experience from replicating empirical studies on prediction models},
  url          = {http://dx.doi.org/10.1109/METRIC.2002.1011340},
  year         = {2002},
}