Advanced

TuneR: A Framework for Tuning Software Engineering Tools with Hands-on Instructions in R

BORG, MARKUS LU (2016) In Journal of software: Evolution and Process 28(6). p.427-459
Abstract
Numerous tools automating various aspects of software engineering have been developed, and many of the tools are highly configurable through parameters. Understanding the parameters of advanced tools often requires deep understanding of complex algorithms. Unfortunately, suboptimal parameter settings limit the performance of tools and hinder industrial adaptation, but still few studies address the challenge of tuning software engineering tools. We present TuneR, an experiment framework that supports finding feasible parameter settings using empirical methods. The framework is accompanied by practical guidelines of how to use R to analyze the experimental outcome. As a proof-of-concept, we apply TuneR to tune ImpRec, a recommendation system... (More)
Numerous tools automating various aspects of software engineering have been developed, and many of the tools are highly configurable through parameters. Understanding the parameters of advanced tools often requires deep understanding of complex algorithms. Unfortunately, suboptimal parameter settings limit the performance of tools and hinder industrial adaptation, but still few studies address the challenge of tuning software engineering tools. We present TuneR, an experiment framework that supports finding feasible parameter settings using empirical methods. The framework is accompanied by practical guidelines of how to use R to analyze the experimental outcome. As a proof-of-concept, we apply TuneR to tune ImpRec, a recommendation system for change impact analysis in a software system that has evolved for more than two decades. Compared with the output from the default setting, we report a 20.9% improvement in the response variable reflecting recommendation accuracy. Moreover, TuneR reveals insights into the interaction among parameters, as well as nonlinear effects. TuneR is easy to use, thus the framework has potential to support tuning of software engineering tools in both academia and industry. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
software engineering tools, parameter tuning, experiment framework, empirical software engineering, change impact analysis
in
Journal of software: Evolution and Process
volume
28
issue
6
pages
33 pages
publisher
John Wiley & Sons
external identifiers
  • Scopus:84965170217
ISSN
2047-7481
DOI
10.1002/smr.1784
language
English
LU publication?
yes
id
84b67024-154d-4c4b-a148-9d36aa98d090
date added to LUP
2016-05-04 08:45:54
date last changed
2016-10-13 05:07:33
@misc{84b67024-154d-4c4b-a148-9d36aa98d090,
  abstract     = {Numerous tools automating various aspects of software engineering have been developed, and many of the tools are highly configurable through parameters. Understanding the parameters of advanced tools often requires deep understanding of complex algorithms. Unfortunately, suboptimal parameter settings limit the performance of tools and hinder industrial adaptation, but still few studies address the challenge of tuning software engineering tools. We present TuneR, an experiment framework that supports finding feasible parameter settings using empirical methods. The framework is accompanied by practical guidelines of how to use R to analyze the experimental outcome. As a proof-of-concept, we apply TuneR to tune ImpRec, a recommendation system for change impact analysis in a software system that has evolved for more than two decades. Compared with the output from the default setting, we report a 20.9% improvement in the response variable reflecting recommendation accuracy. Moreover, TuneR reveals insights into the interaction among parameters, as well as nonlinear effects. TuneR is easy to use, thus the framework has potential to support tuning of software engineering tools in both academia and industry.},
  author       = {BORG, MARKUS},
  issn         = {2047-7481},
  keyword      = {software engineering tools,parameter tuning,experiment framework,empirical software engineering,change impact analysis},
  language     = {eng},
  month        = {05},
  number       = {6},
  pages        = {427--459},
  publisher    = {ARRAY(0x8aa45d8)},
  series       = {Journal of software: Evolution and Process},
  title        = {TuneR: A Framework for Tuning Software Engineering Tools with Hands-on Instructions in R},
  url          = {http://dx.doi.org/10.1002/smr.1784},
  volume       = {28},
  year         = {2016},
}