Advanced

Automated Controlled Experimentation on Software by Evolutionary Bandit Optimization

Ros, Rasmus LU ; Bjarnason, Elizabeth LU and Runeson, Per LU (2017) SSBSE 2017 Symposium on Search-Based Software Engineering In Lecture Notes in Computer Science 10452. p.190-196
Abstract
Controlled experiments, also called A/B tests or split tests, are used in software engineering to improve products by evaluating variants with user data. By parameterizing software systems, multivariate experiments can be performed automatically and in large scale, in this way, controlled experimentation is formulated as an optimization problem. Using genetic algorithms for automated experimentation requires repetitions to evaluate a variant, since the fitness function is noisy. We propose to combine genetic algorithms with bandit optimization to optimize where repetitions are evaluated, instead of uniform sampling. We setup a simulation environment that allows us to evaluate the solution, and see that it leads to increased fitness,... (More)
Controlled experiments, also called A/B tests or split tests, are used in software engineering to improve products by evaluating variants with user data. By parameterizing software systems, multivariate experiments can be performed automatically and in large scale, in this way, controlled experimentation is formulated as an optimization problem. Using genetic algorithms for automated experimentation requires repetitions to evaluate a variant, since the fitness function is noisy. We propose to combine genetic algorithms with bandit optimization to optimize where repetitions are evaluated, instead of uniform sampling. We setup a simulation environment that allows us to evaluate the solution, and see that it leads to increased fitness, population diversity, and rewards, compared to only genetic algorithms. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
in
Lecture Notes in Computer Science
volume
10452
pages
190 - 196
publisher
Springer Verlag
conference name
SSBSE 2017 Symposium on Search-Based Software Engineering
external identifiers
  • scopus:85029371185
ISBN
978-3-319-66299-2
DOI
10.1007/978-3-319-66299-2_18
language
English
LU publication?
yes
id
219e15f8-2f4c-414a-b02d-3d5eb38646a9
date added to LUP
2017-06-21 13:45:56
date last changed
2018-05-29 09:44:21
@inproceedings{219e15f8-2f4c-414a-b02d-3d5eb38646a9,
  abstract     = {Controlled experiments, also called A/B tests or split tests, are used in software engineering to improve products by evaluating variants with user data. By parameterizing software systems, multivariate experiments can be performed automatically and in large scale, in this way, controlled experimentation is formulated as an optimization problem. Using genetic algorithms for automated experimentation requires repetitions to evaluate a variant, since the fitness function is noisy. We propose to combine genetic algorithms with bandit optimization to optimize where repetitions are evaluated, instead of uniform sampling. We setup a simulation environment that allows us to evaluate the solution, and see that it leads to increased fitness, population diversity, and rewards, compared to only genetic algorithms.},
  author       = {Ros, Rasmus and Bjarnason, Elizabeth and Runeson, Per},
  booktitle    = { Lecture Notes in Computer Science},
  isbn         = {978-3-319-66299-2},
  language     = {eng},
  month        = {08},
  pages        = {190--196},
  publisher    = {Springer Verlag},
  title        = {Automated Controlled Experimentation on Software by Evolutionary Bandit Optimization},
  url          = {http://dx.doi.org/10.1007/978-3-319-66299-2_18},
  volume       = {10452},
  year         = {2017},
}