Automated Controlled Experimentation on Software by Evolutionary Bandit Optimization
Ros, Rasmus LU ; Bjarnason, Elizabeth LU
and Runeson, Per
LU
(2017)
SSBSE 2017 Symposium on Search-Based Software Engineering
In Lecture Notes in Computer Science
10452.
p.190-196
- Abstract
- Controlled experiments, also called A/B tests or split tests, are used in software engineering to improve products by evaluating variants with user data. By parameterizing software systems, multivariate experiments can be performed automatically and in large scale, in this way, controlled experimentation is formulated as an optimization problem. Using genetic algorithms for automated experimentation requires repetitions to evaluate a variant, since the fitness function is noisy. We propose to combine genetic algorithms with bandit optimization to optimize where repetitions are evaluated, instead of uniform sampling. We setup a simulation environment that allows us to evaluate the solution, and see that it leads to increased fitness,... (More)
- Controlled experiments, also called A/B tests or split tests, are used in software engineering to improve products by evaluating variants with user data. By parameterizing software systems, multivariate experiments can be performed automatically and in large scale, in this way, controlled experimentation is formulated as an optimization problem. Using genetic algorithms for automated experimentation requires repetitions to evaluate a variant, since the fitness function is noisy. We propose to combine genetic algorithms with bandit optimization to optimize where repetitions are evaluated, instead of uniform sampling. We setup a simulation environment that allows us to evaluate the solution, and see that it leads to increased fitness, population diversity, and rewards, compared to only genetic algorithms. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/219e15f8-2f4c-414a-b02d-3d5eb38646a9
- author
- Ros, Rasmus
LU
; Bjarnason, Elizabeth
LU
and Runeson, Per
LU
- organization
- publishing date
- 2017-08-17
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- host publication
- Symposium on Search-Based Software Engineering
- series title
- Lecture Notes in Computer Science
- volume
- 10452
- pages
- 190 - 196
- publisher
- Springer
- conference name
- SSBSE 2017 Symposium on Search-Based Software Engineering
- conference location
- Paderborn, Germany
- conference dates
- 2017-09-09 - 2017-09-11
- external identifiers
-
- scopus:85029371185
- ISBN
- 978-3-319-66299-2
- DOI
- 10.1007/978-3-319-66299-2_18
- project
- Continuous Experimentation and Optimization
- language
- English
- LU publication?
- yes
- id
- 219e15f8-2f4c-414a-b02d-3d5eb38646a9
- date added to LUP
- 2017-06-21 13:45:56
- date last changed
- 2023-09-07 07:19:33
@inproceedings{219e15f8-2f4c-414a-b02d-3d5eb38646a9,
abstract = {{Controlled experiments, also called A/B tests or split tests, are used in software engineering to improve products by evaluating variants with user data. By parameterizing software systems, multivariate experiments can be performed automatically and in large scale, in this way, controlled experimentation is formulated as an optimization problem. Using genetic algorithms for automated experimentation requires repetitions to evaluate a variant, since the fitness function is noisy. We propose to combine genetic algorithms with bandit optimization to optimize where repetitions are evaluated, instead of uniform sampling. We setup a simulation environment that allows us to evaluate the solution, and see that it leads to increased fitness, population diversity, and rewards, compared to only genetic algorithms.}},
author = {{Ros, Rasmus and Bjarnason, Elizabeth and Runeson, Per}},
booktitle = {{Symposium on Search-Based Software Engineering}},
isbn = {{978-3-319-66299-2}},
language = {{eng}},
month = {{08}},
pages = {{190--196}},
publisher = {{Springer}},
series = {{Lecture Notes in Computer Science}},
title = {{Automated Controlled Experimentation on Software by Evolutionary Bandit Optimization}},
url = {{http://dx.doi.org/10.1007/978-3-319-66299-2_18}},
doi = {{10.1007/978-3-319-66299-2_18}},
volume = {{10452}},
year = {{2017}},
}