Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Data-driven software design with Constraint Oriented Multi-variate Bandit Optimization (COMBO)

Ros, Rasmus LU and Hammar, Mikael LU (2020) In Empirical Software Engineering
Abstract
Software design in e-commerce can be improved with user data through controlled experiments (i.e. A/B tests) to better meet user needs. Machine learning-based algorithmic optimization techniques extends the approach to large number of variables to personalize software to different user needs. So far the optimization techniques has only been applied to optimize software of low complexity, such as colors and wordings of text. In this paper, we introduce the COMBO toolkit with capability to model optimization variables and their relationship constraints specified through an embedded domain-specific language. The toolkit generates personalized software configurations for users as they arrive in the system, and the configurations improve over... (More)
Software design in e-commerce can be improved with user data through controlled experiments (i.e. A/B tests) to better meet user needs. Machine learning-based algorithmic optimization techniques extends the approach to large number of variables to personalize software to different user needs. So far the optimization techniques has only been applied to optimize software of low complexity, such as colors and wordings of text. In this paper, we introduce the COMBO toolkit with capability to model optimization variables and their relationship constraints specified through an embedded domain-specific language. The toolkit generates personalized software configurations for users as they arrive in the system, and the configurations improve over time in in relation to some given metric. COMBO has several implementations of machine learning algorithms and constraint solvers to optimize the model with user data by software developers without deep optimization knowledge. The toolkit was validated in a proof-of-concept by implementing two features that are relevant to Apptus, an e-commerce company that develops algorithms for web shops. The algorithmic performance was evaluated in simulations with realistic historic user data. The validation shows that the toolkit approach can model and improve relatively complex features with many types of variables and constraints, without causing noticeable delays for users. We show that modeling software hierarchies in a formal model facilitates algorithmic optimization of more complex software. In this way, using COMBO, developers can make data-driven and personalized software products. (Less)
Please use this url to cite or link to this publication:
author
and
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
Continuous experimentation, A/B testing, Machine learning, Multi-armed bandits, Combinatorial optimization
in
Empirical Software Engineering
publisher
Springer
external identifiers
  • scopus:85089561236
ISSN
1382-3256
DOI
10.1007/s10664-020-09856-1
project
Continuous Experimentation and Optimization
language
English
LU publication?
yes
id
fa6c314e-3bdc-4150-afd1-98a5c1102bde
date added to LUP
2020-08-25 21:53:56
date last changed
2022-04-19 00:23:38
@article{fa6c314e-3bdc-4150-afd1-98a5c1102bde,
  abstract     = {{Software design in e-commerce can be improved with user data through controlled experiments (i.e. A/B tests) to better meet user needs. Machine learning-based algorithmic optimization techniques extends the approach to large number of variables to personalize software to different user needs. So far the optimization techniques has only been applied to optimize software of low complexity, such as colors and wordings of text. In this paper, we introduce the COMBO toolkit with capability to model optimization variables and their relationship constraints specified through an embedded domain-specific language. The toolkit generates personalized software configurations for users as they arrive in the system, and the configurations improve over time in in relation to some given metric. COMBO has several implementations of machine learning algorithms and constraint solvers to optimize the model with user data by software developers without deep optimization knowledge. The toolkit was validated in a proof-of-concept by implementing two features that are relevant to Apptus, an e-commerce company that develops algorithms for web shops. The algorithmic performance was evaluated in simulations with realistic historic user data. The validation shows that the toolkit approach can model and improve relatively complex features with many types of variables and constraints, without causing noticeable delays for users. We show that modeling software hierarchies in a formal model facilitates algorithmic optimization of more complex software. In this way, using COMBO, developers can make data-driven and personalized software products.}},
  author       = {{Ros, Rasmus and Hammar, Mikael}},
  issn         = {{1382-3256}},
  keywords     = {{Continuous experimentation; A/B testing; Machine learning; Multi-armed bandits; Combinatorial optimization}},
  language     = {{eng}},
  month        = {{08}},
  publisher    = {{Springer}},
  series       = {{Empirical Software Engineering}},
  title        = {{Data-driven software design with Constraint Oriented Multi-variate Bandit Optimization (COMBO)}},
  url          = {{http://dx.doi.org/10.1007/s10664-020-09856-1}},
  doi          = {{10.1007/s10664-020-09856-1}},
  year         = {{2020}},
}