Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Bayesian Optimization with a Prior for the Optimum

Souza, Artur ; Nardi, Luigi LU ; Oliveira, Leonardo B. ; Olukotun, Kunle ; Lindauer, Marius and Hutter, Frank (2021) European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2021 In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12977 LNAI. p.265-296
Abstract

While Bayesian Optimization (BO) is a very popular method for optimizing expensive black-box functions, it fails to leverage the experience of domain experts. This causes BO to waste function evaluations on bad design choices (e.g., machine learning hyperparameters) that the expert already knows to work poorly. To address this issue, we introduce Bayesian Optimization with a Prior for the Optimum (BOPrO). BOPrO allows users to inject their knowledge into the optimization process in the form of priors about which parts of the input space will yield the best performance, rather than BO’s standard priors over functions, which are much less intuitive for users. BOPrO then combines these priors with BO’s standard probabilistic model to form... (More)

While Bayesian Optimization (BO) is a very popular method for optimizing expensive black-box functions, it fails to leverage the experience of domain experts. This causes BO to waste function evaluations on bad design choices (e.g., machine learning hyperparameters) that the expert already knows to work poorly. To address this issue, we introduce Bayesian Optimization with a Prior for the Optimum (BOPrO). BOPrO allows users to inject their knowledge into the optimization process in the form of priors about which parts of the input space will yield the best performance, rather than BO’s standard priors over functions, which are much less intuitive for users. BOPrO then combines these priors with BO’s standard probabilistic model to form a pseudo-posterior used to select which points to evaluate next. We show that BOPrO is around 6.67 × faster than state-of-the-art methods on a common suite of benchmarks, and achieves a new state-of-the-art performance on a real-world hardware design application. We also show that BOPrO converges faster even if the priors for the optimum are not entirely accurate and that it robustly recovers from misleading priors.

(Less)
Please use this url to cite or link to this publication:
author
; ; ; ; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
Machine Learning and Knowledge Discovery in Databases. Research Track - European Conference, ECML PKDD 2021, Proceedings
series title
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
editor
Oliver, Nuria ; Pérez-Cruz, Fernando ; Kramer, Stefan ; Read, Jesse and Lozano, Jose A.
volume
12977 LNAI
pages
32 pages
publisher
Springer Science and Business Media B.V.
conference name
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2021
conference location
Virtual, Online
conference dates
2021-09-13 - 2021-09-17
external identifiers
  • scopus:85115712403
ISSN
1611-3349
0302-9743
ISBN
9783030865221
DOI
10.1007/978-3-030-86523-8_17
language
English
LU publication?
yes
id
fc65e94b-0ef4-4c84-a5ba-f9d93cb6d89a
date added to LUP
2021-10-08 14:29:37
date last changed
2024-04-06 10:16:28
@inproceedings{fc65e94b-0ef4-4c84-a5ba-f9d93cb6d89a,
  abstract     = {{<p>While Bayesian Optimization (BO) is a very popular method for optimizing expensive black-box functions, it fails to leverage the experience of domain experts. This causes BO to waste function evaluations on bad design choices (e.g., machine learning hyperparameters) that the expert already knows to work poorly. To address this issue, we introduce Bayesian Optimization with a Prior for the Optimum (BOPrO). BOPrO allows users to inject their knowledge into the optimization process in the form of priors about which parts of the input space will yield the best performance, rather than BO’s standard priors over functions, which are much less intuitive for users. BOPrO then combines these priors with BO’s standard probabilistic model to form a pseudo-posterior used to select which points to evaluate next. We show that BOPrO is around 6.67 × faster than state-of-the-art methods on a common suite of benchmarks, and achieves a new state-of-the-art performance on a real-world hardware design application. We also show that BOPrO converges faster even if the priors for the optimum are not entirely accurate and that it robustly recovers from misleading priors.</p>}},
  author       = {{Souza, Artur and Nardi, Luigi and Oliveira, Leonardo B. and Olukotun, Kunle and Lindauer, Marius and Hutter, Frank}},
  booktitle    = {{Machine Learning and Knowledge Discovery in Databases. Research Track - European Conference, ECML PKDD 2021, Proceedings}},
  editor       = {{Oliver, Nuria and Pérez-Cruz, Fernando and Kramer, Stefan and Read, Jesse and Lozano, Jose A.}},
  isbn         = {{9783030865221}},
  issn         = {{1611-3349}},
  language     = {{eng}},
  pages        = {{265--296}},
  publisher    = {{Springer Science and Business Media B.V.}},
  series       = {{Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)}},
  title        = {{Bayesian Optimization with a Prior for the Optimum}},
  url          = {{http://dx.doi.org/10.1007/978-3-030-86523-8_17}},
  doi          = {{10.1007/978-3-030-86523-8_17}},
  volume       = {{12977 LNAI}},
  year         = {{2021}},
}