Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces
(2022) Advances in Neural Information Processing Systems 35, NeurIPS 2022- Abstract
- Recent advances have extended the scope of Bayesian optimization (BO) to expensive-to-evaluate black-box functions with dozens of dimensions, aspiring to unlock impactful applications, for example, in the life sciences, neural architecture search, and robotics. However, a closer examination reveals that the state-of-the-art methods for high-dimensional Bayesian optimization (HDBO) suffer from degrading performance as the number of dimensions increases, or even risk failure if certain unverifiable assumptions are not met. This paper proposes BAxUS that leverages a novel family of nested random subspaces to adapt the space it optimizes over to the problem. This ensures high performance while removing the risk of failure, which we assert via... (More)
- Recent advances have extended the scope of Bayesian optimization (BO) to expensive-to-evaluate black-box functions with dozens of dimensions, aspiring to unlock impactful applications, for example, in the life sciences, neural architecture search, and robotics. However, a closer examination reveals that the state-of-the-art methods for high-dimensional Bayesian optimization (HDBO) suffer from degrading performance as the number of dimensions increases, or even risk failure if certain unverifiable assumptions are not met. This paper proposes BAxUS that leverages a novel family of nested random subspaces to adapt the space it optimizes over to the problem. This ensures high performance while removing the risk of failure, which we assert via theoretical guarantees. A comprehensive evaluation demonstrates that BAxUS achieves better results than the state-of-the-art methods for a broad set of applications. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/45016f41-f384-4a10-8784-5a2b542f5da9
- author
- Papenmeier, Leonard LU ; Nardi, Luigi LU and Poloczek, Matthias
- organization
- publishing date
- 2022
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- keywords
- Bayesian Optimization, Global Optimization, Gaussian Process, high-dimensional
- host publication
- Advances in Neural Information Processing Systems, NeurIPS 2022
- publisher
- Curran Associates, Inc
- conference name
- Advances in Neural Information Processing Systems 35, NeurIPS 2022
- conference location
- New Oreleans, United States
- conference dates
- 2022-11-28 - 2022-12-09
- ISBN
- 9781713871088
- language
- English
- LU publication?
- yes
- id
- 45016f41-f384-4a10-8784-5a2b542f5da9
- alternative location
- https://openreview.net/pdf?id=e4Wf6112DI
- date added to LUP
- 2022-09-19 09:24:28
- date last changed
- 2023-10-06 09:31:49
@inproceedings{45016f41-f384-4a10-8784-5a2b542f5da9, abstract = {{Recent advances have extended the scope of Bayesian optimization (BO) to expensive-to-evaluate black-box functions with dozens of dimensions, aspiring to unlock impactful applications, for example, in the life sciences, neural architecture search, and robotics. However, a closer examination reveals that the state-of-the-art methods for high-dimensional Bayesian optimization (HDBO) suffer from degrading performance as the number of dimensions increases, or even risk failure if certain unverifiable assumptions are not met. This paper proposes BAxUS that leverages a novel family of nested random subspaces to adapt the space it optimizes over to the problem. This ensures high performance while removing the risk of failure, which we assert via theoretical guarantees. A comprehensive evaluation demonstrates that BAxUS achieves better results than the state-of-the-art methods for a broad set of applications.}}, author = {{Papenmeier, Leonard and Nardi, Luigi and Poloczek, Matthias}}, booktitle = {{Advances in Neural Information Processing Systems, NeurIPS 2022}}, isbn = {{9781713871088}}, keywords = {{Bayesian Optimization; Global Optimization; Gaussian Process; high-dimensional}}, language = {{eng}}, publisher = {{Curran Associates, Inc}}, title = {{Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces}}, url = {{https://openreview.net/pdf?id=e4Wf6112DI}}, year = {{2022}}, }