Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces

Papenmeier, Leonard LU orcid ; Nardi, Luigi LU and Poloczek, Matthias (2022) Advances in Neural Information Processing Systems 35, NeurIPS 2022
Abstract
Recent advances have extended the scope of Bayesian optimization (BO) to expensive-to-evaluate black-box functions with dozens of dimensions, aspiring to unlock impactful applications, for example, in the life sciences, neural architecture search, and robotics. However, a closer examination reveals that the state-of-the-art methods for high-dimensional Bayesian optimization (HDBO) suffer from degrading performance as the number of dimensions increases, or even risk failure if certain unverifiable assumptions are not met. This paper proposes BAxUS that leverages a novel family of nested random subspaces to adapt the space it optimizes over to the problem. This ensures high performance while removing the risk of failure, which we assert via... (More)
Recent advances have extended the scope of Bayesian optimization (BO) to expensive-to-evaluate black-box functions with dozens of dimensions, aspiring to unlock impactful applications, for example, in the life sciences, neural architecture search, and robotics. However, a closer examination reveals that the state-of-the-art methods for high-dimensional Bayesian optimization (HDBO) suffer from degrading performance as the number of dimensions increases, or even risk failure if certain unverifiable assumptions are not met. This paper proposes BAxUS that leverages a novel family of nested random subspaces to adapt the space it optimizes over to the problem. This ensures high performance while removing the risk of failure, which we assert via theoretical guarantees. A comprehensive evaluation demonstrates that BAxUS achieves better results than the state-of-the-art methods for a broad set of applications. (Less)
Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
Bayesian Optimization, Global Optimization, Gaussian Process, high-dimensional
host publication
Advances in Neural Information Processing Systems, NeurIPS 2022
publisher
Curran Associates, Inc
conference name
Advances in Neural Information Processing Systems 35, NeurIPS 2022
conference location
New Oreleans, United States
conference dates
2022-11-28 - 2022-12-09
ISBN
9781713871088
language
English
LU publication?
yes
id
45016f41-f384-4a10-8784-5a2b542f5da9
alternative location
https://openreview.net/pdf?id=e4Wf6112DI
date added to LUP
2022-09-19 09:24:28
date last changed
2023-10-06 09:31:49
@inproceedings{45016f41-f384-4a10-8784-5a2b542f5da9,
  abstract     = {{Recent advances have extended the scope of Bayesian optimization (BO) to expensive-to-evaluate black-box functions with dozens of dimensions, aspiring to unlock impactful applications, for example, in the life sciences, neural architecture search, and robotics. However, a closer examination reveals that the state-of-the-art methods for high-dimensional Bayesian optimization (HDBO) suffer from degrading performance as the number of dimensions increases, or even risk failure if certain unverifiable assumptions are not met. This paper proposes BAxUS that leverages a novel family of nested random subspaces to adapt the space it optimizes over to the problem. This ensures high performance while removing the risk of failure, which we assert via theoretical guarantees. A comprehensive evaluation demonstrates that BAxUS achieves better results than the state-of-the-art methods for a broad set of applications.}},
  author       = {{Papenmeier, Leonard and Nardi, Luigi and Poloczek, Matthias}},
  booktitle    = {{Advances in Neural Information Processing Systems, NeurIPS 2022}},
  isbn         = {{9781713871088}},
  keywords     = {{Bayesian Optimization; Global Optimization; Gaussian Process; high-dimensional}},
  language     = {{eng}},
  publisher    = {{Curran Associates, Inc}},
  title        = {{Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces}},
  url          = {{https://openreview.net/pdf?id=e4Wf6112DI}},
  year         = {{2022}},
}