Published on Wed Feb 17 2021

BORE: Bayesian Optimization by Density-Ratio Estimation

Louis C. Tiao, Aaron Klein, Matthias Seeger, Edwin V. Bonilla, Cedric Archambeau, Fabio Ramos

Bayesian optimization (BO) is among the most effective and widely-used blackbox optimization methods. We cast the computation of EI as a binary classification problem. By circumventing the tractability constraints, this reformulation provides numerous advantages.

0
0
0
Abstract

Bayesian optimization (BO) is among the most effective and widely-used blackbox optimization methods. BO proposes solutions according to an explore-exploit trade-off criterion encoded in an acquisition function, many of which are computed from the posterior predictive of a probabilistic surrogate model. Prevalent among these is the expected improvement (EI) function. The need to ensure analytical tractability of the predictive often poses limitations that can hinder the efficiency and applicability of BO. In this paper, we cast the computation of EI as a binary classification problem, building on the link between class-probability estimation and density-ratio estimation, and the lesser-known link between density-ratios and EI. By circumventing the tractability constraints, this reformulation provides numerous advantages, not least in terms of expressiveness, versatility, and scalability.

Fri Jun 22 2018
Machine Learning
Bayesian Optimization of Combinatorial Structures
The optimization of expensive-to-evaluate black-box functions is an ubiquitous task in machine learning, engineering and the natural sciences. This article proposes the first algorithm to overcome these challenges. The algorithm is based on an adaptive, scalable model that identifies useful combinatorial structure.
0
0
0
Sun Jul 08 2018
Machine Learning
A Tutorial on Bayesian Optimization
Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression.
3
0
4
Wed Feb 17 2021
Machine Learning
Using Distance Correlation for Efficient Bayesian Optimization
We propose a novel approach for Bayesian optimization. It combines Gaussian processes with distance correlation. It balances exploration and exploitation automatically, and requires no manual tuning.
0
0
0
Fri Mar 30 2012
Machine Learning
A Lipschitz Exploration-Exploitation Scheme for Bayesian Optimization
The problem of optimizing unknown costly-to-evaluate functions has been studied for a long time in the context of Bayesian Optimization. In this paper, we assume the unknown function is Lipschitz continuous. We propose an algorithm with a distinct exploration phase followed by an exploitation phase.
0
0
0
Wed Apr 22 2020
Machine Learning
Bayesian Optimization with Output-Weighted Optimal Sampling
In Bayesian optimization, accounting for the importance of the output is a crucial yet challenging exercise. We advocate the use of the likelihood ratio to guide the search algorithm towards regions of the input space where the objective function to be minimized assumes abnormally small values.
0
0
0
Thu May 27 2021
Machine Learning
Bayesian Optimisation for Constrained Problems
Bayesian optimisation builds a response surface model based on the data collected so far. We propose a novel variant of the well-known Knowledge Gradient acquisition function that allows it to handle constraints.
3
2
2