Class BayesianOptimization

java.lang.Object

smile.hpo.BayesianOptimization

public class BayesianOptimization
extends Object

Bayesian Hyperparameter Optimization (HPO). Bayesian optimization is a sequential design strategy for global optimization of black-box objective functions that does not require derivative information.

It uses a Gaussian Process (GP) as a probabilistic surrogate model of the objective function. After evaluating initial random configurations, an acquisition function (such as Expected Improvement or Upper Confidence Bound) balances exploration of uncertain regions with exploitation of known promising areas to select the next most informative hyperparameter configuration.

References

1. J. Snoek, H. Larochelle, and R. P. Adams. Practical Bayesian Optimization of Machine Learning Algorithms. NIPS 2012.
2. B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proc. IEEE 2016.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	BayesianOptimization.Acquisition	Acquisition function strategy.
static final record	BayesianOptimization.Options	Configuration options for Bayesian Optimization.
static final record	BayesianOptimization.Result	Result of Bayesian hyperparameter optimization.
static final record	BayesianOptimization.Trial	An individual trial evaluation in the optimization process.

Method Summary

Modifier and Type	Method	Description
static BayesianOptimization.Result	fit(Hyperparameters hp, ToDoubleFunction<Properties> objective, int maxTrials)	Fits/runs Bayesian optimization on the given hyperparameter search space.
static BayesianOptimization.Result	fit(Hyperparameters hp, ToDoubleFunction<Properties> objective, int maxTrials, boolean maximize)	Fits/runs Bayesian optimization on the given hyperparameter search space.
static BayesianOptimization.Result	fit(Hyperparameters hp, ToDoubleFunction<Properties> objective, BayesianOptimization.Options options)	Fits/runs Bayesian optimization on the given hyperparameter search space with full options.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

fit

public static BayesianOptimization.Result fit(Hyperparameters hp,
 ToDoubleFunction<Properties> objective,
 int maxTrials)

Fits/runs Bayesian optimization on the given hyperparameter search space.

Parameters:

hp - the hyperparameter search space.

objective - the objective function to evaluate each configuration.

maxTrials - the total number of trials.

Returns:

the optimization result.

fit

public static BayesianOptimization.Result fit(Hyperparameters hp,
 ToDoubleFunction<Properties> objective,
 int maxTrials,
 boolean maximize)

Fits/runs Bayesian optimization on the given hyperparameter search space.

Parameters:

hp - the hyperparameter search space.

objective - the objective function to evaluate each configuration.

maxTrials - the total number of trials.

maximize - true to maximize the objective function, false to minimize.

Returns:

the optimization result.

fit

public static BayesianOptimization.Result fit(Hyperparameters hp,
 ToDoubleFunction<Properties> objective,
 BayesianOptimization.Options options)

Fits/runs Bayesian optimization on the given hyperparameter search space with full options.

Parameters:

hp - the hyperparameter search space.

objective - the objective function to evaluate each configuration.

options - the optimization options.

Returns:

the optimization result.