3 ms·
Practical Bayesian Optimisation of Machine Learning Hyperparameters [1] was a great recent paper on this topic that looked to have very nice results in practice
by ajtulloch 13y ago
Practical Bayesian Optimisation of Machine Learning Hyperparameters [1] was a great recent paper on this topic that looked to have very nice results in practice across a range of models. Is there any interest in implementing that algorithm in hyperopt?
[1]: http://arxiv.org/pdf/1206.2944.pdf http://arxiv.org/pdf/1206.2944.pdf
- davmre 13y agoFrom the documentation: "Hyperopt has been designed to accommodate Bayesian optimization algorithms based on Gaussian processes and regression trees, but these are not currently implemented." [1] So it sounds like they've at least thought about it. [1]: http://hyperopt.github.io/hyperopt/ http://hyperopt.github.io/hyperopt/
- thisisdave 13y agoFor those that don't know: a Bayesian optimization approach based on Gaussian processes has been implemented in Python by the author of that paper [0]. Here's a nice blog post with some information on some problems it can run into in practice [1]. [0]: https://github.com/JasperSnoek/spearmint https://github.com/JasperSnoek/spearmint [1]: http://fastml.com/tuning-hyperparams-automatically-with-spearmint/ http://fastml.com/tuning-hyperparams-automatically-with-spea...
- jaberg99 13y agoHey, hyperopt author here. I started something of a spearmint binding [1] but I didn't get very far yet. The main trouble is that spearmint currently does not have proper support for parameters that apply "sometimes". There's a paper at the upcoming NIPS workshop on Bayesian optimization with an interesting scheme for doing that, but still very much a research question. [1]: https://github.com/hyperopt/hyperopt-spearmint https://github.com/hyperopt/hyperopt-spearmint