While a lot of Bayesian optimization methods use GPs (MOE, Spearmint, BayesOpt, etc) some use TPEs as well (most notably, hyperopt [0]), and some ensemble these

While a lot of Bayesian optimization methods use GPs (MOE, Spearmint, BayesOpt, etc) some use TPEs as well (most notably, hyperopt [0]), and some ensemble these methods and others like SigOpt (YC W15) [1] (disclaimer: I'm one of the founders). I tried to briefly go over the functional interpretation of GPs in this talk [2], although the book by Rasmussen and Williams does a much more thorough job [3] (free online, check out chapter 2 for this approach). I'm happy to answer any questions about the differences. If you're a student/academic SigOpt is also completely free [4]. [0]: http://hyperopt.github.io/hyperopt/ http://hyperopt.github.io/hyperopt/ [1]: https://sigopt.com/research/ https://sigopt.com/research/ [2]: https://www.youtube.com/watch?v=J6UcAdH54RE&list=PLbSwfqjMfjGc6Z-WP9VYBCZsHHyOImabQ https://www.youtube.com/watch?v=J6UcAdH54RE&list=PLbSwfqjMfj... [3]: http://www.gaussianprocess.org/gpml/chapters/RW.pdf http://www.gaussianprocess.org/gpml/chapters/RW.pdf [4]: https://sigopt.com/solution/for-academia https://sigopt.com/solution/for-academia