4 ms·
A very easy simple improvement over random search is to use a low discrepancy sequence. They were designed almost for this purpose (avoiding the problems caused
by murbard2 11y ago
A very easy simple improvement over random search is to use a low discrepancy sequence. They were designed almost for this purpose (avoiding the problems caused by irrelevant dimensions). I don't know why I never see it suggested... it's not as good as gaussian process modeling, but it's very easy to implement and it clearly dominates random search for this application.
- alicez 11y agoGreat idea. Will run some experiments to see how it performs. It sounds analogous to kmeans++ initialization. Sobol sequences ring a bell. Some of the Bayesian optimization software libraries may in fact use a Sobol sequence of initial evaluations. But it may not be well documented.
- doobwa 11y agoSpearmint, for example: https://github.com/JasperSnoek/spearmint/blob/master/spearmint-lite/ExperimentGrid.py#L188 https://github.com/JasperSnoek/spearmint/blob/master/spearmi...