4 ms·
Very interesting. Wondering what is the state of the art in Hyperparameter Optimization at the moment. Does this method apply to all Deep Learning systems?
by jackylupino23 3y ago
Very interesting. Wondering what is the state of the art in Hyperparameter Optimization at the moment. Does this method apply to all Deep Learning systems?
- mlminer 3y agoI think so, they apply it to Computer Vision datasets as well
- rch 3y agoI just got back into hyperopt a couple weeks ago. It's easy enough and worked for me, but I was thinking there had to be some new things I'm not aware of.
- gillesjacobs 3y agoHyperband [1] has been my go-to hyperparam optimization method over the past few years. Handily beats Bayesian search wherever I applied it, also implemented in most frameworks. 1. https://arxiv.org/abs/1603.06560 https://arxiv.org/abs/1603.06560
- Lindizz 3y agoThe work compares against Hyperband and the new method is significantly better (Figure 2, Hypothesis 2).
- blackbear_ 3y agoFor a general overview, this could be a good starting point [1]. As for deep learning, you may wanna start from here [2], but I personally had good results with Hyperband [3] for DL. [1] https://wires.onlinelibrary.wiley.com/doi/full/10.1002/widm.1484 https://wires.onlinelibrary.wiley.com/doi/full/10.1002/widm.... [2] https://github.com/google-research/tuning_playbook https://github.com/google-research/tuning_playbook [3] https://jmlr.csail.mit.edu/papers/v18/16-558.html https://jmlr.csail.mit.edu/papers/v18/16-558.html