4 ms·
I'm using NNI[1] with decent success for hyperparameter optimization. It implements a number of different approaches, from a simple random search to a Tree Parz
by wongarsu 8y ago
I'm using NNI[1] with decent success for hyperparameter optimization. It implements a number of different approaches, from a simple random search to a Tree Parzen Estimator (TPE) and specialized algorithms for automatically designing networks.
It's very powerful and gives you a lot of freedom (it can minimize/maximize the output of fundamentally any python program). The main drawback is that you are on your own to figure out which paramters go well together: For example using an assesor to stop underperforming attempts early is great for random search, but devastating for TPE. You have to figure that out on your own. You inevitably spend some time tuning your hyperparameter tuner. It's still a big win in terms of human effort, at the expense of doing a lot more computing.
1: https://github.com/Microsoft/nni https://github.com/Microsoft/nni
- MasterScrat 8y agoI'm used to hyperopt, do you know how they compare?
- Zephyr314 8y agohyperopt also uses TPEs [0], this may be a variant/fork of that. [0]: http://hyperopt.github.io/hyperopt/ http://hyperopt.github.io/hyperopt/