3 ms·
Both are tricky subject and lot of cooking-like thinking (there are popular recipes, general rules, but in all practical cases to master it there is a lot of no
by stared 9y ago
Both are tricky subject and lot of cooking-like thinking (there are popular recipes, general rules, but in all practical cases to master it there is a lot of non-transferable experience and knowledge).
Some pointers:
* http://rishy.github.io/ml/2017/01/05/how-to-train-your-dnn/ http://rishy.github.io/ml/2017/01/05/how-to-train-your-dnn/
* http://www.alexirpan.com/2017/06/27/hyperparam-spectral.html http://www.alexirpan.com/2017/06/27/hyperparam-spectral.html
At deepsense.io we are developing Neptune to facilitate that process (compare models, version-control them), here: https://go.neptune.deepsense.io https://go.neptune.deepsense.io (a public release in less that a month!).
- putnam 9y agoAnother resource - https://sigopt.com/ https://sigopt.com/, in the vein of Whetlab. e.g. For NLP: * https://aws.amazon.com/blogs/ai/fast-cnn-tuning-with-aws-gpu-instances-and-sigopt/ https://aws.amazon.com/blogs/ai/fast-cnn-tuning-with-aws-gpu... Disclaimer: they did share free trials with some of our students.
- stared 9y agoIt looks interesting, and talk with these guys (including the author) on a conference (GPU Tech Conf in San Jose). However, for other much more transparent systems, I am kind of anxious to add a black box. Especially as most of neural network tweaking go well beyond hyperparameter tweaking - one need to adjust architecture, make sensible cross-validation, good augmentation, etc.