There are lots of tools to help with the hyperparameter problem to make that faster/cheaper as well. This problem is often orthogonal to the domain expertise re

There are lots of tools to help with the hyperparameter problem to make that faster/cheaper as well. This problem is often orthogonal to the domain expertise required to do good feature selection. Scikit-learn implements things like grid search natively [0], and tools like SigOpt [1] (YC W15, disclosure: I'm a founder) do this automatically as well. [0]: http://scikit-learn.org/stable/modules/generated/sklearn.grid_search.GridSearchCV.html http://scikit-learn.org/stable/modules/generated/sklearn.gri... [1]: https://github.com/sigopt/sigopt_sklearn https://github.com/sigopt/sigopt_sklearn