For things like the number of layers, the total number of options is relatively small -- usually people try between 1 and 6-7. For most of the other parameters

For things like the number of layers, the total number of options is relatively small -- usually people try between 1 and 6-7. For most of the other parameters you have to be smarter than that, especially since a lot of them are real-valued, so you can't really explore them all. One of the trends these days is to perform automatic hyper-parameter tuning, especially for cases where a full exploration of hyper-parameters via grid-search would mean a combinatorial explosion of possibilities (and for neural networks you can conceivably explore dozen of hyper-parameters). A friend of mine just got a paper published at NIPS (same conference) on using Bayesian optimization/Gaussian processes for optimizing the hyper-parameters of a model -- http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips.pdf http://www.dmi.usherb.ca/~larocheh/publications/gpopt_nips.p.... They get better than state of the art results on a couple of benchmarks, which is neat. The code is public -- http://www.cs.toronto.edu/~jasper/software.html http://www.cs.toronto.edu/~jasper/software.html -- and in python, so you could potentially try it out (runs on EC2, too). Btw, Geoff Hinton is teaching an introductory neural nets class on Coursera these days, you should check it out, he's a great teacher. Also, you can always come back to Google, we're doing cool stuff with this :)