3 ms·

First off, thanks for your efforts! From what I've followed of machine learning, I feel like there are areas that are not very rigorous, for example where huma

First off, thanks for your efforts! From what I've followed of machine learning, I feel like there are areas that are not very rigorous, for example where humans have to tune weights and parameters. Does your work address those places? Specifically do you think you can train on enough parameters, and compare enough results, to begin to estimate how that tuning can be done automatically?

duvenaud 11y ago
Yes, gradient-based hyperparameter optimization is one option for taking the human out of the loop to some extent. Of course, we can't avoid having to specify hyper-hyper-parameters! More generally, automatic hyperparameter tuning (without gradients) is already a standard part of many researchers' pipelines. It's also part of good scientific practice, since it makes it harder to bias the results towards a particular method.