3 ms·
How do evolutionary hyper-parameter optimizations compare to Bayesian ones? My impression always was that Bayesian optimization is more targeted, and therefore
by fnl 9y ago
How do evolutionary hyper-parameter optimizations compare to Bayesian ones? My impression always was that Bayesian optimization is more targeted, and therefore more efficient and ultimately finds optimal parameters faster. However, evolutionary algorithms are easier to parallelize, so maybe EAs indeed have a place in a non-research-oriented, applied DL setting?
- levesque 9y agoCovariance matrix adaptation is comparable on real-valued hyperparameters, but you're stuck outside of that. More typical genetic algorithms waste a TON of compute time, so they might end up finding a good solution, but the computational budget is out of reach of normal companies.