3 ms·
Relatedly, optimizer performance can behave very wildly on different types of models. Although many new DL projects use Adam as a baseline, in one of my project
by minimaxir 8y ago
Relatedly, optimizer performance can behave very wildly on different types of models. Although many new DL projects use Adam as a baseline, in one of my projects simple RMSprop performed significantly better after a bit of testing.
The optimizer is just another hyperparameter to tune.