From the article: > 200% speed up in training! “Overall, we found Adam to be robust and well-suited to a wide range of non-convex optimization problems in the

From the article: > 200% speed up in training! “Overall, we found Adam to be robust and well-suited to a wide range of non-convex optimization problems in the field machine learning” concluded the paper. Ah yes, those were the days, over three years ago now, a life-time in deep-learning-years. But it started to become clear that all was not as we hoped. Few research articles used it to train their models, new studies began to clearly discourage to apply it and showed on several experiments that plain ole SGD with momentum was performing better. This is not true for all domains. In machine translation Adam is used in all top results papers I can think of.