3 ms·
Not only the it seems to be AI generated, it seems these guys don't even know about best practices or even what works. e.g. It contains archaic comparison of op
by YetAnotherNick 2y ago
Not only the it seems to be AI generated, it seems these guys don't even know about best practices or even what works. e.g. It contains archaic comparison of optimizers and its pros and cons, but for LLMs no optimizer other than Adam and new ones like Lion works.
- abc-1 2y agoIs there a paper on this? Why do no other optimizers give good results? Adam requires insane amounts of memory so alternatives would be welcome.