3 ms·
What are your thoughts on Sophia, an algorithmic replacement for ADAM which promises to converge in fewer steps? https://arxiv.org/abs/2305.14342 https://arxiv
by josephg 3y ago
What are your thoughts on Sophia, an algorithmic replacement for ADAM which promises to converge in fewer steps?
https://arxiv.org/abs/2305.14342 https://arxiv.org/abs/2305.14342
- tbalsam 3y agoThere are dozens of dozens of optimizers. Almost every single one of them claims superiority. Some of them, like MADGRAD, are great. But sticking with the basics is pretty good in my experience.
- lucidrains 3y agohttps://arxiv.org/abs/2307.06440 https://arxiv.org/abs/2307.06440