3 ms·
Introducing the Sophia optimizer, a faster alternative to AdamW
- m00x 3y agoPaper: https://arxiv.org/pdf/2305.14342.pdf https://arxiv.org/pdf/2305.14342.pdf
- alephxyz 3y agoUnofficial implementation: https://github.com/kyegomez/Sophia https://github.com/kyegomez/Sophia Looking forward to trying it out this week
- alephxyz 3y agoIgnore that link. Their implementation doesn't even run and seems to be almosy entirely chatgpt-generated.
- tylergem 3y agoCode: https://github.com/Liuhong99/Sophia https://github.com/Liuhong99/Sophia