This looks great! The authors are reputable and claim to have discovered a simple lr optimizer than trains models up to 5x faster and converges to better local

This looks great! The authors are reputable and claim to have discovered a simple lr optimizer than trains models up to 5x faster and converges to better local optima than Adam/AdamW. As usual, YMMV. You can find a PyTorch implementation here: https://github.com/lucidrains/lion-pytorch https://github.com/lucidrains/lion-pytorch Preliminary tests seem to confirm the paper's claims.