Yes exactly this!! If Aaron et al are willing to place their optimizer on this leaderboard, that'll be fantastic. And https://dawn.cs.stanford.edu/benchmark/Ima

Yes exactly this!! If Aaron et al are willing to place their optimizer on this leaderboard, that'll be fantastic. And https://dawn.cs.stanford.edu/benchmark/ImageNet/train.html https://dawn.cs.stanford.edu/benchmark/ImageNet/train.html - (ignoring the top 3 due to massively more GPUs) The issue I have with Schedule-Free is you need tuning, but a well tuned SOTA can already skyrocket past a plain tuned AdamW.