Because the published result is almost surely not repeatable in each nuanced case for day to day practitioners that have custom layers, extra regularization, he

Because the published result is almost surely not repeatable in each nuanced case for day to day practitioners that have custom layers, extra regularization, heterogeneous hardware for training, etc. When you’ve seen enough of these same papers come and go, your eyes glaze over. It’s like p-hacking, but for finding tasks that show X% improvement. Basically it needs a huge Your Mileage May Vary sticker attached, because for practitioner’s specific situations, you still have to do the experiments to tune these parameters. Use AdamW or rmsprop or adagrad or ... and use dropout (what fraction?) and use penalty terms, and use early stopping, etc. etc., ... ? You can’t rely on papers like this for much insight beyond just saying AdamW is yet another tool in the toolbox... and that’s all. It would be comparable to someone writing a paper about some slight variation of Timsort that is ~10% faster on some empirical distribution of data, like data already 80% sorted. Is it good for you? You have to do legwork to know. Such a result is perfectly good research, but would be silly for a bunch of blog posts about “state of the art” sorting, and plugs for some software package that has it.