> Transformers are extremely interesting. And this is about the least interesting transformer paper one can imagine in 2020. Because it's not a transformer pap

> Transformers are extremely interesting. And this is about the least interesting transformer paper one can imagine in 2020. Because it's not a transformer paper. This paper goal was to see how far can an increase in compute continue to deliver an increase in model performance. There is no better way to study this than to take a very well known architecture and keep it the same as possible, otherwise it becomes very hard to know what is due to the increase size of the model and what is due to the tweaks you make. So yes, it's a disappointing paper if you expect it to be on a different topic than what it is.