I disagree. If as a researcher you come up with a new kind of model that (1) performs better than small-scale transformers given the same computational budget,

I disagree. If as a researcher you come up with a new kind of model that (1) performs better than small-scale transformers given the same computational budget, and (2) improves its performance as you increase the computation budget (model size), quite a few people will be curious to see how your model would perform at greater and greater scale. BTW, I like the clever username, Al-Khwarizmi: https://en.wikipedia.org/wiki/Muhammad_ibn_Musa_al-Khwarizmi https://en.wikipedia.org/wiki/Muhammad_ibn_Musa_al-Khwarizmi