We showed this in our original scaling law paper: https://arxiv.org/pdf/1712.00409.pdf https://arxiv.org/pdf/1712.00409.pdf The slope of the power law is dete

We showed this in our original scaling law paper: https://arxiv.org/pdf/1712.00409.pdf https://arxiv.org/pdf/1712.00409.pdf The slope of the power law is determined by the problem and dataset. Compute, parameter count, and data move you along the curve. Change in architecture/bias is a constant offset. So architecture can give an advantage, but that advantage can be overcome by scale.