4 ms·

I remember some talk in which the researcher quantitatively said that parameter count is the only important metric for transformers it seems. It doesn't matter

I remember some talk in which the researcher quantitatively said that parameter count is the only important metric for transformers it seems. It doesn't matter if the model is taller or thicker or have more number of attention heads.

ShamelessC 5y ago
That's correct. The original paper is "Scaling Laws for Transformers".