3 ms·

I'm pretty sure models are at their end game already. Now the law of diminishing returns is at play.

• treprinum 2y ago
  Transformers still didn't hit their scaling limits wrt number of tokens trained on and the number of layers. Model size limits are now purely given by finances ($100M/training run still seems to be too excessive).
  • wg0 2y ago
    That is true. And that financial limits might change however.