3 ms·

> because we've already seen where scaling these mechanisms leads. In the case of GPT-3, scaling seemed to continuously improve results, they just kinda ran ou

by sva_ 4y ago

> because we've already seen where scaling these mechanisms leads. In the case of GPT-3, scaling seemed to continuously improve results, they just kinda ran out of data. Are you implying this must be the same for this model? Or were you intending to say something different that I didn't see?

mach1ne 4y ago
To be precise, scaling improved results logarithmically. Accuracy seems to plateau with scaling and never reach any magical limit of true general intelligence.
- Tenoke 4y ago
  There's been no observed plateu with scaling. As far as we can tell further scaling will keep improving results.