3 ms·
IIRC, no, it doesn't. It redefines language-model scaling as a function of both training dataset size as well as number of parameters, as opposed to only the la
by cs702 4y ago
IIRC, no, it doesn't. It redefines language-model scaling as a function of both training dataset size as well as number of parameters, as opposed to only the latter. Also, all predictions extrapolated from past experimental results are hypothetical, subject to future experimental verification. We don't yet have a body of theory good enough to trust those extrapolations.