3 ms·
Chinchilla paper proposes the following empirical scaling power law Loss= (N0/N)^a+(D0/D)^b Where N is number of parameters, D is number of examples, a,b are
by ivalm 4y ago
Chinchilla paper proposes the following empirical scaling power law
Loss= (N0/N)^a+(D0/D)^b
Where N is number of parameters, D is number of examples, a,b are exponents. N0,D0 are scaling factors.