3 ms·
Do you want to see scaling curves wrt data and param size? I agree that 1.2B and 10B tokens is not representative, but what scale of parameters and dataset siz
by ketchup32613 4mo ago
Do you want to see scaling curves wrt data and param size? I agree that 1.2B and 10B tokens is not representative, but what scale of parameters and dataset sizes would be convincing?
- zxexz 4mo agoNot to sound facetious, but perhaps enough runs at different param/token sizings to define a curve?