4 ms·
All the curves have been levelling off as expected. Not really sure what you're talking about.
by 0x3f 6mo ago
All the curves have been levelling off as expected. Not really sure what you're talking about.
- solenoid0937 6mo agoThey have not, every successful pre-train as of late has had performance increases greater than what the scaling laws predict.
- 0x3f 6mo agoThose gains are arch based, data quality based, etc. Scaling laws only relate to data volume and compute, holding other factors constant.