3 ms·
I mean 800B tokens on a 3B model and 7B model is still way beyond the Chinchilla scale.
by GaggiX 4y ago
I mean 800B tokens on a 3B model and 7B model is still way beyond the Chinchilla scale.
- MacsHeadroom 4y agoThey're going to 1.5T and possibly 3T. The 800B is just for the "Alpha" checkpoints released today. New checkpoints will be released later.