2 ms·
Yep, but if stability has the goal of training the best possible model then that would explain the choices they made.
by whimsicalism 4y ago
Yep, but if stability has the goal of training the best possible model then that would explain the choices they made.
- GaggiX 4y agoI mean 800B tokens on a 3B model and 7B model is still way beyond the Chinchilla scale.
- MacsHeadroom 4y agoThey're going to 1.5T and possibly 3T. The 800B is just for the "Alpha" checkpoints released today. New checkpoints will be released later.