3 ms·
Devs confirmed that the small ones use 800B, 1.5T is for the large ones
by Taek 4y ago
Devs confirmed that the small ones use 800B, 1.5T is for the large ones
- GaggiX 4y ago@thunderbird120 asked a Stability employee and say that the plan is going to keep training the models up to 1.5T. So I don't know where do you read this.
- nickthegreek 4y agohttps://github.com/Stability-AI/StableLM#stablelm-alpha https://github.com/Stability-AI/StableLM#stablelm-alpha shows that the 3b and 7B had 800b training tokens.
- Taek 4y agoThat may be, but the weights you can download today were trained on 800B
- GaggiX 4y agoyes of course that's why they use "will be trained" on the GH repo.
- sroussey 4y agoI think they are “checkpoint” models in this case. Will be fun to compare when completed!
- oehtXRwMkIs 4y agoAre not all models checkpoints? I think you may be interpreting it too colloquially.