3 ms·
Interesting to see that there’s both Google Cloud and Stability AI mentioned in providing the compute. Did Stability pay the bill for the resources used for tra
by 0xferruccio 3y ago
Interesting to see that there’s both Google Cloud and Stability AI mentioned in providing the compute. Did Stability pay the bill for the resources used for training?
- emadm 3y agoWe (Stability AI) trained it on our TPUs with input from the OpenLM team as an OpenLLaMA collaboration. The 20b model is 780b tokens in, lots of learnings so we can optimise future runs. Hopefully these will be useful bases for continued research, we will have some SFT/RLHF variants in due course from our Carper AI lab.