We (Stability AI) trained it on our TPUs with input from the OpenLM team as an OpenLLaMA collaboration. The 20b model is 780b tokens in, lots of learnings so w

We (Stability AI) trained it on our TPUs with input from the OpenLM team as an OpenLLaMA collaboration. The 20b model is 780b tokens in, lots of learnings so we can optimise future runs. Hopefully these will be useful bases for continued research, we will have some SFT/RLHF variants in due course from our Carper AI lab.