4 ms·
Coincidentally this comes after MosaicML released the best open source commercially usable LLMs on huggingface: mpt-30b, the first open source LLM with 8k conte
by underlines 3y ago
Coincidentally this comes after MosaicML released the best open source commercially usable LLMs on huggingface: mpt-30b, the first open source LLM with 8k context length that can be extended even further with ALiBi and has been trained on a whopping 1 trillion tokens vs. 300 billion for Pythia and OpenLLaMA, and 800 billion for StableLM.
- jncraton 3y agoOpenLLaMA models up to 13B parameters have now been trained on 1T tokens: https://github.com/openlm-research/open_llama https://github.com/openlm-research/open_llama
- underlines 3y agounfortunately not openllama-33b yet
- emadm 3y ago20b done
- brucethemoose2 3y agoIs it better than Falcon 40B?
- swyx 3y agodo we know how it ranks vs the other models yet? its not yet up on https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...