3 ms·
> Language models and giant training runs So... Its November 2023, and we have a handful of really good open source LLM base models (Mistral, Llama, MPT, Falco
by brucethemoose2 3y ago
> Language models and giant training runs
So... Its November 2023, and we have a handful of really good open source LLM base models (Mistral, Llama, MPT, Falcon, a few code models, a few Chinese ones like Yi, a Arabaic one) and a few notable proprietary ones, and a few cool vision models.
... Where is all that hoovered up accelerator compute going? Shouldnt we be swimming in open and closed LLMs by now?
- Palmik 3y agoJust a guess: (1) A lot of "me too" pre-training that should really be fine tuning; (2) lot of proprietary models whose weights will never be released (or even used :)).