4 ms·
Huh? There's plenty of open source LLMs. Pythia, GPT-NeoX, GPT-J, GPT-2, BLOOM-176, are ones I can think of off the top of my head. Pythia is the best performin
by ipsum2 4y ago
Huh? There's plenty of open source LLMs. Pythia, GPT-NeoX, GPT-J, GPT-2, BLOOM-176, are ones I can think of off the top of my head. Pythia is the best performing one IIRC.
- bradknowles 4y agoHmm. Would you be able to combine these LLMs? Or are they already supersets of each other?
- MacsHeadroom 4y agoPythia was trained on only 300B tokens and is pretty dumb compared to LLaMA. Pythia 13B is worse than LLaMA-7B and requires double the resources.
- Tepix 4y agoNot all use cases need GPT-4 level performance. I'd argue that even LLaMA-7B is quite limited. Also, new and improved models are being released all the time.
- lhl 4y agoI started keeping a list btw, there are about 20 completely open (Apache 2.0, BSD, MIT, CC-BY) 1B+ parameter foundational LLMs at the moment: https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYpb63e1ZR3aePczz3zlbJW-Y4/edit?usp=sharing https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYp...