3 ms·
Lvl 3 is supposed to support other models and providers in the future too. I hope it will support every server with simple, standard API so I can run self-hoste
by terramex 2y ago
Lvl 3 is supposed to support other models and providers in the future too. I hope it will support every server with simple, standard API so I can run self-hosted LLama 3 (or whatever will be released in next 6-12 months).
- hmottestad 2y agoOr Groq. They can do 1250 tokens/s with Llama 3 8B.