2 ms·
AI Proxy to swap in any LLM while using OpenAI's SDK
- westurner 3y agoHow does this compare to LocalAI? https://github.com/mudler/LocalAI https://github.com/mudler/LocalAI
- Tiberium 3y agoThe AI Proxy from the post is for using multiple local and remote LLMs over the OpenAI API (along with API key management apparently), while LocalAI is only for using local LLMs.
- westurner 3y agopromptfoo and ChainForge do multi-LLM comparisons and benchmarking: https://news.ycombinator.com/item?id=37447885 https://news.ycombinator.com/item?id=37447885
- davidatbt 3y agoThis looks awesome - thanks for sharing! Yeah I think it's just local vs. hosted models.
- Havoc 3y agoBeen meaning to build similar. Specifically with a view towards mixing local LLMs and openAI. Locals are cheaper and more interesting but less powerful. eg some tasks will need gbnf grammar some won’t. Some can get away with 3.5 others need 4 Suspect I’ll end up duplicating OPs efforts though because hosted doesn’t really work for that.
- davidatbt 3y agoSounds interesting - feel free to reach out or discuss on the Braintrust discord :)!