5 ms·

Does anyone know a good alternate project that works similarly (share multipple LLMs across a set of users)? LiteLLM has been getting worse and trying to get me

Does anyone know a good alternate project that works similarly (share multipple LLMs across a set of users)? LiteLLM has been getting worse and trying to get me to upgrade to a paid version. I also had issues with creating tokens for other users etc.

river_otter 7mo ago
github.com/mozilla-ai/any-llm :)
tacoooooooo 7mo ago
pydantic-ai
sschueller 7mo ago
I just found https://github.com/jasmedia/InferXgate https://github.com/jasmedia/InferXgate which looks interesting although quite new and not supporting so many providers.
redrove 7mo ago
Bifrost is the only real alternative I'm aware of https://github.com/maximhq/bifrost https://github.com/maximhq/bifrost
- sschueller 7mo ago
  Virtual Keys is an Enterprise feature. I am not going to pay for something like this in order to provide my family access to all my models. I can do without cost control (although it would be nice) but I need for users to be able to generate a key and us this key to access all the models I provide.
  - NeutralCrane 7mo ago
    I don’t believe it is an enterprise feature. I did some testing on Bifrost just last month on a free open source instance and was able to set up virtual keys.
  - redrove 7mo ago
    I just deployed it to test it out and this is FALSE. I was able to create Virtual Keys on the free version with no issues. Please do a double take on the facts, you might falsely deter people.
- beanaroo 7mo ago
  We have tried reaching out to their sales multiple times but never get a response.
thibault000 7mo ago
[dead]
treefarmer 7mo ago
If you're talking about their proxy offering, I had this exact same issue and switched to Portkey. I just use their free plan and don't care about the logs (I log separately on my own). It's way faster (probably cause their code isn't garbage like the LiteLLM code - they had a 5K+ line Python file with all their important code in it the last time I checked).
howardjohn 7mo ago
agentgateway.dev is one I have been working on that is worth a look if you are using the proxy side of LiteLLM. It's open source part of the Linux foundation.

river_otter 7mo ago

github.com/mozilla-ai/any-llm :)

tacoooooooo 7mo ago

pydantic-ai

sschueller 7mo ago

I just found https://github.com/jasmedia/InferXgate https://github.com/jasmedia/InferXgate which looks interesting although quite new and not supporting so many providers.

redrove 7mo ago

Bifrost is the only real alternative I'm aware of https://github.com/maximhq/bifrost https://github.com/maximhq/bifrost

sschueller 7mo ago
Virtual Keys is an Enterprise feature. I am not going to pay for something like this in order to provide my family access to all my models. I can do without cost control (although it would be nice) but I need for users to be able to generate a key and us this key to access all the models I provide.
- NeutralCrane 7mo ago
  I don’t believe it is an enterprise feature. I did some testing on Bifrost just last month on a free open source instance and was able to set up virtual keys.
- redrove 7mo ago
  I just deployed it to test it out and this is FALSE. I was able to create Virtual Keys on the free version with no issues. Please do a double take on the facts, you might falsely deter people.
beanaroo 7mo ago
We have tried reaching out to their sales multiple times but never get a response.

thibault000 7mo ago

[dead]

treefarmer 7mo ago

If you're talking about their proxy offering, I had this exact same issue and switched to Portkey. I just use their free plan and don't care about the logs (I log separately on my own). It's way faster (probably cause their code isn't garbage like the LiteLLM code - they had a 5K+ line Python file with all their important code in it the last time I checked).

howardjohn 7mo ago

agentgateway.dev is one I have been working on that is worth a look if you are using the proxy side of LiteLLM. It's open source part of the Linux foundation.