4 ms·
LiteLLM doesn't quite live up to its name. With all those features, there is nothing "lite" about it. It is essential for a project to live up to its name. Ima
by OutOfHere 23d ago
LiteLLM doesn't quite live up to its name. With all those features, there is nothing "lite" about it. It is essential for a project to live up to its name.
Imagine Sqlite adding heavy features from Postgresql, e.g. row-level security.
- sv123 23d agoBut imagine Sqlite not supporting joins or window functions... sure they are useful but look how many LOC it adds! Who is the arbiter of what Lite actually means?
- datadrivenangel 23d agoLiteLLM's problem isn't really features, it's how bloated all the features are, and specifically how AI maximalist and janky their dev practices are.
- yujonglee 23d agothanks for the feedback. genuinely curious what you think we could be doing better, especially around our dev practices. Would love to hear specifics.
- deleted 23d ago[deleted]
- OutOfHere 23d agoDidn't yall have a release of a package that stole user keys or such?
- otabdeveloper4 23d agoDon't worry about it. Claude did a thorough security review of their code.
- miki123211 23d agoMaybe this has improved since I last used it, but I was dismayed at not being able to do something like: litellm.register("foo", CustomAIProvider) litellm.do_whatever("foo/my-cool-model", "what is 2+2")
- mpyne 23d agoWe run it at my org and it's never been a noticeable resource hog. It's actually the best performer between it, our AI observability stack and the front end.
- blazarquasar 23d agoIt may not be a huge resource hog, but it adds a ton of latency. https://www.getmaxim.ai/bifrost/resources/benchmarks https://www.getmaxim.ai/bifrost/resources/benchmarks Having ran both LiteLLM and Bifrost for months, I can largely confirm the numbers from those benchmarks for myself.
- mpyne 22d agoIt may, but the latency it contributes to the end-to-end AI processing has been not noticeable in practice for our users. That's not to say Bifrost wouldn't have been better, but the choice to use LiteLLM was arrived at after a fair bit of internal discussion (most of which predated my addition to the team), and so far we've seen nothing from LiteLLM that has been contradictory to the pros/cons they thought would be the case when LiteLLM was adopted. Or in other words, the org will be happy indeed when they have solved so many of the rest of the problems we've had in AI uptake that the difference in latency between one AI gateway or the other becomes a problem to be solved.
- otabdeveloper4 23d agoLiteLLM is vibecoded trash.