3 ms·
Self-hosting LLMs is expensive at scale. It's cheaper to use VC subsidized model inference like the OpenAI APIs.
by dtquad 2y ago
Self-hosting LLMs is expensive at scale. It's cheaper to use VC subsidized model inference like the OpenAI APIs.
- CyberDildonics 2y agoAt "scale"? At what scale?
- littlestymaar 2y agoThere are plenty of VC-subsidized inference provider which uses open source LLM for much cheaper than OpenAI (which isn't really VC-subsidized at this point but Microsoft-subsidized).
- refulgentis 2y agoMy anecdata is most teams I've talked to say its below OpenAI at scale, and vLLM is a beast. It's interesting to hear the opposite, there's lots of cheaper providers, but the "VC dollars" argument can go "turtles all the way down", I suppose. Still, reality seems to differ.