7 ms·
Too bad GPT3.5 Turbo is dirt cheap. Open source models are substantially more expensive when you factor in operating costs. There is no mature ecosystem where y
by anonylizard 3y ago
Too bad GPT3.5 Turbo is dirt cheap. Open source models are substantially more expensive when you factor in operating costs. There is no mature ecosystem where you can just plug in a model and spin up a robust infrastructure to run a local LLM at scale, aka you need infrastructure/ML engineers to do it, aka extremely expensive unless you are using LLMs at extremely large scales.
- dcastm 3y agoI think we'll start seeing a lot more services like https://www.together.ai https://www.together.ai soon. Having open-weight models better than gpt-3.5 will drive a lot of competition on the LLM infra.
- happycube 3y agoMistral's endpoint for mistral-small is slightly cheaper.
- Der_Einzige 3y agoThe additional control/features (support for grammars, constraints, fine-tuning, etc) far o/w the cost savings.