3 ms·
> but if inference is this cheap then why aren't there multiple API providers offering models at dirt cheap prices There are multiple API providers offering mo
by dragonwriter 1y ago
> but if inference is this cheap then why aren't there multiple API providers offering models at dirt cheap prices
There are multiple API providers offering models at dirt cheap prices, enough so that there is at least one well-known API provider that is an aggreggator of other API providers that offers lots of models at $0.
> The only cheap-ass providers I've seen only run tiny models. Where's my cheap deepseek-R1?
https://openrouter.ai/deepseek/deepseek-r1-0528:free https://openrouter.ai/deepseek/deepseek-r1-0528:free
- booi 1y agoyou can also run deepseek for free on a modestly sized laptop
- dragonwriter 1y agoAt 4-bit quant, R1 takes 300+ gigs just for weights. You can certainly run smaller models into which R1 has been distilled on a modest laptop, but I don't see how you can run R1 itself on anything that wouldn't be considered extreme for a laptop in at least one dimension.
- svachalek 1y agoYou're probably thinking of what ollama labels "deepseek" which is not in fact deepseek, but other models with some deepseek distilled into them.
- idiotsecant 1y agoHow is this possible? I imagine someone is finding some value in the prompts themselves but this cant possibly be paying for itself.
- tick_tock_tick 1y agoInference is just that cheap plus they hope that you'll start using the ones they charge for as you become more used to using AI in your workflow.