3 ms·
> If token costs converge toward zero for most AI use cases... In the real world, token costs seem to be going up, as early stage pricing at a loss gives way t
by Animats 3mo ago
> If token costs converge toward zero for most AI use cases...
In the real world, token costs seem to be going up, as early stage pricing at a loss gives way to pricing that generates revenue.
Compute costs might go down a little over the next five years, but there's nothing coming along in hardware that leads to huge reductions in price. NVidia says don't expect better price/performance before 2030.
The models keep getting bigger, and people put loops around them which iterate, burning tokens.
Where is this cost reduction coming from?
- fc417fc802 3mo agoI don't expect any of the third party openrouter providers sell tokens at a loss. Agreed that increasing model size could drive token prices up however so far there's been a very strong trend in the opposite direction with smaller models becoming increasingly capable thanks to advances in theory and implementation. Edit: A glaring omission on my part there is that growth of aggregate industry demand for tokens has the potential to outpace increases in supply provided by new datacenters buildouts. So tokens certainly could go up depending on how things play out.
- decimalenough 3mo agoI've switched to non-SOTA models, which deliver comparable value at a fraction of the cost. A full day of coding with Deepseek is approx $1 in tokens, and at least for my use cases the quality is equivalent to Claude.
- profsummergig 3mo agoIf you don't mind sharing... what's your preferred vendor for Deepseek
- fancy_pantser 3mo agoNot who you asked, but if you're looking for a rec, I've been using GLM and Deepseek models through Deepinfra because the prices are decent for US-based deployments and there's acceptable throughput and policy/terms. I'd love to find another that lowers prices over time when they roll out e.g. multi-token prediction instead of just taking a larger margin, but I don't know of any today.
- christophilus 3mo agoHm. I must be holding it wrong. I hit $20 easily in 5 hours with deepseek in opencode.
- p1necone 3mo agoSame here - curious which deepseek model and what op was doing. I suspect it might be a question of conversational loop vs agentic dev - the former uses much less tokens than letting an autonomous agent churn away on your codebase.
- p1necone 3mo agoMy apologies, deepseek v4 flash through pi is indeed super cheap, I can see only using $1 in a day and getting a decent amount done.
- fragmede 3mo agoDo a through audit of every single token you're sending and receiving. Some harnesses include things you didn't expect.
- zurtri 3mo agoThis website https://tokenpriceindex.com/ https://tokenpriceindex.com/ tracks Token Cost. You are right - tokens are going up currently.
- fc417fc802 3mo agoThat chart is a blended metric though. I don't think anyone is using the latest and greatest frontier models (which seem to get more expensive with each generation) for mundane tasks that are solvable with existing models. It seems unreasonable to average deepseek v4 flash and fable pricing.
- eru 3mo agoThe following two things can be true at the same time: - Frontier state-of-the-art performance keeps getting more expensive (and better). - Any fixed performance level is becoming cheaper. (And a third: if you still want to see improvements instead of a fixed level, you can trail the frontier a bit and still see price some reductions over time.)
- Ferret7446 3mo agoDon't index on tokens. Costs for useful unit of work had gone down in my experience, and that's what actually matters. Counting tokens is like counting lines of code.