22 ms·
> I noted that my own token usage comes to about $1,000/month against each of Anthropic and OpenAI - which currently costs me just $100 per provider thanks to t
by ValentineC 5mo ago
> I noted that my own token usage comes to about $1,000/month against each of Anthropic and OpenAI - which currently costs me just $100 per provider thanks to their generous subsidized plans for individual subscribers.
Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China?
Many lower-budget individuals are now moving to China open weight models like DeepSeek. I wonder if China's really subsidising the providers, or if inferencing costs are actually much lower, and Anthropic/OpenAI are just making sure no money's left on the table for their eventual IPOs.
- testdelacc1 5mo agoPer token costs will fall, but the harnesses will get more token hungry. Instead of just centering the div it’ll spin up a battery of agents to architect, critique, advise, code, review, refactor and so on.
- sevenzero 5mo agoI wish I could disable most of these. I already hate all the "oh you're actually right, let me fix that" nonsense. Then it proceeds to burn 50k tokens on the git history instead of copying logic A from a different part of the codebase to logic B, where I want that exact logic without having to write the boilerplate myself...
- apsurd 5mo agoMakes me think of how my Claude.md files specifies to use the built in framework code-generators (rails). Those generators are deterministically right every time. I wonder how often the Agent actually follows the guidance. I do see them follow it when I look. But it doesn't seem so every time.
- thefunnyman 5mo agoThis is tricky since it can and will ignore your md directions. When possible I try to lean on tool call hooks or skills that invoke deterministic scripts. As much as you can remove the "choice" the better though still there's a lot of randomness in how reliably it invokes skills ime.
- internet101010 5mo agoHooks are incredibly underused by most people and are the easiest way to establish a first line of defense against bad behavior. Things like blocking tool calls that will read .env file or execute "create or replace table".
- apsurd 5mo agoim implementing this now. thanks. the guides specify the exact intention of more determinism.
- sfn42 5mo agoA lot of the time if you're copying code from one place to another what you actually want to do is abstract it so you can reuse it in both places. The LLM can easily do this type of stuff, just tell it and it'll happily do it. This is exactly what I mean when I tell people they need to work closer with the AI, tell it how to do things. Don't just tell it what to do and get frustrated when it does it differently than you would. A good way to achieve this without writing huge prompts is tell it to plan the change first. Just give it some vague low-effort directions. It'll usually get most things right, you tell it what you want different and once you're happy you tell it to go ahead.
- sevenzero 5mo agoNah the codebase is legacy fucked and I cant be bothered to try and optimize business flows without the fear of other stuff breaking. Claude 100% of the time even thinks we use laravel despite the project being some old lumen codebase, so most of laravels features are not available. It also gets the PHP version we are using wrong 100% of the time.
- sfn42 5mo agoHave you tried adding this information to claude.md so it knows? I also think your excuse is bad. "The code is legacy fucked so I'll just legacy fuck it some more because I can't be bothered to make an effort"
- adithyassekhar 5mo agoThis is a spicy take, unless the business is willing to face some down time, and I am hired to do exactly what you said, I’d never touch any line of code unless I absolutely have to. Different environments don’t help as much. We tend to obsess over software quality when it’s the least important thing for a business. It’s just a means to an end.
- sevenzero 5mo agoThis is what its about, we have multiple ecom shops running 24/7 and cant simply afford downtime or a change of business flow that maybe doesnt affect shop A and B but definitely affects shop C and D...
- KaiShips 5mo ago[flagged]
- SecretDreams 5mo ago> Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? I genuinely do not know how prices can get lower from the current major providers in NA without the whole market collapsing. Everyone is spending copious amounts of money to presumably make more money back.
- aDyslecticCrow 5mo agoAn inference only platform selling good open weight model inference without the research overhead could capture a-lot of market for lower size model uses (haiky, gemeni flash). Diffusion-transformers and clever cashing can drop inference even lower, which is improving at a high rate. The biggest reason large models are un-attainable for local applications is the lack hardware with large amount of unified/graphics memory (and the cost of the platforms that do). Once the memory slog goes back to normal and hardware manufacturers adapt to demand, we may see consumer hardware with large memory capacity effectively opening the door for slow but usable frontier model inference (assuming improvements in model efficiency and compute capacity) At that point, inference becomes a race to the bottom. The large labs hope they can attain a leap in capability (which is increasingly looking bleak, with a average catch-up of just a few months) or market dominance through integration (integration in platforms and OS, exclusive deals with companies or governments). For coding agents, i suspect no player will manage lock in enough market to enforce pricing much higher than the true inference cost, and catering to programmers becomes an unsustainable proposition. We will instead be further hit with a lot of AI integrated into our other tooling costs, such as GitHub, Microsoft suite, G-suite, forcing in AI functions as a value-ad into the total cost without giving the option to exclude them. (using their market position)
- SecretDreams 5mo agoI agree with all of this. So my question remains the same: How are the players investing 100s of billions in buildout going to hope to make this back? Market capture looks bleak, inference looks like a race to the bottom. End users look like they could be beneficiaries. Where do the big boys go?
- cyanydeez 5mo agoid be amazed any american business will aend data to china
- linkregister 5mo agoHuggingFace offers DeepSeek as one of its models— it's pretty simple to spin up instances under your control. I'm not sure about OpenRouter but I wouldn't be surprised if they offer a US-based provider of DeepSeek. For reference, Cursor has their first own light fork of Kimi that they use as their baseline coding and review model.
- dghlsakjg 5mo agoThe majority of Deepseek providers on OpenRouter for v4 pro are in the US. Especially interesting is that they are in the same ballpark for pricing.
- eikenberry 5mo agoThey are in the same ballpark for deepseek-v4-flash, but deepseek-v4-pro from deepseek is still around 1/2 of the alternatives.
- dghlsakjg 5mo agoI'm pretty sure that Deepseek said that pricing was promotional. Be curious to see if it lasts. V3 pricing from them was right in line with what the commodity providers are charging.
- eikenberry 5mo agoThey announced a few weeks back that the promotional pricing was permanent.
- alpinisme 5mo ago“Any” is a very high bar Unless laws prevent it, I don’t see why a substantial minority wouldn’t buy services from where they can get them at a similar quality and much lower price.
- freediddy 5mo agoMost sane US companies will disallow use of cloud-based Chinese AI providers, because everything including code, data, PII, etc is being sent to them.
- cheeze 5mo agoDeepseek has some models in Bedrock. There is definitely a huge market for a "good enough" model running within the country of the company
- KronisLV 5mo ago> Deepseek has some models in Bedrock. Just looked into it, seems like at most they have just 3.2, not 4: https://aws.amazon.com/bedrock/pricing/ https://aws.amazon.com/bedrock/pricing/ Looking around their catalogue more, most of their models seem quite outdated, aside from the OpenAI and Anthropic ones (but those get more expensive). I wouldn't willingly pick Bedrock and would instead throw money at OpenRouter, that has both a bunch of providers, as well as almost any model for you to try.
- upcoming-sesame 5mo agoAzure AI Foundry has Deepseek 4
- ceejayoz 5mo agoSaner companies ask the same question about models from their own country too.
- amunozo 5mo agoYou can run DeepSeek as it's open weights, unlike Claude or GPT.
- rd 5mo agoI wonder if I could start a US-based company with good data regulation and just serve open-weight models at a competitive price. I feel like the real barrier is just that most companies willing to adopt AI usage enough to make it worth it at this point don't want to be using inferior models.
- dgellow 5mo agoOne aspect Paul Kedrosky mentioned recently is the concept of „duration mismatch“. The price per token goes down over time (either because the AI vendor reduces due to competition pressure, or because customers are now incentivized to use older cheaper models). But datacenters are financed through debt, with the assumption their revenue increases over time. Quoting him: „[AI vendors are] paying for a fixed cost with a depreciating commodity“[0]. So you have on one end the token revenue trending down, on the other end the training cost going up for the next frontier models, and you need to pay back your 10y debt. 0: https://youtu.be/wGZboZcSGDY?is=64GuKyqBh_4aSjTE https://youtu.be/wGZboZcSGDY?is=64GuKyqBh_4aSjTE
- bijowo1676 5mo agodo GPU chips really depreciate physically? There are no moving parts, I dont think memory chips or GPU chips deteriorate naturally. I think its only accounting depreciation. I have been using my laptop for a decade, what is stopping datacenters from using the purchased GPU chips for a decade?
- dgellow 5mo agoGPU do depreciate indeed, but here the depreciating commodity is the token, not the hardware. You sell cheaper token with the same hardware
- xyzsparetimexyz 5mo agoWhen everything is said and done it'll be datacenters in American competing with ones in China that have several times lower electricity prices. Token prices will drop to a level that will be unprofitable for American data centers and they will need to close. Thats the main issue here.
- dgellow 5mo agoYep, that’s one source of pressure, for sure
- 5mo ago
- Animats 5mo agoRaise them, more likely. NVidia says that GPU hardware prices won't decrease until at least 2030. The world is out of fab capacity.
- EA-3167 5mo agoSeriously, they’re trying to justify trillion+ IPO’s while setting piles of money on fire, prices aren’t going DOWN.
- criddell 5mo agoToday's frontier models will be tomorrows low-end option. I think whatever model you are using today will be less expensive to use a year or two from now.
- missedthecue 5mo agoLast year's o3 was more expensive than 5.5 is. Whatever model we are using now is probably be more expensive than next year's leading models will be.
- dakolli 5mo agoThey aren't going down, but in the meantime they'll cover their ass by bribing their way into the S&P 500 and then use your 60 year old mother's 401k and teacher's pension to fund their risky capital expenditure.
- 5mo ago
- mcmcmc 5mo ago[dead]
- satvikpendem 5mo agoDon't worry, they'll just lobby to ban Chinese models instead to keep their token revenues high. > Compounding the problem, labs in China often release dual-use capable models as open-weight. Once a model is open-weight, safeguards that do exist can be removed, making the model available to any state or non-state actor to use for malicious purposes, including the cyber and CBRN misuse those safeguards were built to prevent. https://www.anthropic.com/research/2028-ai-leadership https://www.anthropic.com/research/2028-ai-leadership
- CuriouslyC 5mo agoIf you do the math, they don't have a choice. If China captures America's AI market it'll cause a major depression. They'll give it the BYD treatment, though it'll be a lot less effective.
- WarmWash 5mo agoThey'll ban them because (unless run locally or self-hosted) they are just data capture tools for the China.
- dakolli 5mo agoPlease explain to me how that works. If I download gguf file and run inference with it, how is it collecting and sending data back to China? This makes no sense, 99% of the people using Chinese models are using them via Western inference providers who are running them and serving them to people over openrouter or whatever. If anyone is stealing your data it would be an American or European inference provider. A model has no ability to send data anywhere. China bad by default, right?
- satvikpendem 5mo ago> unless run locally or self-hosted
- deleted 5mo ago[deleted]
- vidarh 5mo agoWe can tell that the inferencing costs for many of these models are low enough that these models are being sold close to real costs on the basis that many of them are open weight and available from third party providers who have no incentive to subsidize them. I think the frontier labs will need to drop their high per-token prices at least for their low and mid-level models for the reason that several Chinese models (at least Qwen, DeepSeek, Kimi and GLM) are "close enough" that with the right harness they are cost effective alternatives. They won't necessarily need to close the gap - at least not yet -, because these models won't necessarily compete at the same token counts. E.g. at least some of them need to do far more work to solve the same problems. But, yeah, the prices will come down one way or the other. At the same time, even the subscriptions for the cheap Chinese models are probably subsidised, and those subscriptions are likely to get less generous over time.
- White_Wolf 5mo agoI really doubt Deepseek is subsidised. It's roughly the same price everywhere you look. Deepseek is using the Huawei hardware (as far as I managed to understand from various articles) and hence the savings.
- xyzsparetimexyz 5mo agoAnd Chinese electricity prices are some of the lowest
- dubcanada 5mo agoDon't know why people keep parroting this, this is incorrect. Chinese electricity prices are equal or slightly cheaper then most of North America. But significant pockets such as those around the Quebec or other hydro plants are significantly cheaper then Chinese power pricing. Not only that, China may subsidize AI, but so does the US.
- xyzsparetimexyz 5mo agoOkay interesting. I presume that China also has low cost areas too no? Their grid at least seems more stable. Datacenter construction is more likely to raise prices in the US than there.
- LastTrain 5mo ago> Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? Raise, they are going to raise the prices. We will spend more on AI infrastructure in 2026 and 2027 than the gross sales of the entire global software and services sector. Current pricing is at a major loss for current providers.
- bigbuppo 5mo agoThey're going to need to bring in a few trillion dollars fast to meet wall street expectations. Expect prices to rise.
- ed_elliott_asc 5mo agoIf Anthropic are then they are making a big mistake, their token hungry Claude code is far too greedy
- vinzenzu 5mo agoAPI prices of Anthropic, OpenAI, and Google are massively inflated. https://martinalderson.com/posts/no-it-doesnt-cost-anthropic-5k-per-claude-code-user/ https://martinalderson.com/posts/no-it-doesnt-cost-anthropic... There's no way that all AI inference providers are colluding and/or all running at a massive loss, meaning the cheap Chinese model prices must be the real cost it takes to run frontier-class models PLUS their margin. Look at Deepseek 4 Pro. https://openrouter.ai/deepseek/deepseek-v4-pro/providers https://openrouter.ai/deepseek/deepseek-v4-pro/providers Deepseek and Baidu are subsidising prices but they probably train on inputs. I have no model training and ZDR in OpenRouter enabled, and the first provider that shows up there is Deepinfra, significantly more expensive than Deepseek. BUT much cheaper than Sonnet 4.6 and ChatGPT GPT-5.4.
- PunchyHamster 5mo ago> Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? Are they even making money off them now ?
- xyzsparetimexyz 5mo agoWhy would I even pay for deepseek? I get deepseek v4 flash for free with opencode. If I somehow run out of tokens for the day, I can just then on my vpn
- hanzeweiasa 5mo ago[flagged]
- gizajob 5mo agoI really don’t get it - why not put a Mac Studio with 128gb of ram on every engineers desk and be like “engineer, engineer your local LLM”. Makes no sense to be spending $20-30,000+ per year on cloud providers when Qwen et al are available. And even less sense to be sending all your company code and data to Anthropic and OpenAI when you can keep all that IP in the building.
- fulafel 5mo agoThe Mac is very feeble compared to the big iron that the providers run so will be much lower performance. Also many companies would prefer engineers work on the domain problems instead of working on novel LLMs.
- gizajob 5mo agoI meant “roll your own” LLM for use not build new ones.
- bigyabai 4mo agoThe Mac Studio (and DGX Spark, for that matter) aren't running SOTA-level models by a large margin. Time is money, and waiting on these half-baked solutions is a waste of them both. Especially concerning the Mac Studio, the GPU is far too weak for enterprise-scale context prefill. You'd need 2 or 4 Studios to process 250k contexts quickly, and even then you'd get bottlenecked by the relatively slow memory bandwidth during the decode stage. It is simply terrible hardware for quick or power efficient inference.
- throwaw12 5mo agobecause local models which can run well using 128gb ram are still not SOTA, yes Qwen is amazing, but nor Qwen 27B neither 35B can outperform Opus 4.6, so why increase rework for your engineers even more, if you can pay slightly more and always use SOTA, until others figure out best practices for running local SOTA's
- nektro 5mo ago
- jeron 5mo agopersonal take? token prices are a race to the bottom, as long as open source models remain competitive. that's why OSS is so important