6 ms·

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are

by dijit 5mo ago

Frontier AI companies are selling at a loss. Excusing everything else that u/bastawhiz said[0]; the obvious fact here is that Claude, OpenAI, Gemini et al. are quite literally burning through 100's of billions of dollars and selling it back to you for pennies on the dollar in the hopes that they get to be the only one left. If I spend $10 growing Oranges and sell them to you for $1; then of course it's more expensive for you to do the growing. I feel like I'm taking crazy pills. These models will become more expensive over time, it's functionally impossible for them not to, they just want to capture the market before they have to stop selling at a huge loss. [0]: https://news.ycombinator.com/item?id=48168433 https://news.ycombinator.com/item?id=48168433

poly2it 5mo ago
Well, I'd be surprised if non-R&D inference providers were selling at a loss. There are a plethora to choose from, competition is quite healthy. Will they keep providing cheap tokens while the labs raise their prices? Probably, but then I don't see how they could be raised in the first place. And what timescale are you talking about? A couple of years? It is appropriate to assume inference will become more efficient over time. If you raise your prices, you are going to be out competed before it's profitable (if you assume it is unprofitable) which would be negligent. I don't see how this makes sense.
vanviegen 5mo ago
That seems unlikely. There are many providers for open models on openrouter. It seems unlikely that they are throwing money away for each token they sell. Also, there a good technical reasons for inference being much more efficient at scale.
- dijit 5mo ago
  The providers on OpenRouter serving open models aren't "throwing money away", agreed. But that's not the point I'm making. (or, it kind of is, but it's more high level than that). They're running spot and preemptible GPU instances (60-80% cheaper than on-demand), paying wholesale industrial electricity rates, and running at multi-tenant utilisation densities that make your MacBook look like a bonfire. Of course they're not individually loss-making on inference, they're aggregating cheap commodity compute and skimming a margin, and on paper that's what makes it seem like a good idea, certainly not a loss leader right? But zoom out a bit; the entire stack is swimming in VC money. OpenRouter itself just raised at a $1.3B valuation backed by a16z. The Chinese models that now account for 36% of all tokens routed through the platform (DeepSeek, Qwen) are priced the way they are because Beijing-adjacent capital has decided market share matters more than margin right now. So yes, technically no single party is "throwing money away" on each token; they're just all simultaneously subsidising different parts of the stack for strategic reasons. The floor price you're seeing isn't a stable equilibrium, it's a pile of investor money that hasn't entirely finished burning yet.
  - vlovich123 5mo ago
    > The floor price you're seeing isn't a stable equilibrium, it's a pile of investor money that hasn't entirely finished burning yet. All that says is that it gets more expensive in the future as competitors exit the market and sustainability becomes important. That’s why Uber and Lyft were so cheap until they killed taxis. One major difference of course is that some models will remain largely good enough and the incremental cost of running will keep dropping to 0 over time since the hardware needed doesn’t get more expensive and is already purchased.
    - dijit 5mo ago
      I think we agree. I only object to taking current prices as if they are perpetual prices.
  - reissbaker 5mo ago
    Most of these folks aren't running on spot/preemptible instances, they're doing 1-2 year reserved rentals. There isn't that much cheap compute floating around and you can be quite profitable on reservations (if you can get them — the compute shortage is real). I think the question in terms of throwing money away isn't the inference layer: it's whether the companies training open models will be able to financially keep doing so. How long will Moonshot keep releasing future Kimi models? I think there's an interesting wedge they're exploring with being basically a base-model-trainer-as-a-service, i.e. selling rights to Fireworks to sell finetuning services to the Cursors of the world, but it's entirely possible it doesn't pan out. That being said, Nvidia seems willing to step up to being the base model trainer of last resort via the Nemotron family of open models, since it helps sell more of their hardware — similar to their investments in the CUDA stack to sell hardware (unsurprisingly, Nemotron is designed to run most efficiently on Nvidia hardware, e.g. native NVFP4). So I suspect there will continue to be a pretty good market here.
- lowbloodsugar 5mo ago
  Sure. And there’s Lyft and Uber and plenty of others. And Grubhub and DoorDash and uber and how many others. And I don’t even fucking remember how many electric fucking scooter companies, I’m practically falling over scooters! I’m sure they aren’t earning market share by selling at a loss either.
  - jolux 5mo ago
    Uber has been profitable since 2023.
    - tngranados 5mo ago
      And it was founded in 2009, 14 years without turning a profit
      - doodlesdev 5mo ago
        Which, by analogy, would mean you could use subsided tokens for over 10 years and then, and only then, actually purchase your own hardware to do inference, with over a decade of technological advancements to compound. I'll invest in inference hardware whenever the economics make sense, not because of my prediction that prices will get higher (also, because smaller models keep getting better and they might just suffice for most of my use cases in a few years time). My biggest worries in terms of cost is in regards to training. Whenever that gets too expensive for Beijing to pay, we won't be getting new SOTA open small models to run on local hardware, which, again, reinforces the decision to use providers for AI inference for now.
        dijit 5mo ago
        > Whenever that gets too expensive for Beijing to pay, we won't be getting new SOTA open small models to run on local hardware, which, again, reinforces the decision to use providers for AI inference for now. Valid takeaway. "Invest" in the companies that release their models, because the alternative is to reward the people who will capture the market and jack up the prices to unreasonable level. Unfortunately I think this is a bit like game theory, people will choose the local optimum despite the long term problems with doing so.
- ares623 5mo ago
  It is hard for the engineer to grasp the lengths VCs will go to to grab market and mind share.
MuffinFlavored 5mo ago
> Frontier AI companies are selling at a loss. How big/deep of a loss? I feel like I read this every day for years that Uber did this same "idiotic, losing" strategy (how it was pitched/discussed) and then one day we woke up and... without much fuss, boom, they were profitable seemingly overnight.
- spzb 5mo ago
  Ed Zitron discusses this as part of his post on AI economics : https://www.wheresyoured.at/ais-economics-dont-make-sense/ https://www.wheresyoured.at/ais-economics-dont-make-sense/
- brianwawok 5mo ago
  Well and uber cut the driver pay in half and doubled the price. They didn’t really find any efficiencies, robo drivers don’t exist yet. Also why I hardly touch them anymore.
  - onesociety2022 5mo ago
    All that tells me is they did find an efficiency. If they didn’t, their driver supply would have dropped. Unlike the taxi business, Uber/Lyft can tap into otherwise dormant supply of drivers who already own a car but aren’t willing to spend all 40-60 hours a week driving a taxi. With Uber/Lyft, they can become part-time drivers (they have flexibility and they can use an asset they already own anyway). Is it worse for the full time taxi drivers who used to have the supply artificially constrained in the old medallion system? Yes, but does it also benefit others who want to do this as a flexible job, zero skills required other than driving, no boss to deal with, no job interviews, etc. Yes!
  - MuffinFlavored 5mo ago
    > Well and uber cut the driver pay in half and doubled the price Devil's advocate: * inflation caused everything to go up to some degree since then * if it was "that bad" as you say, they wouldn't be extremely profitable and have so many users both things can be true? "they cut the driver pay in half and doubled the price" did not lead to the collapse of the business/people to stop using it.
MattRix 5mo ago
The inference is absolutely not sold at a loss, at least not when paying API prices (the subscriptions are less clear). The reason frontier model companies aren’t profitable is because training the models is so costly, not inference.
NicuCalcea 5mo ago
The blog compares the cost of running Gemma4 31b, which on OpenRouter is offered by small no-name inference providers, not by frontier AI companies. It seems like a fair comparison to me.
- pornel 5mo ago
  LLM generation is bottlenecked by RAM bandwidth and latency. You can get almost linear scaling by evaluating more prompts in parallel, because the GPU has nothing to for the relative eternity it takes to read all of the weights from DRAM for every layer for every token. On Apple Silicon you can get 4x-8x more tokens per second if you run more queries in parallel (as long as your inference server supports it, and has enough spare RAM for more KV caches). When inference is done at datacenter scales, when you distribute generation across multiple GPUs and have kernels carefully tuned to specific hardware, the compute vs DRAM bandwidth speed ratio gets absurd like 200:1. That's why everyone gives you batch inference at a steep discount.
- jeffybefffy519 5mo ago
  Interestingly if you look at the cost of Gemma3 (this is 12 months old, but demonstrates the authors point) on Vertex AI versus Gemini 2.5 pro, the cost per million tokens WAS very similar.
ianberdin 5mo ago
Do you have a proof? Anthropic’s CEO said they Are profitable. Same with OpenAI.
- dijit 5mo ago
  Profitable for inference if you completely ignore training costs and that you absolutely must continuously train new models.
  - spzb 5mo ago
    And ignore capital costs, depreciation, user churn etc
  - vlovich123 5mo ago
    Which is where your analogy breaks down and why you think you’re taking crazy pills. Inference is growing and selling the oranges in your analogy. Model building is growing the farm to sell larger, juicier more addicting oranges.
    - dijit 5mo ago
      Are ya fuckin' serious mate? The restaurant next to the mines were profitable up until the moment the mines themselves shut down: one doesn't exist without the other. You can't ringfence inference as "the profitable bit" and then hand-wave away the training. Without continuous training there is no inference product. Claude 3 Opus isn't sitting there making revenue in 2026 - the thing is just deprecated. The moment you stop spending billions on the next model, your "profitable" inference business is on borrowed time until someone else makes it obsolete. Maybe I made a mistake in my analogy... They're not growing a farm and then selling oranges. They're on a treadmill where stopping is death, and the treadmill costs $10bn a year to keep running.
      - vlovich123 5mo ago
        > They're on a treadmill where stopping is death, and the treadmill costs $10bn a year to keep running. You’re literally describing all companies. Google takes about $270bn/year to run. If they stopped spending that they’d die pretty darn quick. It’s also a description of working - unless you’d built up significant savings, if you stopped working you’re also going to die.
ajross 5mo ago
> I feel like I'm taking crazy pills. Why? It's no less crazy than when Uber and Lyft were doing the same thing. Or when the entire tech industry was doing it in the dot com boom. Investment-driven market growth at a loss is like the least surprising thing in all of this. The tech is new and fascinating. The bubble is just another trip through the funhouse.
vlovich123 5mo ago
Except that’s not what the analysis is. They’re spending < $1 to get $1 from you and the other $9 to figure out how to improve the model further and build up products on top of that to turn that $1 spend into $5 in the future. In other words, inference is fairly profitable for them and the rest of the money is spent growing revenue as quickly as possible. Building models is still an expensive line item but the costs for that are going down with time. There is also maybe a “capture the market” mentality but I don’t think that’s necessarily it - the tools and processes are largely fungible and that’s a huge problem. They need to figure out how to make it sticky for “capture the market”, but there’s also a very real “grow as big as possible as quickly as possible to take on Google”; Google has an existential threat here.
EGreg 5mo ago
These models will become more expensive over time, it's functionally impossible for them not to, they just want to capture the market before they have to stop selling at a huge loss. They could have said the same about transistors. People keep inventing new ways to keep the costs down. Just look at the latest Qwen, DeepSeek, BitNet. Interesting tidbit: they’re all open, and as Google said in 2022: they have no moat.
OsrsNeedsf2P 5mo ago
The models have been dropping 10x in price for completing the same tasks, year over year. Even if you think Anthropic is losing money charging 10x more than everyone else for their 400B model, the prices will continue to go down based on model improvement alone
brianwawok 5mo ago
So many more efficiencies possible at scale though. I cannot keep a local model 98% utilized 24/7, at least not with my current workload. A big cloud can. I can’t power my servers with DC, I have this AC to DV conversion nonsense. The list goes on.
- visarga 5mo ago
  Besides fill factor being hard to match, there is also scaling - you can't scale local inference 10x for a spike, but you can with cloud inference.
tempest_ 5mo ago
It is the model training that is dragging them down. If the arms race stopped tomorrow the current price pays for the inference.
- Danox 5mo ago
  But isn’t training models, a forever task like iterating in tech you can never take a day off, adding humans to the equation don’t humans train/teach themselves new skills over a lifetime, and isn’t one of the selling points in the future when selling this AI slop your AI never goes to sleep and can always be trained forever? The AI price for entry as we go on into the future will only increase.
  - asjir 5mo ago
    Just keeping it up to date with competitors is much cheaper, by copying better ones like Qwen did with Claude. Also a bunch of research is trickling into open source / arxiv so catching up should continue becoming cheaper at least as a fraction of training from scratch
  - atq2119 5mo ago
    I agree that training is a forever task, and the current rate of training is probably not sustainable. But all that means is that once the current investment mania ends, the market will most likely find a new equilibrium where continuous training still happens, but at a slower rate that can be sustained by inference revenue.
    - tonfa 5mo ago
      > but at a slower rate that can be sustained by inference revenue. reply also it's possible that the scale of inference needed (e.g. Jevons paradox) keeps growing to the point that training costs can fully be absorbed (since training cost is one off vs. inference that can scale). (I suspect that might be the thinking, I don't know if it will be true, it's also possible that no model will create a moat big enough to attract enough of the inference traffic to make it true). Depending on the chips/architecture used, the off-peak traffic from inference can also subsidize the training costs.
visarga 5mo ago
> Frontier AI companies are selling at a loss. There are huge economies to be had by batching requests and using lots of RAM for MoE (sparse models). You can't achieve that efficiency at batch size 1 on a single node.
- asjir 5mo ago
  Exactly, they put a lot of money into engineering and it does give results
- eikenberry 5mo ago
  Inference isn't the problem.
throwatdem12311 5mo ago
The Michael Scott AI Companies.
rprend 5mo ago
This is not true. API tokens are not sold at a loss, and hardware gets more efficient over time, so serving inference on the same model gets cheaper. LLAMA 3.1 405B parameters was $6/$12/M tokens in 2024, but in 2026 that same model is $3/$3/M tokens. The most intelligent model at a given time is much larger than the previous, which is why token costs for GPT5.5 are higher than 5.4. But you should expect that 2 years from now, serving a GPT5.5 sized model will be cheaper than GPT5.5 today. You should expect it to be even cheaper to get an equally intelligent model 2 years from now, because distillation techniques are effective at reducing the necessary parameter count for the same benchmark scores.
- eikenberry 5mo ago
  So are they going to stop at GPT 5.5? This analysis only seems to be counting inference cost when the majority of the cost, and why they are burning through money, is the training.
Groxx 5mo ago
https://old.reddit.com/r/GithubCopilot/comments/1tbb5bj/github_copilot_has_finally_released_a_preview_of/ https://old.reddit.com/r/GithubCopilot/comments/1tbb5bj/gith... Seems to be on its way! I know of at least one person whose company is looking at a 20x increase, and afaict (from related looking around, nothing concrete tho) business accounts are missing some costs in the calculator so it'll likely be higher.
raincole 5mo ago
You should probably take some stay-on-topic pills, as this article is clearly and unambiguously talking about open weight models (e.g. gemma 4), not the ones allegedly being sold at a loss (Opus, ChatGPT, etc)[0]. [0]: these API are not sold at a loss either, by the way. But it's a nice meme so let's just pretend they are.

by dijit 5mo ago

poly2it 5mo ago

Well, I'd be surprised if non-R&D inference providers were selling at a loss. There are a plethora to choose from, competition is quite healthy. Will they keep providing cheap tokens while the labs raise their prices? Probably, but then I don't see how they could be raised in the first place. And what timescale are you talking about? A couple of years? It is appropriate to assume inference will become more efficient over time. If you raise your prices, you are going to be out competed before it's profitable (if you assume it is unprofitable) which would be negligent. I don't see how this makes sense.

vanviegen 5mo ago

That seems unlikely. There are many providers for open models on openrouter. It seems unlikely that they are throwing money away for each token they sell. Also, there a good technical reasons for inference being much more efficient at scale.

dijit 5mo ago
The providers on OpenRouter serving open models aren't "throwing money away", agreed. But that's not the point I'm making. (or, it kind of is, but it's more high level than that). They're running spot and preemptible GPU instances (60-80% cheaper than on-demand), paying wholesale industrial electricity rates, and running at multi-tenant utilisation densities that make your MacBook look like a bonfire. Of course they're not individually loss-making on inference, they're aggregating cheap commodity compute and skimming a margin, and on paper that's what makes it seem like a good idea, certainly not a loss leader right? But zoom out a bit; the entire stack is swimming in VC money. OpenRouter itself just raised at a $1.3B valuation backed by a16z. The Chinese models that now account for 36% of all tokens routed through the platform (DeepSeek, Qwen) are priced the way they are because Beijing-adjacent capital has decided market share matters more than margin right now. So yes, technically no single party is "throwing money away" on each token; they're just all simultaneously subsidising different parts of the stack for strategic reasons. The floor price you're seeing isn't a stable equilibrium, it's a pile of investor money that hasn't entirely finished burning yet.
- vlovich123 5mo ago
  > The floor price you're seeing isn't a stable equilibrium, it's a pile of investor money that hasn't entirely finished burning yet. All that says is that it gets more expensive in the future as competitors exit the market and sustainability becomes important. That’s why Uber and Lyft were so cheap until they killed taxis. One major difference of course is that some models will remain largely good enough and the incremental cost of running will keep dropping to 0 over time since the hardware needed doesn’t get more expensive and is already purchased.
  - dijit 5mo ago
    I think we agree. I only object to taking current prices as if they are perpetual prices.
- reissbaker 5mo ago
  Most of these folks aren't running on spot/preemptible instances, they're doing 1-2 year reserved rentals. There isn't that much cheap compute floating around and you can be quite profitable on reservations (if you can get them — the compute shortage is real). I think the question in terms of throwing money away isn't the inference layer: it's whether the companies training open models will be able to financially keep doing so. How long will Moonshot keep releasing future Kimi models? I think there's an interesting wedge they're exploring with being basically a base-model-trainer-as-a-service, i.e. selling rights to Fireworks to sell finetuning services to the Cursors of the world, but it's entirely possible it doesn't pan out. That being said, Nvidia seems willing to step up to being the base model trainer of last resort via the Nemotron family of open models, since it helps sell more of their hardware — similar to their investments in the CUDA stack to sell hardware (unsurprisingly, Nemotron is designed to run most efficiently on Nvidia hardware, e.g. native NVFP4). So I suspect there will continue to be a pretty good market here.
lowbloodsugar 5mo ago
Sure. And there’s Lyft and Uber and plenty of others. And Grubhub and DoorDash and uber and how many others. And I don’t even fucking remember how many electric fucking scooter companies, I’m practically falling over scooters! I’m sure they aren’t earning market share by selling at a loss either.
- jolux 5mo ago
  Uber has been profitable since 2023.
  - tngranados 5mo ago
    And it was founded in 2009, 14 years without turning a profit
    - doodlesdev 5mo ago
      Which, by analogy, would mean you could use subsided tokens for over 10 years and then, and only then, actually purchase your own hardware to do inference, with over a decade of technological advancements to compound. I'll invest in inference hardware whenever the economics make sense, not because of my prediction that prices will get higher (also, because smaller models keep getting better and they might just suffice for most of my use cases in a few years time). My biggest worries in terms of cost is in regards to training. Whenever that gets too expensive for Beijing to pay, we won't be getting new SOTA open small models to run on local hardware, which, again, reinforces the decision to use providers for AI inference for now.
      - dijit 5mo ago
        > Whenever that gets too expensive for Beijing to pay, we won't be getting new SOTA open small models to run on local hardware, which, again, reinforces the decision to use providers for AI inference for now. Valid takeaway. "Invest" in the companies that release their models, because the alternative is to reward the people who will capture the market and jack up the prices to unreasonable level. Unfortunately I think this is a bit like game theory, people will choose the local optimum despite the long term problems with doing so.
ares623 5mo ago
It is hard for the engineer to grasp the lengths VCs will go to to grab market and mind share.

MuffinFlavored 5mo ago

> Frontier AI companies are selling at a loss. How big/deep of a loss? I feel like I read this every day for years that Uber did this same "idiotic, losing" strategy (how it was pitched/discussed) and then one day we woke up and... without much fuss, boom, they were profitable seemingly overnight.

spzb 5mo ago
Ed Zitron discusses this as part of his post on AI economics : https://www.wheresyoured.at/ais-economics-dont-make-sense/ https://www.wheresyoured.at/ais-economics-dont-make-sense/
brianwawok 5mo ago
Well and uber cut the driver pay in half and doubled the price. They didn’t really find any efficiencies, robo drivers don’t exist yet. Also why I hardly touch them anymore.
- onesociety2022 5mo ago
  All that tells me is they did find an efficiency. If they didn’t, their driver supply would have dropped. Unlike the taxi business, Uber/Lyft can tap into otherwise dormant supply of drivers who already own a car but aren’t willing to spend all 40-60 hours a week driving a taxi. With Uber/Lyft, they can become part-time drivers (they have flexibility and they can use an asset they already own anyway). Is it worse for the full time taxi drivers who used to have the supply artificially constrained in the old medallion system? Yes, but does it also benefit others who want to do this as a flexible job, zero skills required other than driving, no boss to deal with, no job interviews, etc. Yes!
- MuffinFlavored 5mo ago
  > Well and uber cut the driver pay in half and doubled the price Devil's advocate: * inflation caused everything to go up to some degree since then * if it was "that bad" as you say, they wouldn't be extremely profitable and have so many users both things can be true? "they cut the driver pay in half and doubled the price" did not lead to the collapse of the business/people to stop using it.

MattRix 5mo ago

The inference is absolutely not sold at a loss, at least not when paying API prices (the subscriptions are less clear). The reason frontier model companies aren’t profitable is because training the models is so costly, not inference.

NicuCalcea 5mo ago

The blog compares the cost of running Gemma4 31b, which on OpenRouter is offered by small no-name inference providers, not by frontier AI companies. It seems like a fair comparison to me.

pornel 5mo ago
LLM generation is bottlenecked by RAM bandwidth and latency. You can get almost linear scaling by evaluating more prompts in parallel, because the GPU has nothing to for the relative eternity it takes to read all of the weights from DRAM for every layer for every token. On Apple Silicon you can get 4x-8x more tokens per second if you run more queries in parallel (as long as your inference server supports it, and has enough spare RAM for more KV caches). When inference is done at datacenter scales, when you distribute generation across multiple GPUs and have kernels carefully tuned to specific hardware, the compute vs DRAM bandwidth speed ratio gets absurd like 200:1. That's why everyone gives you batch inference at a steep discount.
jeffybefffy519 5mo ago
Interestingly if you look at the cost of Gemma3 (this is 12 months old, but demonstrates the authors point) on Vertex AI versus Gemini 2.5 pro, the cost per million tokens WAS very similar.

ianberdin 5mo ago

Do you have a proof? Anthropic’s CEO said they Are profitable. Same with OpenAI.

dijit 5mo ago
Profitable for inference if you completely ignore training costs and that you absolutely must continuously train new models.
- spzb 5mo ago
  And ignore capital costs, depreciation, user churn etc
- vlovich123 5mo ago
  Which is where your analogy breaks down and why you think you’re taking crazy pills. Inference is growing and selling the oranges in your analogy. Model building is growing the farm to sell larger, juicier more addicting oranges.
  - dijit 5mo ago
    Are ya fuckin' serious mate? The restaurant next to the mines were profitable up until the moment the mines themselves shut down: one doesn't exist without the other. You can't ringfence inference as "the profitable bit" and then hand-wave away the training. Without continuous training there is no inference product. Claude 3 Opus isn't sitting there making revenue in 2026 - the thing is just deprecated. The moment you stop spending billions on the next model, your "profitable" inference business is on borrowed time until someone else makes it obsolete. Maybe I made a mistake in my analogy... They're not growing a farm and then selling oranges. They're on a treadmill where stopping is death, and the treadmill costs $10bn a year to keep running.
    - vlovich123 5mo ago
      > They're on a treadmill where stopping is death, and the treadmill costs $10bn a year to keep running. You’re literally describing all companies. Google takes about $270bn/year to run. If they stopped spending that they’d die pretty darn quick. It’s also a description of working - unless you’d built up significant savings, if you stopped working you’re also going to die.

ajross 5mo ago

> I feel like I'm taking crazy pills. Why? It's no less crazy than when Uber and Lyft were doing the same thing. Or when the entire tech industry was doing it in the dot com boom. Investment-driven market growth at a loss is like the least surprising thing in all of this. The tech is new and fascinating. The bubble is just another trip through the funhouse.

vlovich123 5mo ago

Except that’s not what the analysis is. They’re spending < $1 to get $1 from you and the other $9 to figure out how to improve the model further and build up products on top of that to turn that $1 spend into $5 in the future. In other words, inference is fairly profitable for them and the rest of the money is spent growing revenue as quickly as possible. Building models is still an expensive line item but the costs for that are going down with time. There is also maybe a “capture the market” mentality but I don’t think that’s necessarily it - the tools and processes are largely fungible and that’s a huge problem. They need to figure out how to make it sticky for “capture the market”, but there’s also a very real “grow as big as possible as quickly as possible to take on Google”; Google has an existential threat here.

EGreg 5mo ago

These models will become more expensive over time, it's functionally impossible for them not to, they just want to capture the market before they have to stop selling at a huge loss. They could have said the same about transistors. People keep inventing new ways to keep the costs down. Just look at the latest Qwen, DeepSeek, BitNet. Interesting tidbit: they’re all open, and as Google said in 2022: they have no moat.

OsrsNeedsf2P 5mo ago

The models have been dropping 10x in price for completing the same tasks, year over year. Even if you think Anthropic is losing money charging 10x more than everyone else for their 400B model, the prices will continue to go down based on model improvement alone

brianwawok 5mo ago

So many more efficiencies possible at scale though. I cannot keep a local model 98% utilized 24/7, at least not with my current workload. A big cloud can. I can’t power my servers with DC, I have this AC to DV conversion nonsense. The list goes on.

visarga 5mo ago
Besides fill factor being hard to match, there is also scaling - you can't scale local inference 10x for a spike, but you can with cloud inference.

tempest_ 5mo ago

It is the model training that is dragging them down. If the arms race stopped tomorrow the current price pays for the inference.

Danox 5mo ago
But isn’t training models, a forever task like iterating in tech you can never take a day off, adding humans to the equation don’t humans train/teach themselves new skills over a lifetime, and isn’t one of the selling points in the future when selling this AI slop your AI never goes to sleep and can always be trained forever? The AI price for entry as we go on into the future will only increase.
- asjir 5mo ago
  Just keeping it up to date with competitors is much cheaper, by copying better ones like Qwen did with Claude. Also a bunch of research is trickling into open source / arxiv so catching up should continue becoming cheaper at least as a fraction of training from scratch
- atq2119 5mo ago
  I agree that training is a forever task, and the current rate of training is probably not sustainable. But all that means is that once the current investment mania ends, the market will most likely find a new equilibrium where continuous training still happens, but at a slower rate that can be sustained by inference revenue.
  - tonfa 5mo ago
    > but at a slower rate that can be sustained by inference revenue. reply also it's possible that the scale of inference needed (e.g. Jevons paradox) keeps growing to the point that training costs can fully be absorbed (since training cost is one off vs. inference that can scale). (I suspect that might be the thinking, I don't know if it will be true, it's also possible that no model will create a moat big enough to attract enough of the inference traffic to make it true). Depending on the chips/architecture used, the off-peak traffic from inference can also subsidize the training costs.

visarga 5mo ago

> Frontier AI companies are selling at a loss. There are huge economies to be had by batching requests and using lots of RAM for MoE (sparse models). You can't achieve that efficiency at batch size 1 on a single node.

asjir 5mo ago
Exactly, they put a lot of money into engineering and it does give results
eikenberry 5mo ago
Inference isn't the problem.

throwatdem12311 5mo ago

The Michael Scott AI Companies.

rprend 5mo ago

This is not true. API tokens are not sold at a loss, and hardware gets more efficient over time, so serving inference on the same model gets cheaper. LLAMA 3.1 405B parameters was $6/$12/M tokens in 2024, but in 2026 that same model is $3/$3/M tokens. The most intelligent model at a given time is much larger than the previous, which is why token costs for GPT5.5 are higher than 5.4. But you should expect that 2 years from now, serving a GPT5.5 sized model will be cheaper than GPT5.5 today. You should expect it to be even cheaper to get an equally intelligent model 2 years from now, because distillation techniques are effective at reducing the necessary parameter count for the same benchmark scores.

eikenberry 5mo ago
So are they going to stop at GPT 5.5? This analysis only seems to be counting inference cost when the majority of the cost, and why they are burning through money, is the training.

Groxx 5mo ago

https://old.reddit.com/r/GithubCopilot/comments/1tbb5bj/github_copilot_has_finally_released_a_preview_of/ https://old.reddit.com/r/GithubCopilot/comments/1tbb5bj/gith... Seems to be on its way! I know of at least one person whose company is looking at a 20x increase, and afaict (from related looking around, nothing concrete tho) business accounts are missing some costs in the calculator so it'll likely be higher.

raincole 5mo ago

You should probably take some stay-on-topic pills, as this article is clearly and unambiguously talking about open weight models (e.g. gemma 4), not the ones allegedly being sold at a loss (Opus, ChatGPT, etc)[0]. [0]: these API are not sold at a loss either, by the way. But it's a nice meme so let's just pretend they are.