48 ms·

They are not worse - the results are not repeatable. The problem is much worse. Like with cab hailing, shopping, social media ads, food delivery, etc: there wi

They are not worse - the results are not repeatable. The problem is much worse. Like with cab hailing, shopping, social media ads, food delivery, etc: there will be a whole ecosystem, workflows, and companies built around this. Then the prices will start going up with nowhere to run. Their pricing models are simply not sustainable. I hope everyone realizes that the current LLMs are subsidized, like your Seamless and Uber was in the early days.

Kuinox 10mo ago
It would mean that inference is not profitable. Calculating inference costs show it's profitable, or close to.
- renegade-otter 10mo ago
  Inference costs have in fact been crashing, going from astronomical to... lower. That said, I am not sure that this indicator alone tells the whole story, if not hides it - sort of like EBITDA.
  - Kuinox 10mo ago
    I think there will still be cheap inference, what will rise in costs will be frontier model subscriptions. This is the thing that is not profitable.
chiengineer 10mo ago
They just need to figure out KV cache turned into a magic black box after that it'll be fine
snarf21 10mo ago
I'm not sure. I asked one about a potential bug in iOS 26 yesterday and it told me that iOS 26 does not exist and that I must have meant iOS 16. iOS 26 was announced last June and has been live since September. Of course, I responded that 26 is the current iOS version is 26 and got the obligatory meme of "Of course, you are right! ramble ramble ramble...."
- doug_durham 10mo ago
  Sure. You have to be mindful of the training cut off date for the model. By default models won't search the web and rely on data baked into their internal model. That said the ergonomics of this is horrible and a huge time waste. If I run into this situation I just say "Search the web".
  - realharo 10mo ago
    That will only work as long as there is an active "the web" to search. Unless the models get smart enough to figure out the answer from scratch.
  - bluGill 10mo ago
    If the traning cutoff is before iOS 26 then the correct answer is 'i don't know anything about it, but it is reasonable to think it will exist soon'. saying 'of course you are right' is a lie
    - deleted 10mo ago
      [deleted]
- jerezzprime 10mo ago
  Let's imagine a scenario. For your entire life, you have been taught to respond to people in a very specific way. Someone will ask you a question via email and you must respond with two or three paragraphs of useful information. Sometimes when the person asks you a question, they give you books that you can use, sometimes they don't. Now someone sends you an email and asks you to help them fix a bug in Windows 12. What would you tell them?
  - mock-possum 10mo ago
    I mean I would want to tell them that windows 11 is the most recent version of windows… but also I’d check real quick to make sure windows 12 hadn’t actually come out without me noticing.
IMTDb 10mo ago
A key difference is that the cost to execute a cab ride largely stayed the same. Gas to get you from point A to point B is ~$5, and there's a floor on what you can pay the driver. If your ride costs $8 today, you know that's unsustainable; it'll eventually climb to $10 or $12. But inference costs are dropping dramatically over time, and that trend shows no signs of slowing. So even if a task costs $8 today thanks to VC subsidies, I can be reasonably confident that the same task will cost $8 or less without subsidies in the not-too-distant future. Of course, by then we'll have much more capable models. So if you want SOTA, you might see the jump to $10-12. But that's a different value proposition entirely: you're getting significantly more for your money, not just paying more for the same thing.
- SecretDreams 10mo ago
  > But inference costs are dropping dramatically over time, and that trend shows no signs of slowing. So even if a task costs $8 today thanks to VC subsidies, I can be reasonably confident that the same task will cost $8 or less without subsidies in the not-too-distant future. I'd like to see this statement plotted against current trends in hardware prices ISO performance. Ram, for example, is not meaningfully better than it was 2 years ago, and yet is 3x the price. I fail to see how costs can drop while valuations for all major hardware vendors continue to go up. I don't think the markets would price companies in this way if the thought all major hardware vendors were going to see margins shrink a la commodity like you've implied.
  - PaulHoule 10mo ago
    It's not the hardware getting cheaper, it's that LLMs were developed when we really didn't understand how they worked, and there is still some room to improve the implementations, particularly do more with less RAM... And that's everything from doing more with fewer weights to things like FP16, not to mention if you can 2x the speed you can get twice as much done with the same RAM and all the other parts.
    - SecretDreams 10mo ago
      Improvements in LLM efficiency should be driving hardware to get cheaper. I agree with everything you've said, I'm just not seeing any material benefit to the statement as of now.
oceanplexian 10mo ago
> Their pricing models are simply not sustainable. I hope everyone realizes that the current LLMs are subsidized, like your Seamless and Uber was in the early days. If you run these models at home it's easy to see how this is totally untrue. You can build a pretty competent machine that will run Kimi or Deepseek for $10-20k and generate an unlimited amount of tokens all day long (I did a budget version with an Epyc machine for about $4k). Amortize that over a couple years, and it's cheaper than most people spend on a car payment. The pricing is sustainable, and that's ignoring the fact that these big model providers are operating on economies of scale, they're able to parallelize the GPUs and pack in requests much more efficiently.
- lelanthran 10mo ago
  > Amortize that over a couple years, and it's cheaper than most people spend on a car payment. I'm not parsing that: do you mean that the monthly cost of running your own 24x7 is less than the monthly cost of a car payment? Whether true or false, I don't get how that is relevant to proving either that the current LLMs are not subsidised, or proving that they are.
  - franktankbank 10mo ago
    If true it means there's a lower bound that is profitable at least taking into account current apparent purchasing costs and energy consumption.
- deleted 10mo ago
  [deleted]
- Unit327 10mo ago
  Ignores the cost of model training, R&D, managing the data centers and more. OpenAI etc regularly admit that all their products lose money. Not to mention the fact that it isn't enough to cover their costs, they have to pay back all those investors while actually generating a profit at some point in the future.
- Denzel 10mo ago
  Uhm, you actually just proved their point if you run the numbers. For simplicity’s sake we’ll assume DeepSeek 671B on 2 RTX 5090 running at 2 kW full utilization. In 3 years you’ve paid $30k total: $20k for system + $10k in electric @ $0.20/kWh The model generates 500M-1B tokens total over 3 years @ 5-10 tokens/sec. Understand that’s total throughput for reasoning and output tokens. You’re paying $30-$60/Mtok - more than both Opus 4.5 and GPT-5.2, for less performance and less features. And like the other commenters point out, this doesn’t even factor in the extra DC costs when scaling it up for consumers, nor the costs to train the model. Of course, you can play around with parameters of the cost model, but this serves to illustrate it’s not so clear cut whether the current AI service providers are profitable or not.
featherless 10mo ago
Except most of those services don't have at-home equivalents that you can increasingly run on your own hardware.
- oceanplexian 10mo ago
  I run models with Claude Code (Using the Anthropic API feature of llama.cpp) on my own hardware and it works every bit as well as Claude worked literally 12 months ago. If you don't believe me and don't want to mess around with used server hardware you can walk into an Apple Store today, pick up a Mac Studio and do it yourself.
  - Eggpants 10mo ago
    I’ve been doing the same with GPT-OSS-120B and have been impressed. Only gotcha is Claude code expects a 200k context window while that model max supports 130k or so. I have to do a /compress when it gets close. I’ll have to see if there is a way to set the max context window in CC. Been pretty happy with the results so far as long as I keep the tasks small and self contained.
    - petesergeant 10mo ago
      I've been making use of gpt-oss-120b extensively for a range of projects, commercial and price, because providers on OpenRouter make it essentially free and instant, and it's roughly as capable at o4-mini was in my experience. That said, I'm a little surprised to hear you're having great success with it as a coding agent. It's "obviously" worse than the frontier models, and even they can making blindly dumb decisions pretty regularly. Maybe I should give it a shot.
  - icedchai 10mo ago
    Whats your preferred local model?
- deleted 10mo ago
  [deleted]
makach 10mo ago
AI is built to be non-deterministic. Variation is built into each response. If it wasn't I would expect AI to have died out years ago. The pricing and quality on the copilot, codex (which I am experienced in) feels like it is getting worse, but I suspect it may be my expectations are getting higher as the technology is maturing...
startupsfail 10mo ago
The results are repeatable. Models are performing with predictable error rates on the tasks that these models had been trained and tested.
brightball 10mo ago
I've been explaining that to people for a bit now as well as a strong caution for how people are pricing tools. It's all going to go up once dependency is established. The AWS price increase on 1/5 for GPU's on EC2 was a good example.
- renegade-otter 10mo ago
  AWS in general is a good example. It used to be much more affordable and better than boutique hosting. Now AWS costs can easily spiral out of control. Somehow I can run a site for $20 on Digital Ocean, but with AWS it always ends up $120. RDS is a particular racket that will cost you hundreds of dollars for a rock bottom tier. Again, Digital Ocean is below $20 per month that will serve many a small business. And yet, AWS is the default goto at this point because the lockin is real.
  - xienze 10mo ago
    > RDS is a particular racket that will cost you hundreds of dollars for a rock bottom tier. Again, Digital Ocean is below $20 per month that will serve many a small business. And yet, AWS is the default goto at this point because the lockin is real. This is a little disingenuous though. Yeah you can run a database server on DO cheaper than using RDS, but you’ll have to roll all that stuff that RDS does yourself: automatic backups/restores, tuning, monitoring, failover, etc. etc. I’m confident that the engineers who’ve set up those RDS servers and the associated plumbing/automation have done a far better job of all that stuff than I ever could unless I spent a lot of time and effort on it. That’s worth a premium.
threethirtytwo 10mo ago
The pricing will go down once the hardware prices go down. Historically hardware prices always go down. Once the hardware prices go low enough pricing will go down to the point where it doesn't even make sense to sell current LLMs as a service. I would imagine that it's possible that if ever the aforementioned future comes to pass that there will be new forms of ultra high tier compute running other types of AI more powerful than an LLM? But I'm pretty sure AI at it's current state will one day be running locally on desktops and/or handhelds with the former being more likely.
- _puk 10mo ago
  Hopefully we'll get some real focus on making LLMs work amazingly well with limited hardware.. the knock on effect of that would be amazing when the hardware eventually drops in price.
- notTooFarGone 10mo ago
  Are Hardware prices going down when the next generations get less and less better?
  - threethirtytwo 10mo ago
    Yeah it’s not just a demand side thing. Costs go down as well. Every leap in new hardware costs a lot in initial investment and that’s included in a lot of the pricing.
scuff3d 10mo ago
We're building a house on sand. Eventually the whole damn thing is going to come crashing down.
turtletontine 10mo ago
On the bright side, I do think at some point after the bubble pops, we’ll have high quality open source models that you can run locally. Most other tech company business plans follow the enshittification cycle [1], but the interchangeability of LLMs makes it hard to imagine they can be monopolized in the same way. 1: I mean this in the strict sense of Cory Doctorow’s theory (https://en.wikipedia.org/wiki/Enshittification?wprov=sfti1#History_and_definition https://en.wikipedia.org/wiki/Enshittification?wprov=sfti1#H...)
wvenable 10mo ago
> I hope everyone realizes that the current LLMs are subsidized Hell ya, get in and get out before the real pricing comes in.
- Terr_ 10mo ago
  "I'm telling ya kid, the value of nostalgia can only go up! This is your chance to get in on the ground-floor so you can tell people about how things used to be so much better..."
Night_Thastus 10mo ago
Yep. The goal is to build huge amounts of hype and demand, get their hooks into everyone, and once they've killed off any competition and built up the walls then they crank up the price. The prices now are completely unsustainable. They'd go broke if it weren't for investors dumping their pockets out. People forget that what we have now only exists because of absurd amounts of spending on R+D, mountains of dev salaries, huge data centers, etc. That cannot go on forever.
ssss11 10mo ago
Wait for the ads
DamnInteresting 10mo ago
> I hope everyone realizes that the current LLMs are subsidized, like your Seamless and Uber was in the early days. A.I. == Artificially Inexpensive
djeastm 10mo ago
>I hope everyone realizes that the current LLMs are subsidized This is why I'm using it now as much as possible to build as much as possible in the hopes of earning enough to afford the later costs :D
adam_patarino 9mo ago
This is why we HAVE to have a local option and why we're building cortex.build. It's based on a small language model we trained exlusively for coding. We combine it with tools and a context graph designed to be more consistent than what's available today, especially with large codebases.

Kuinox 10mo ago

It would mean that inference is not profitable. Calculating inference costs show it's profitable, or close to.

renegade-otter 10mo ago
Inference costs have in fact been crashing, going from astronomical to... lower. That said, I am not sure that this indicator alone tells the whole story, if not hides it - sort of like EBITDA.
- Kuinox 10mo ago
  I think there will still be cheap inference, what will rise in costs will be frontier model subscriptions. This is the thing that is not profitable.

chiengineer 10mo ago

They just need to figure out KV cache turned into a magic black box after that it'll be fine

snarf21 10mo ago

I'm not sure. I asked one about a potential bug in iOS 26 yesterday and it told me that iOS 26 does not exist and that I must have meant iOS 16. iOS 26 was announced last June and has been live since September. Of course, I responded that 26 is the current iOS version is 26 and got the obligatory meme of "Of course, you are right! ramble ramble ramble...."

doug_durham 10mo ago
Sure. You have to be mindful of the training cut off date for the model. By default models won't search the web and rely on data baked into their internal model. That said the ergonomics of this is horrible and a huge time waste. If I run into this situation I just say "Search the web".
- realharo 10mo ago
  That will only work as long as there is an active "the web" to search. Unless the models get smart enough to figure out the answer from scratch.
- bluGill 10mo ago
  If the traning cutoff is before iOS 26 then the correct answer is 'i don't know anything about it, but it is reasonable to think it will exist soon'. saying 'of course you are right' is a lie
  - deleted 10mo ago
    [deleted]
jerezzprime 10mo ago
Let's imagine a scenario. For your entire life, you have been taught to respond to people in a very specific way. Someone will ask you a question via email and you must respond with two or three paragraphs of useful information. Sometimes when the person asks you a question, they give you books that you can use, sometimes they don't. Now someone sends you an email and asks you to help them fix a bug in Windows 12. What would you tell them?
- mock-possum 10mo ago
  I mean I would want to tell them that windows 11 is the most recent version of windows… but also I’d check real quick to make sure windows 12 hadn’t actually come out without me noticing.

IMTDb 10mo ago

A key difference is that the cost to execute a cab ride largely stayed the same. Gas to get you from point A to point B is ~$5, and there's a floor on what you can pay the driver. If your ride costs $8 today, you know that's unsustainable; it'll eventually climb to $10 or $12. But inference costs are dropping dramatically over time, and that trend shows no signs of slowing. So even if a task costs $8 today thanks to VC subsidies, I can be reasonably confident that the same task will cost $8 or less without subsidies in the not-too-distant future. Of course, by then we'll have much more capable models. So if you want SOTA, you might see the jump to $10-12. But that's a different value proposition entirely: you're getting significantly more for your money, not just paying more for the same thing.

SecretDreams 10mo ago
> But inference costs are dropping dramatically over time, and that trend shows no signs of slowing. So even if a task costs $8 today thanks to VC subsidies, I can be reasonably confident that the same task will cost $8 or less without subsidies in the not-too-distant future. I'd like to see this statement plotted against current trends in hardware prices ISO performance. Ram, for example, is not meaningfully better than it was 2 years ago, and yet is 3x the price. I fail to see how costs can drop while valuations for all major hardware vendors continue to go up. I don't think the markets would price companies in this way if the thought all major hardware vendors were going to see margins shrink a la commodity like you've implied.
- PaulHoule 10mo ago
  It's not the hardware getting cheaper, it's that LLMs were developed when we really didn't understand how they worked, and there is still some room to improve the implementations, particularly do more with less RAM... And that's everything from doing more with fewer weights to things like FP16, not to mention if you can 2x the speed you can get twice as much done with the same RAM and all the other parts.
  - SecretDreams 10mo ago
    Improvements in LLM efficiency should be driving hardware to get cheaper. I agree with everything you've said, I'm just not seeing any material benefit to the statement as of now.

oceanplexian 10mo ago

> Their pricing models are simply not sustainable. I hope everyone realizes that the current LLMs are subsidized, like your Seamless and Uber was in the early days. If you run these models at home it's easy to see how this is totally untrue. You can build a pretty competent machine that will run Kimi or Deepseek for $10-20k and generate an unlimited amount of tokens all day long (I did a budget version with an Epyc machine for about $4k). Amortize that over a couple years, and it's cheaper than most people spend on a car payment. The pricing is sustainable, and that's ignoring the fact that these big model providers are operating on economies of scale, they're able to parallelize the GPUs and pack in requests much more efficiently.

lelanthran 10mo ago
> Amortize that over a couple years, and it's cheaper than most people spend on a car payment. I'm not parsing that: do you mean that the monthly cost of running your own 24x7 is less than the monthly cost of a car payment? Whether true or false, I don't get how that is relevant to proving either that the current LLMs are not subsidised, or proving that they are.
- franktankbank 10mo ago
  If true it means there's a lower bound that is profitable at least taking into account current apparent purchasing costs and energy consumption.
deleted 10mo ago
[deleted]
Unit327 10mo ago
Ignores the cost of model training, R&D, managing the data centers and more. OpenAI etc regularly admit that all their products lose money. Not to mention the fact that it isn't enough to cover their costs, they have to pay back all those investors while actually generating a profit at some point in the future.
Denzel 10mo ago
Uhm, you actually just proved their point if you run the numbers. For simplicity’s sake we’ll assume DeepSeek 671B on 2 RTX 5090 running at 2 kW full utilization. In 3 years you’ve paid $30k total: $20k for system + $10k in electric @ $0.20/kWh The model generates 500M-1B tokens total over 3 years @ 5-10 tokens/sec. Understand that’s total throughput for reasoning and output tokens. You’re paying $30-$60/Mtok - more than both Opus 4.5 and GPT-5.2, for less performance and less features. And like the other commenters point out, this doesn’t even factor in the extra DC costs when scaling it up for consumers, nor the costs to train the model. Of course, you can play around with parameters of the cost model, but this serves to illustrate it’s not so clear cut whether the current AI service providers are profitable or not.

featherless 10mo ago

Except most of those services don't have at-home equivalents that you can increasingly run on your own hardware.

oceanplexian 10mo ago
I run models with Claude Code (Using the Anthropic API feature of llama.cpp) on my own hardware and it works every bit as well as Claude worked literally 12 months ago. If you don't believe me and don't want to mess around with used server hardware you can walk into an Apple Store today, pick up a Mac Studio and do it yourself.
- Eggpants 10mo ago
  I’ve been doing the same with GPT-OSS-120B and have been impressed. Only gotcha is Claude code expects a 200k context window while that model max supports 130k or so. I have to do a /compress when it gets close. I’ll have to see if there is a way to set the max context window in CC. Been pretty happy with the results so far as long as I keep the tasks small and self contained.
  - petesergeant 10mo ago
    I've been making use of gpt-oss-120b extensively for a range of projects, commercial and price, because providers on OpenRouter make it essentially free and instant, and it's roughly as capable at o4-mini was in my experience. That said, I'm a little surprised to hear you're having great success with it as a coding agent. It's "obviously" worse than the frontier models, and even they can making blindly dumb decisions pretty regularly. Maybe I should give it a shot.
- icedchai 10mo ago
  Whats your preferred local model?
deleted 10mo ago
[deleted]

makach 10mo ago

AI is built to be non-deterministic. Variation is built into each response. If it wasn't I would expect AI to have died out years ago. The pricing and quality on the copilot, codex (which I am experienced in) feels like it is getting worse, but I suspect it may be my expectations are getting higher as the technology is maturing...

startupsfail 10mo ago

The results are repeatable. Models are performing with predictable error rates on the tasks that these models had been trained and tested.

brightball 10mo ago

I've been explaining that to people for a bit now as well as a strong caution for how people are pricing tools. It's all going to go up once dependency is established. The AWS price increase on 1/5 for GPU's on EC2 was a good example.

renegade-otter 10mo ago
AWS in general is a good example. It used to be much more affordable and better than boutique hosting. Now AWS costs can easily spiral out of control. Somehow I can run a site for $20 on Digital Ocean, but with AWS it always ends up $120. RDS is a particular racket that will cost you hundreds of dollars for a rock bottom tier. Again, Digital Ocean is below $20 per month that will serve many a small business. And yet, AWS is the default goto at this point because the lockin is real.
- xienze 10mo ago
  > RDS is a particular racket that will cost you hundreds of dollars for a rock bottom tier. Again, Digital Ocean is below $20 per month that will serve many a small business. And yet, AWS is the default goto at this point because the lockin is real. This is a little disingenuous though. Yeah you can run a database server on DO cheaper than using RDS, but you’ll have to roll all that stuff that RDS does yourself: automatic backups/restores, tuning, monitoring, failover, etc. etc. I’m confident that the engineers who’ve set up those RDS servers and the associated plumbing/automation have done a far better job of all that stuff than I ever could unless I spent a lot of time and effort on it. That’s worth a premium.

threethirtytwo 10mo ago

The pricing will go down once the hardware prices go down. Historically hardware prices always go down. Once the hardware prices go low enough pricing will go down to the point where it doesn't even make sense to sell current LLMs as a service. I would imagine that it's possible that if ever the aforementioned future comes to pass that there will be new forms of ultra high tier compute running other types of AI more powerful than an LLM? But I'm pretty sure AI at it's current state will one day be running locally on desktops and/or handhelds with the former being more likely.

_puk 10mo ago
Hopefully we'll get some real focus on making LLMs work amazingly well with limited hardware.. the knock on effect of that would be amazing when the hardware eventually drops in price.
notTooFarGone 10mo ago
Are Hardware prices going down when the next generations get less and less better?
- threethirtytwo 10mo ago
  Yeah it’s not just a demand side thing. Costs go down as well. Every leap in new hardware costs a lot in initial investment and that’s included in a lot of the pricing.

scuff3d 10mo ago

We're building a house on sand. Eventually the whole damn thing is going to come crashing down.

turtletontine 10mo ago

On the bright side, I do think at some point after the bubble pops, we’ll have high quality open source models that you can run locally. Most other tech company business plans follow the enshittification cycle [1], but the interchangeability of LLMs makes it hard to imagine they can be monopolized in the same way. 1: I mean this in the strict sense of Cory Doctorow’s theory (https://en.wikipedia.org/wiki/Enshittification?wprov=sfti1#History_and_definition https://en.wikipedia.org/wiki/Enshittification?wprov=sfti1#H...)

wvenable 10mo ago

> I hope everyone realizes that the current LLMs are subsidized Hell ya, get in and get out before the real pricing comes in.

Terr_ 10mo ago
"I'm telling ya kid, the value of nostalgia can only go up! This is your chance to get in on the ground-floor so you can tell people about how things used to be so much better..."

Night_Thastus 10mo ago

Yep. The goal is to build huge amounts of hype and demand, get their hooks into everyone, and once they've killed off any competition and built up the walls then they crank up the price. The prices now are completely unsustainable. They'd go broke if it weren't for investors dumping their pockets out. People forget that what we have now only exists because of absurd amounts of spending on R+D, mountains of dev salaries, huge data centers, etc. That cannot go on forever.

ssss11 10mo ago

Wait for the ads

DamnInteresting 10mo ago

> I hope everyone realizes that the current LLMs are subsidized, like your Seamless and Uber was in the early days. A.I. == Artificially Inexpensive

djeastm 10mo ago

>I hope everyone realizes that the current LLMs are subsidized This is why I'm using it now as much as possible to build as much as possible in the hopes of earning enough to afford the later costs :D

adam_patarino 9mo ago

This is why we HAVE to have a local option and why we're building cortex.build. It's based on a small language model we trained exlusively for coding. We combine it with tools and a context graph designed to be more consistent than what's available today, especially with large codebases.