9 ms·

LLM providers on the cusp of an 'extinction' phase as capex realities bite

88 points by abawany 2y ago

siscia 2y ago
Found funny that for something that is pretty much a commodity at this point, adoption seems to be the most important metrics. Yes, there are differences between the models, and yes some may work better. But picking the model at this point is just picking the cheapest option. For most use cases any model will do.
- timewizard 2y ago
  > and yes some may work better. Isn't that where the cost lies? Data, annotation, and model generation all have mostly linear responses to changes in spending. > For most use cases any model will do. They'll operate. They will not produce reliable results. Adoption is one metric, but intentional avoidance should be another.
- PeterStuer 2y ago
  That is not my experience at all. Models are still leapfrogging each other every month in e.g. coding or research capability, or even in more mundane tasks such as summerizing long multi topic texts. Depending on which side of the issue you fall, you're hoping this will go on for a long time to come, or praying that it will end asap. I'm not using the cheapest in neither my own support, nor in my production systems.
  - nitwit005 2y ago
    There are a ton of users that just want help generating emails, or adding some stock photo-like images to a blog post. If the choice is between something that costs $10 a month or $20 a month, and both solve those use cases, it's rational to pick the cheap one.
    - Ekaros 2y ago
      And there will probably also be choice of something free and something trial... Which mean even less money spend.
  - senectus1 2y ago
    from what I've seen the "leapfrogging" is very very incremental. They all seems to racing to the plateau... It doesn't look like there will ever be a "stand out" leader and the product that each company presents to the market appears to be essentially the same product that everyone else presents to the market. Maybe with some slight twist to it that is easily recruitable or exceedable within a few months. This is the issue really. at some point the investors are all going to realize that non of their investments are going to be market leaders. When they get to that stage the bubble will well and truly pop.
    - raducu 2y ago
      > They all seems to racing to the plateau. To me it feels there is no plateau and the models are already very useful and impactful. I believe there is no plateau because there is nothing objectively special or magical about the human mind and it all can and will be eventually solved, one hack at a time.
  - xiphias2 2y ago
    There seems to be some part of the LLM capabilities being lost by hacking some benchmarks. Claude 3.7 is a great example of a model clearly beating 3.5 in all benchmarks, but slowly destroying my code base by adding lots of extra lines or hacking around my instructions (adding ,,if'' statements when I want it to change the code to handle a case instead of understanding what change is really needed to be done for it). I still prefer o1 pro and a lot of those leapfrogging in benchmarks don't translate to being smarter anymore.
- acchow 2y ago
  How do you compose the cheapest models to create a software engineer?
  - dickersnoodle 2y ago
    You don't have to create a real software engineer, you just have to create one that looks close enough to get some executive his bonus and won't fall over before he's moved on to another company...
- alexdoesstuff 2y ago
  Full agree! Being close to the edge of AI usage, it's important to realize that most AI use cases are not "fully autonomous AI software engineer" or "deep research into a niche topic" but way more innocuous: Improve my blog post, what's the capital of France, what are some nice tourist sites to see around my next vacation destination. For those non-edge use cases, costs are an issue, but so are inertia and switching costs. A big reason OpenAI and ChatGPT are so huge is that it's still their go-to model for all of these non-edge use cases as it's well known, well adopted, and quite frankly very efficiently priced.
- Yizahi 2y ago
  Adoption is critical for these LLM corporations, because unlike in other industries, here free tier users incur almost the same costs as the paid tier users. They really can't degrade free tier experience too much, or their customers will flee to the competitors. I've read one guy calculating expenses of these corpos and they are truly insane by now and are constantly rising.
throwup238 2y ago
> As the global tech research company forecasts worldwide generative AI (GenAI) spending will reach $644 billion in 2025, up around 76 percent from 2024 I’m having a hard time squaring the number $644 billion and the phrase “extinction phase.” I don’t believe their actual estimate of GenAI spending but if it’s even in the same ballpark as the real value, that’s not an extinction.
- somenameforme 2y ago
  That's the entire point. Bubbles are caused when future valuations drive excessive spending far beyond a reasonable valuation of something. Then at some point reality hits and it turns into a game of hot potato as people don't want to left holding the bag. Pushing towards a trillion bucks a year for what LLMs are mostly currently used for does not seem like a sustainable system.
  - pjc50 2y ago
    Where are they going to get the trillions in revenue to pay any of that back? That's 10% of 2023 US total wages and salaries. Do people really believe it'll replace that much labour?
    - esseph 2y ago
      VC demands their valuation multipliers.
- alexdoesstuff 2y ago
  Reading through the source [1] they basically get to that huuuuge number by including AI-enabled devices such as phones that have some AI functionality even if not core to their value proposition. That's basically reclassifying a big chunk of smartphones, TVs, and other consumer tech as GenAI spending. Of the "real" categories, they expect: Service 27bn (+162% y/y) Software 37bn (+93% y/y) Servers 180bn (+33% y/y) for a total of $245bn (+58% y/y) That's not shabby numbers, but way more reasonable. Hyperscaler total capex [2] is expected to be around $330bn in 2025 (up +32% y/y) so that'll most likely include a good chunk of the server spend. [1] https://www.gartner.com/en/newsroom/press-releases/2025-03-31-gartner-forecasts-worldwide-genai-spending-to-reach-644-billion-in-2025 https://www.gartner.com/en/newsroom/press-releases/2025-03-3... [2] https://www.marvin-labs.com/blog/deepseek-impact-of-high-quality-low-cost-inference-on-ai-landscape/ https://www.marvin-labs.com/blog/deepseek-impact-of-high-qua...
PeterStuer 2y ago
It took Amazon around six to seven years to see its first profitable quarter, and they still went into the red sometimes when doing major investments thereafter.
- paulluuk 2y ago
  I could be wrong, but I believe Amazon's business model was very simple: do everything as cheaply as possible, run at a loss until all competition is dead, and then raise prices once we dominate the market. I don't think OpenAI has that option.
  - dukeyukey 2y ago
    I don't think that's true, even in rural areas without choices Amazon is pretty cheap.
  - karmakurtisaani 2y ago
    Also, I think Amazon invested everything into growth, for which there was a lot of potential. Seems different for the AI companies.
  - delecti 2y ago
    Eh, it's more that per-transaction they were profitable, so they needed to scale enough to make up for their fixed costs. OpenAI's fixed costs are enormous, and they're only bringing in pennies from subscriptions.
  - rongrobert 2y ago
    This wasn't the business model. The business model was you could sell books over the internet at a much cheaper cost compared to Barnes and Noble or Borders because they weren't paying for physical locations and there was no sales tax on the transactions because it was on the internet.
- jasode 2y ago
  >It took Amazon around six to seven years to see its first profitable quarter, A key difference from OpenAI is that Amazon was cash flow positive from very early on and before that first profitable quarter. They only needed one funding round Series A of $8 million instead of repeatedly trying to raise extra rounds of funding from new VC investors. The Amazon startup already had enough free cash from operations to internally fund their warehouse expansions. The "Amazon had no profits" was an accounting side-effect because of re-investment. Anybody seriously studying Amazon's financial statements in the late 1990s would have paid more attention to their cash flow rather than "accounting profits". On the other hand, OpenAI doesn't have the same positive cash flow situation as early Amazon. They are truly burning more money than they take in. They have to get billions from new investors to buy GPUs and pay salaries. ($40 billion raised in latest investment round.) They are cash flow negative. The cash flow from ChatGPT subscription fees is not enough to internally fund their growth. [1] https://en.wikipedia.org/wiki/Free_cash_flow https://en.wikipedia.org/wiki/Free_cash_flow
isoprophlex 2y ago
In the light of this article, it makes sense that OpenAI are taking a "lmao we dont even pretend to care" approach to safety and intellectual property right now. Altman loudly hyping "look you can ghibli-fy yourself", stating inflammatory things like "we are the death of the graphic designer"; a desparate ploy to rapidly consume the market before the bubble bursts.
iteratethis 2y ago
Just out of curiosity, I wish online LLMs would show real-time power usage and actual dollar costs as you interact with it. It would be so insightful to understand to which degree the technology is subsidized and what the actual value/cost ratio is. I've read somewhere that generating a single AI image draws as much power as a full smartphone charge. In case the suspicion is true that costs are too high to be monetized, then the current scale-up phase is going to be interesting. Right now people infrequently have a chat with AI. That's quite a different scenario from having it integrated across every stack and it constantly being used in the background, by billions of people. Late as they may be, for the consumer space I think Apple is clever to push as much as possible to the local device.
- polytely 2y ago
  feel like if they did this the whole AI bubble would pop
  - keyringlight 2y ago
    It's not just Apple integrating AI into the hardware, Microsoft has been part of a big push to "AI PCs" with a certain minimum capabilities (and I'm sure their partners don't mind selling new gear) and the copilot button on keyboards, and certain android models have the processors and memory capacities specifically for running AI
- Saigonautica 2y ago
  Also out of curiosity, I did some quick math regarding that claim you read somewhere. Cellphone battery charge: I have a 5000mAh cellphone battery. If we ignore charging losses (pretty low normally, but not sure at 67W fast charging)... That battery stores about 18.5 watt-hours of energy, or about 67 kilojoules. Generating a single image at 1024x1024 resolution with Stable Diffusion on my PC takes somewhere under a minute at a maximum power draw under 500W. Lets cap that at 500*60 = 30 kilojoules. So it seems plausible that for cellphones with smaller batteries, and/or using intense image generation settings, there could be overlap! For typical cases, I think that you could get multiple (but low single digit) of AI generated images for the power cost of a cellphone charge, maybe a bit better at scale. So in other words, maybe "technically incorrect" but not a bad approximation to communicate power use in terms most people would understand. I've heard worse!
DonHopkins 2y ago
I'm using the (more expensive) Gemini 2.5 pro and it's like talking to an adult again after claud went all RFK Jr. Brain Worm on me. People have mentioned on hacker news that there seems to kind of "weather patterns" with how hard the various llms think, like during business hours they get stupid. But of course there is some disagreement about what "business hours" are. It's one of those "vibes". Imagine scheduling your life around the moods of AIs. That's the business model. If you don't want a surly and moody AI with a hangover and bad attitude, you gotta pay more! Like isitdownrightnow.com for crowd sourcing web site availability, there should be a isitdumbrightnow.ai site!
DebtDeflation 2y ago
Shortly after ChatGPT hit the scene, everyone said "Google invented this technology, how could they fall so far behind in commercializing it, haha they're IBM now". Maybe they didn't fall behind in anything, maybe they just did an analysis of what it would cost to train transformer models with hundreds of billions of parameters, to run inferencing on them, and then decided that there was no way to actually be profitable doing this.
- amelius 2y ago
  Not with an ad-based monetization model. But what if consumers opened their wallets?
  - discreteevent 2y ago
    There is no what if. The consumers haven't opened their wallets (in time for these companies to survive).
  - rchaud 2y ago
    Ad revenue dwarfs any number that consumer dollars could put up. ChatGPT has been a household name for years, Copilot bloatware is shoved into every Office subscription on earth, and it still runs deep in the red. Ads are the only way.
  - DebtDeflation 2y ago
    What if they considered that and determined that at typical consumer SaaS pricing of low tens of dollars per user per month it's "still impossible to make money"? What if they went a step further and looked at typical Enterprise SaaS pricing (low hundreds of dollars per seat per month) and determined "still can't make money"?
seydor 2y ago
and how much value would be lost?
ConSeannery 2y ago
Ads will eventually make their way into the responses or side bars. It will be interesting (and depressing) to see who does it first and who holds out hoping to squeeze out the ad-supported LLM providers.
Havoc 2y ago
A slow pruning here seems healthy. The more interesting question to me is how gpu vs tpu plays out. Plus the other npu like approaches. Sambanova cerebras groq etc
- internet_points 2y ago
  If only. I think the more likely path is enshittification (ads etc. inside the llm's)
meltyness 2y ago
It sounds like you probably mean OPEX, I mean unless you explicitly are talking about loan payments.

88 points by abawany 2y ago

Found funny that for something that is pretty much a commodity at this point, adoption seems to be the most important metrics. Yes, there are differences between the models, and yes some may work better. But picking the model at this point is just picking the cheapest option. For most use cases any model will do.

timewizard 2y ago
> and yes some may work better. Isn't that where the cost lies? Data, annotation, and model generation all have mostly linear responses to changes in spending. > For most use cases any model will do. They'll operate. They will not produce reliable results. Adoption is one metric, but intentional avoidance should be another.
PeterStuer 2y ago
That is not my experience at all. Models are still leapfrogging each other every month in e.g. coding or research capability, or even in more mundane tasks such as summerizing long multi topic texts. Depending on which side of the issue you fall, you're hoping this will go on for a long time to come, or praying that it will end asap. I'm not using the cheapest in neither my own support, nor in my production systems.
- nitwit005 2y ago
  There are a ton of users that just want help generating emails, or adding some stock photo-like images to a blog post. If the choice is between something that costs $10 a month or $20 a month, and both solve those use cases, it's rational to pick the cheap one.
  - Ekaros 2y ago
    And there will probably also be choice of something free and something trial... Which mean even less money spend.
- senectus1 2y ago
  from what I've seen the "leapfrogging" is very very incremental. They all seems to racing to the plateau... It doesn't look like there will ever be a "stand out" leader and the product that each company presents to the market appears to be essentially the same product that everyone else presents to the market. Maybe with some slight twist to it that is easily recruitable or exceedable within a few months. This is the issue really. at some point the investors are all going to realize that non of their investments are going to be market leaders. When they get to that stage the bubble will well and truly pop.
  - raducu 2y ago
    > They all seems to racing to the plateau. To me it feels there is no plateau and the models are already very useful and impactful. I believe there is no plateau because there is nothing objectively special or magical about the human mind and it all can and will be eventually solved, one hack at a time.
- xiphias2 2y ago
  There seems to be some part of the LLM capabilities being lost by hacking some benchmarks. Claude 3.7 is a great example of a model clearly beating 3.5 in all benchmarks, but slowly destroying my code base by adding lots of extra lines or hacking around my instructions (adding ,,if'' statements when I want it to change the code to handle a case instead of understanding what change is really needed to be done for it). I still prefer o1 pro and a lot of those leapfrogging in benchmarks don't translate to being smarter anymore.
acchow 2y ago
How do you compose the cheapest models to create a software engineer?
- dickersnoodle 2y ago
  You don't have to create a real software engineer, you just have to create one that looks close enough to get some executive his bonus and won't fall over before he's moved on to another company...
alexdoesstuff 2y ago
Full agree! Being close to the edge of AI usage, it's important to realize that most AI use cases are not "fully autonomous AI software engineer" or "deep research into a niche topic" but way more innocuous: Improve my blog post, what's the capital of France, what are some nice tourist sites to see around my next vacation destination. For those non-edge use cases, costs are an issue, but so are inertia and switching costs. A big reason OpenAI and ChatGPT are so huge is that it's still their go-to model for all of these non-edge use cases as it's well known, well adopted, and quite frankly very efficiently priced.
Yizahi 2y ago
Adoption is critical for these LLM corporations, because unlike in other industries, here free tier users incur almost the same costs as the paid tier users. They really can't degrade free tier experience too much, or their customers will flee to the competitors. I've read one guy calculating expenses of these corpos and they are truly insane by now and are constantly rising.

throwup238 2y ago

> As the global tech research company forecasts worldwide generative AI (GenAI) spending will reach $644 billion in 2025, up around 76 percent from 2024 I’m having a hard time squaring the number $644 billion and the phrase “extinction phase.” I don’t believe their actual estimate of GenAI spending but if it’s even in the same ballpark as the real value, that’s not an extinction.

somenameforme 2y ago
That's the entire point. Bubbles are caused when future valuations drive excessive spending far beyond a reasonable valuation of something. Then at some point reality hits and it turns into a game of hot potato as people don't want to left holding the bag. Pushing towards a trillion bucks a year for what LLMs are mostly currently used for does not seem like a sustainable system.
- pjc50 2y ago
  Where are they going to get the trillions in revenue to pay any of that back? That's 10% of 2023 US total wages and salaries. Do people really believe it'll replace that much labour?
  - esseph 2y ago
    VC demands their valuation multipliers.
alexdoesstuff 2y ago
Reading through the source [1] they basically get to that huuuuge number by including AI-enabled devices such as phones that have some AI functionality even if not core to their value proposition. That's basically reclassifying a big chunk of smartphones, TVs, and other consumer tech as GenAI spending. Of the "real" categories, they expect: Service 27bn (+162% y/y) Software 37bn (+93% y/y) Servers 180bn (+33% y/y) for a total of $245bn (+58% y/y) That's not shabby numbers, but way more reasonable. Hyperscaler total capex [2] is expected to be around $330bn in 2025 (up +32% y/y) so that'll most likely include a good chunk of the server spend. [1] https://www.gartner.com/en/newsroom/press-releases/2025-03-31-gartner-forecasts-worldwide-genai-spending-to-reach-644-billion-in-2025 https://www.gartner.com/en/newsroom/press-releases/2025-03-3... [2] https://www.marvin-labs.com/blog/deepseek-impact-of-high-quality-low-cost-inference-on-ai-landscape/ https://www.marvin-labs.com/blog/deepseek-impact-of-high-qua...

PeterStuer 2y ago

It took Amazon around six to seven years to see its first profitable quarter, and they still went into the red sometimes when doing major investments thereafter.

paulluuk 2y ago
I could be wrong, but I believe Amazon's business model was very simple: do everything as cheaply as possible, run at a loss until all competition is dead, and then raise prices once we dominate the market. I don't think OpenAI has that option.
- dukeyukey 2y ago
  I don't think that's true, even in rural areas without choices Amazon is pretty cheap.
- karmakurtisaani 2y ago
  Also, I think Amazon invested everything into growth, for which there was a lot of potential. Seems different for the AI companies.
- delecti 2y ago
  Eh, it's more that per-transaction they were profitable, so they needed to scale enough to make up for their fixed costs. OpenAI's fixed costs are enormous, and they're only bringing in pennies from subscriptions.
- rongrobert 2y ago
  This wasn't the business model. The business model was you could sell books over the internet at a much cheaper cost compared to Barnes and Noble or Borders because they weren't paying for physical locations and there was no sales tax on the transactions because it was on the internet.
jasode 2y ago
>It took Amazon around six to seven years to see its first profitable quarter, A key difference from OpenAI is that Amazon was cash flow positive from very early on and before that first profitable quarter. They only needed one funding round Series A of $8 million instead of repeatedly trying to raise extra rounds of funding from new VC investors. The Amazon startup already had enough free cash from operations to internally fund their warehouse expansions. The "Amazon had no profits" was an accounting side-effect because of re-investment. Anybody seriously studying Amazon's financial statements in the late 1990s would have paid more attention to their cash flow rather than "accounting profits". On the other hand, OpenAI doesn't have the same positive cash flow situation as early Amazon. They are truly burning more money than they take in. They have to get billions from new investors to buy GPUs and pay salaries. ($40 billion raised in latest investment round.) They are cash flow negative. The cash flow from ChatGPT subscription fees is not enough to internally fund their growth. [1] https://en.wikipedia.org/wiki/Free_cash_flow https://en.wikipedia.org/wiki/Free_cash_flow

isoprophlex 2y ago

In the light of this article, it makes sense that OpenAI are taking a "lmao we dont even pretend to care" approach to safety and intellectual property right now. Altman loudly hyping "look you can ghibli-fy yourself", stating inflammatory things like "we are the death of the graphic designer"; a desparate ploy to rapidly consume the market before the bubble bursts.

iteratethis 2y ago

Just out of curiosity, I wish online LLMs would show real-time power usage and actual dollar costs as you interact with it. It would be so insightful to understand to which degree the technology is subsidized and what the actual value/cost ratio is. I've read somewhere that generating a single AI image draws as much power as a full smartphone charge. In case the suspicion is true that costs are too high to be monetized, then the current scale-up phase is going to be interesting. Right now people infrequently have a chat with AI. That's quite a different scenario from having it integrated across every stack and it constantly being used in the background, by billions of people. Late as they may be, for the consumer space I think Apple is clever to push as much as possible to the local device.

polytely 2y ago
feel like if they did this the whole AI bubble would pop
- keyringlight 2y ago
  It's not just Apple integrating AI into the hardware, Microsoft has been part of a big push to "AI PCs" with a certain minimum capabilities (and I'm sure their partners don't mind selling new gear) and the copilot button on keyboards, and certain android models have the processors and memory capacities specifically for running AI
Saigonautica 2y ago
Also out of curiosity, I did some quick math regarding that claim you read somewhere. Cellphone battery charge: I have a 5000mAh cellphone battery. If we ignore charging losses (pretty low normally, but not sure at 67W fast charging)... That battery stores about 18.5 watt-hours of energy, or about 67 kilojoules. Generating a single image at 1024x1024 resolution with Stable Diffusion on my PC takes somewhere under a minute at a maximum power draw under 500W. Lets cap that at 500*60 = 30 kilojoules. So it seems plausible that for cellphones with smaller batteries, and/or using intense image generation settings, there could be overlap! For typical cases, I think that you could get multiple (but low single digit) of AI generated images for the power cost of a cellphone charge, maybe a bit better at scale. So in other words, maybe "technically incorrect" but not a bad approximation to communicate power use in terms most people would understand. I've heard worse!

DonHopkins 2y ago

I'm using the (more expensive) Gemini 2.5 pro and it's like talking to an adult again after claud went all RFK Jr. Brain Worm on me. People have mentioned on hacker news that there seems to kind of "weather patterns" with how hard the various llms think, like during business hours they get stupid. But of course there is some disagreement about what "business hours" are. It's one of those "vibes". Imagine scheduling your life around the moods of AIs. That's the business model. If you don't want a surly and moody AI with a hangover and bad attitude, you gotta pay more! Like isitdownrightnow.com for crowd sourcing web site availability, there should be a isitdumbrightnow.ai site!

DebtDeflation 2y ago

Shortly after ChatGPT hit the scene, everyone said "Google invented this technology, how could they fall so far behind in commercializing it, haha they're IBM now". Maybe they didn't fall behind in anything, maybe they just did an analysis of what it would cost to train transformer models with hundreds of billions of parameters, to run inferencing on them, and then decided that there was no way to actually be profitable doing this.

amelius 2y ago
Not with an ad-based monetization model. But what if consumers opened their wallets?
- discreteevent 2y ago
  There is no what if. The consumers haven't opened their wallets (in time for these companies to survive).
- rchaud 2y ago
  Ad revenue dwarfs any number that consumer dollars could put up. ChatGPT has been a household name for years, Copilot bloatware is shoved into every Office subscription on earth, and it still runs deep in the red. Ads are the only way.
- DebtDeflation 2y ago
  What if they considered that and determined that at typical consumer SaaS pricing of low tens of dollars per user per month it's "still impossible to make money"? What if they went a step further and looked at typical Enterprise SaaS pricing (low hundreds of dollars per seat per month) and determined "still can't make money"?

seydor 2y ago

and how much value would be lost?

ConSeannery 2y ago

Ads will eventually make their way into the responses or side bars. It will be interesting (and depressing) to see who does it first and who holds out hoping to squeeze out the ad-supported LLM providers.

Havoc 2y ago

A slow pruning here seems healthy. The more interesting question to me is how gpu vs tpu plays out. Plus the other npu like approaches. Sambanova cerebras groq etc

internet_points 2y ago
If only. I think the more likely path is enshittification (ads etc. inside the llm's)

meltyness 2y ago

It sounds like you probably mean OPEX, I mean unless you explicitly are talking about loan payments.