3 ms·

Oh, they burned a lot more than $200, you just paid only $200. These things are costing way more than what people pay for them, the price heavily subsidized.

simonw 1y ago
I think the opposite is much more likely to be true: that vendors who charge money for inference are charging more than it costs them to service a prompt. I've heard from sources that I trust that both AWS and Google Gemini charge more than it costs them in energy to run inference. You can get a good estimate for the truth here by considering open weight models. It's possible to determine exactly how much energy it costs to serve DeepSeek V3.2 Exp, since that model is open weight. So run that calculation, then take a look at how much providers are charging to serve it and see if they are likely operating at a loss. Here are some prices for that particular model: https://openrouter.ai/deepseek/deepseek-v3.2-exp/providers https://openrouter.ai/deepseek/deepseek-v3.2-exp/providers
- Tade0 1y ago
  If that's the case, then why are AI companies bleeding money? Or: what are they bleeding money on?
  - anupsingh123 1y ago
    btw this was DeepSeek-V3.2. If I'd been using Claude Sonnet 4.5, we'd be looking at a $2000 bill instead.
    - Tade0 1y ago
      Okay, yikes. Good thing that you even can set up those controls, unlike with that other company in the compute infrastructure business.
  - barrkel 1y ago
    Research runs mostly. https://epoch.ai/data-insights/openai-compute-spend https://epoch.ai/data-insights/openai-compute-spend
  - simonw 1y ago
    They lose money on research and training and offering model trials for free (a marketing expenses). That doesn't mean that when they do charge for the models - especially via their APIs - that they are serving them at a unit cost loss.
    - surgical_fire 1y ago
      Depends on the vendor and how they charge. OpenAI loses money on subscriptions [1]. Maybe the people who pay 200 bucks on a subscription are exactly the kind of people that will try to use the maximum out of it, and if you go down to the 20 bucks tier you will find more of the type of user that pays but doesn't use it all that much? I would presume that companies selling compute for AI inference either make some money or at least break even when they serve a request. But I wouldn't b surprised if they are subsidizing this cost for the time being. [1]: https://finance.yahoo.com/news/sam-altman-says-losing-money-080700756.html?guccounter=1 https://finance.yahoo.com/news/sam-altman-says-losing-money-...
      - simonw 1y ago
        That "losing money on subscriptions" story is a one-off Sam Altman tweet from January 2025, when they were promoting their brand new $200 account and the first version of Sora. I wouldn't treat that as a universal truth. https://twitter.com/sama/status/1876104315296968813 https://twitter.com/sama/status/1876104315296968813 "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected"
        surgical_fire 1y ago
        Sam Altman is a bullshitter. A liar cares about the truth and attempts to hide it. A bullshitter doesn't care if something is true of false, and is just using rhetoric to convince you of something. I don't doubt that it is true that they lose money on a 200 subscription because the people that pay 200 are probably the same people that will max out usage over time, no matter how wasteful. Sam Altman was framing it in a way to say "it's so useful people are using it more than we expected!", because he is interested in having everyone believe that LLMs are the future. It's all bullshit. If I had to guess, they probably at least break even on API calls, and might make some money on lower tier subscriptions (i.e.: people that pay for it but use it sparingly on a as-need basis). But that is boring, and hints at limited usability. Investors won't want to burn hundreds of billions in cash for something that may be sort of useful. They want destructive amounts of money in return.
  - Ferret7446 1y ago
    On building the next new feature/integration/whatever? I feel like this should be a rhetorical question, but the fact that it was asked I also feel it is not so...
- beAbU 1y ago
  You cant conveniently ignore the cost of model development and training. This is like saying solar power is free if you ignore the equipment and installation costs. Even worse still, model creators are in an arms race. They can't release a model and call it a day, waiting for it to start paying for itself. They need to immediately jump on to the next version of the model or risk falling behind.
deleted 1y ago
[deleted]

simonw 1y ago

I think the opposite is much more likely to be true: that vendors who charge money for inference are charging more than it costs them to service a prompt. I've heard from sources that I trust that both AWS and Google Gemini charge more than it costs them in energy to run inference. You can get a good estimate for the truth here by considering open weight models. It's possible to determine exactly how much energy it costs to serve DeepSeek V3.2 Exp, since that model is open weight. So run that calculation, then take a look at how much providers are charging to serve it and see if they are likely operating at a loss. Here are some prices for that particular model: https://openrouter.ai/deepseek/deepseek-v3.2-exp/providers https://openrouter.ai/deepseek/deepseek-v3.2-exp/providers

Tade0 1y ago
If that's the case, then why are AI companies bleeding money? Or: what are they bleeding money on?
- anupsingh123 1y ago
  btw this was DeepSeek-V3.2. If I'd been using Claude Sonnet 4.5, we'd be looking at a $2000 bill instead.
  - Tade0 1y ago
    Okay, yikes. Good thing that you even can set up those controls, unlike with that other company in the compute infrastructure business.
- barrkel 1y ago
  Research runs mostly. https://epoch.ai/data-insights/openai-compute-spend https://epoch.ai/data-insights/openai-compute-spend
- simonw 1y ago
  They lose money on research and training and offering model trials for free (a marketing expenses). That doesn't mean that when they do charge for the models - especially via their APIs - that they are serving them at a unit cost loss.
  - surgical_fire 1y ago
    Depends on the vendor and how they charge. OpenAI loses money on subscriptions [1]. Maybe the people who pay 200 bucks on a subscription are exactly the kind of people that will try to use the maximum out of it, and if you go down to the 20 bucks tier you will find more of the type of user that pays but doesn't use it all that much? I would presume that companies selling compute for AI inference either make some money or at least break even when they serve a request. But I wouldn't b surprised if they are subsidizing this cost for the time being. [1]: https://finance.yahoo.com/news/sam-altman-says-losing-money-080700756.html?guccounter=1 https://finance.yahoo.com/news/sam-altman-says-losing-money-...
    - simonw 1y ago
      That "losing money on subscriptions" story is a one-off Sam Altman tweet from January 2025, when they were promoting their brand new $200 account and the first version of Sora. I wouldn't treat that as a universal truth. https://twitter.com/sama/status/1876104315296968813 https://twitter.com/sama/status/1876104315296968813 "insane thing: we are currently losing money on openai pro subscriptions! people use it much more than we expected"
      - surgical_fire 1y ago
        Sam Altman is a bullshitter. A liar cares about the truth and attempts to hide it. A bullshitter doesn't care if something is true of false, and is just using rhetoric to convince you of something. I don't doubt that it is true that they lose money on a 200 subscription because the people that pay 200 are probably the same people that will max out usage over time, no matter how wasteful. Sam Altman was framing it in a way to say "it's so useful people are using it more than we expected!", because he is interested in having everyone believe that LLMs are the future. It's all bullshit. If I had to guess, they probably at least break even on API calls, and might make some money on lower tier subscriptions (i.e.: people that pay for it but use it sparingly on a as-need basis). But that is boring, and hints at limited usability. Investors won't want to burn hundreds of billions in cash for something that may be sort of useful. They want destructive amounts of money in return.
- Ferret7446 1y ago
  On building the next new feature/integration/whatever? I feel like this should be a rhetorical question, but the fact that it was asked I also feel it is not so...
beAbU 1y ago
You cant conveniently ignore the cost of model development and training. This is like saying solar power is free if you ignore the equipment and installation costs. Even worse still, model creators are in an arms race. They can't release a model and call it a day, waiting for it to start paying for itself. They need to immediately jump on to the next version of the model or risk falling behind.

deleted 1y ago

[deleted]