4 ms·

> As long as every AI provider is operating at a loss None of them are doing that. They need funding because the next model has always been much more expensiv

> As long as every AI provider is operating at a loss None of them are doing that. They need funding because the next model has always been much more expensive to train than the profits of the previous model. And many do offer a lot of free usage which is of course operated at a loss. But I don't think any are operating inference at a loss, I think their margins are actually rather large.

hoppp 1y ago
When comparing the cost of an H100 GPU per hour and calculating cost of tokens, it seems the OpenAI offering for the latest model is 5 times cheaper than renting the hardware. OpenAI balance sheet also shows an $11 billion loss . I can't see any profit on anything they create. The product is good but it relies on investors fueling the AI bubble.
- Workaccount2 1y ago
  https://martinalderson.com/posts/are-openai-and-anthropic-really-losing-money-on-inference/ https://martinalderson.com/posts/are-openai-and-anthropic-re... All the labs are going hard on training and new GPUs. If we ever level off, they probably will be immensely profitable. Inference is cheap, training is expensive.
  - svnt 1y ago
    To do this analysis on an hourly retail cost and an open weight model and infer anything about the situation at OpenAI or Anthropic is quite a reach. For one (basic) thing, they buy and own their hardware, and have to size their resources for peak demand. For another, Deepseek R1 does not come close to matching claude performance in many real tasks.
- simonw 1y ago
  > When comparing the cost of an H100 GPU per hour and calculating cost of tokens, it seems the OpenAI offering for the latest model is 5 times cheaper than renting the hardware. How did you come to that conclusion? That would be a very notable result if it did turn out OpenAI were selling tokens for 5x the cost it took to serve them.
  - khimaros 1y ago
    it seems to me they are saying the opposite
  - necovek 1y ago
    I am reading it as OpenAI selling them for 20% of the cost to serve them (serving at the equivalent token/s with cloud pay-per-use GPUs).
    - simonw 1y ago
      You're right, I misunderstood.
lmm 1y ago
> But I don't think any are operating inference at a loss, I think their margins are actually rather large. Citation needed. I haven't seen any of them claim to have even positive gross margins to shareholders/investors, which surely they would do if they did.
- furyofantares 1y ago
  https://officechai.com/ai/each-individual-ai-model-can-already-be-profitable-anthropic-ceo-dario-amodei/ https://officechai.com/ai/each-individual-ai-model-can-alrea...
  - svnt 1y ago
    > “if you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There’s some cost to inference with the model, but let’s just assume, in this cartoonish cartoon example, that even if you add those two up, you’re kind of in a good state. So, if every model was a company, the model, in this example, profitable,” he added. “What’s going on is that while you’re reaping the benefits from one company, you’re founding another company that’s much more expensive and requires much more upfront R&D investment. The way this is going to shake out is that it’s going to keep going up until the numbers get very large, and the models can’t get larger, and then there will be a large, very profitable business. Or at some point, the models will stop getting better, and there will perhaps be some overhang — we spent some money, and we didn’t get anything for it — and then the business returns to whatever scale it’s at,” he said. This take from Amodei is hilarious but explains so much.
GoatInGrey 1y ago
So AI companies are profitable when you ignore some of the things they have to spend money on to operate? Snark aside, inference is still being done at a loss. Anthropic, the most profitable AI vendor, is operating at a roughly -140% margin. xAI is the worst at somewhere around -3,600% margin.
- fluidcruft 1y ago
  If they are not operating inference at a loss and current models remain useful (why would they regress?), they could just stop developing the next model.
  - balder1991 1y ago
    They could, but that’s a recipe for going out of business in the current environment.
    - fluidcruft 1y ago
      Yes, but at the same time it's unlikely for existing models to disappear. You won't get the next model, but there is no choice but to keep inference running to pay off creditors.
  - philipwhiuk 1y ago
    At minimum they have to incorporate new data every month or the models will fail to know how many Shrek movies there are and become increasingly wrong in a world that isn't static.
    - fluidcruft 1y ago
      That sort of thing isn't necessary for all use cases. But if you're relying on the system to encode wikipedia or the zeitgeist then sure.
      - thunderfork 1y ago
roadside_picnic 1y ago
Parent comment never said operating inference at a loss, though it wouldn't surprise me, they just said "operating at a loss" which they most definitely are [0]. However, knowing a few people on teams at inference-only providers, I can promise you some of them absolutely are operating inference at a loss. 0. https://www.theregister.com/2025/10/29/microsoft_earnings_q1_26_openai_loss/ https://www.theregister.com/2025/10/29/microsoft_earnings_q1...
- furyofantares 1y ago
  > Parent comment never said operating inference at a loss Context. Whether inference is profitable at current prices is what informs how risky it is to build a product that depends on buying inference, which is what the post was about.
  - roadside_picnic 1y ago
    So you're assuming there's a world where these companies exist solely by providing inference? The first obvious limitation of this would be that all models would be frozen in time. These companies are operating at an insane loss and a major part of that loss is required to continue existing. It's not realistic to imagine that there is an "inference" only future for these large AI companies. And again, there are many inference only startups right now, and I know plenty of them are burning cash providing inference. I've done a lot of work fairly close to the inference layer and getting model serving happening with the requirements for regular business use is fairly tricky business and not as cheap as you seem to think.
    - furyofantares 1y ago
      > So you're assuming there's a world where these companies exist solely by providing inference? Yes, obviously? There is no world where the models and hardware just vanish.
      - HDThoreaun 1y ago
throwaway8xak92 1y ago
> None of them are doing that. Can you point us to the data?
necovek 1y ago
Sounds quite a bit like pyramid scheme "business model": how is it different? If a company stops training new models until they can fund it out of previous profits, do we only slow down or halt altogether? If they all do?
welcome_dragon 1y ago
Isn't that operating at a loss

hoppp 1y ago

When comparing the cost of an H100 GPU per hour and calculating cost of tokens, it seems the OpenAI offering for the latest model is 5 times cheaper than renting the hardware. OpenAI balance sheet also shows an $11 billion loss . I can't see any profit on anything they create. The product is good but it relies on investors fueling the AI bubble.

Workaccount2 1y ago
https://martinalderson.com/posts/are-openai-and-anthropic-really-losing-money-on-inference/ https://martinalderson.com/posts/are-openai-and-anthropic-re... All the labs are going hard on training and new GPUs. If we ever level off, they probably will be immensely profitable. Inference is cheap, training is expensive.
- svnt 1y ago
  To do this analysis on an hourly retail cost and an open weight model and infer anything about the situation at OpenAI or Anthropic is quite a reach. For one (basic) thing, they buy and own their hardware, and have to size their resources for peak demand. For another, Deepseek R1 does not come close to matching claude performance in many real tasks.
simonw 1y ago
> When comparing the cost of an H100 GPU per hour and calculating cost of tokens, it seems the OpenAI offering for the latest model is 5 times cheaper than renting the hardware. How did you come to that conclusion? That would be a very notable result if it did turn out OpenAI were selling tokens for 5x the cost it took to serve them.
- khimaros 1y ago
  it seems to me they are saying the opposite
- necovek 1y ago
  I am reading it as OpenAI selling them for 20% of the cost to serve them (serving at the equivalent token/s with cloud pay-per-use GPUs).
  - simonw 1y ago
    You're right, I misunderstood.

lmm 1y ago

> But I don't think any are operating inference at a loss, I think their margins are actually rather large. Citation needed. I haven't seen any of them claim to have even positive gross margins to shareholders/investors, which surely they would do if they did.

furyofantares 1y ago
https://officechai.com/ai/each-individual-ai-model-can-already-be-profitable-anthropic-ceo-dario-amodei/ https://officechai.com/ai/each-individual-ai-model-can-alrea...
- svnt 1y ago
  > “if you consider each model to be a company, the model that was trained in 2023 was profitable. You paid $100 million, and then it made $200 million of revenue. There’s some cost to inference with the model, but let’s just assume, in this cartoonish cartoon example, that even if you add those two up, you’re kind of in a good state. So, if every model was a company, the model, in this example, profitable,” he added. “What’s going on is that while you’re reaping the benefits from one company, you’re founding another company that’s much more expensive and requires much more upfront R&D investment. The way this is going to shake out is that it’s going to keep going up until the numbers get very large, and the models can’t get larger, and then there will be a large, very profitable business. Or at some point, the models will stop getting better, and there will perhaps be some overhang — we spent some money, and we didn’t get anything for it — and then the business returns to whatever scale it’s at,” he said. This take from Amodei is hilarious but explains so much.

GoatInGrey 1y ago

So AI companies are profitable when you ignore some of the things they have to spend money on to operate? Snark aside, inference is still being done at a loss. Anthropic, the most profitable AI vendor, is operating at a roughly -140% margin. xAI is the worst at somewhere around -3,600% margin.

fluidcruft 1y ago
If they are not operating inference at a loss and current models remain useful (why would they regress?), they could just stop developing the next model.
- balder1991 1y ago
  They could, but that’s a recipe for going out of business in the current environment.
  - fluidcruft 1y ago
    Yes, but at the same time it's unlikely for existing models to disappear. You won't get the next model, but there is no choice but to keep inference running to pay off creditors.
- philipwhiuk 1y ago
  At minimum they have to incorporate new data every month or the models will fail to know how many Shrek movies there are and become increasingly wrong in a world that isn't static.
  - fluidcruft 1y ago
    That sort of thing isn't necessary for all use cases. But if you're relying on the system to encode wikipedia or the zeitgeist then sure.
    - thunderfork 1y ago

roadside_picnic 1y ago

Parent comment never said operating inference at a loss, though it wouldn't surprise me, they just said "operating at a loss" which they most definitely are [0]. However, knowing a few people on teams at inference-only providers, I can promise you some of them absolutely are operating inference at a loss. 0. https://www.theregister.com/2025/10/29/microsoft_earnings_q1_26_openai_loss/ https://www.theregister.com/2025/10/29/microsoft_earnings_q1...

furyofantares 1y ago
> Parent comment never said operating inference at a loss Context. Whether inference is profitable at current prices is what informs how risky it is to build a product that depends on buying inference, which is what the post was about.
- roadside_picnic 1y ago
  So you're assuming there's a world where these companies exist solely by providing inference? The first obvious limitation of this would be that all models would be frozen in time. These companies are operating at an insane loss and a major part of that loss is required to continue existing. It's not realistic to imagine that there is an "inference" only future for these large AI companies. And again, there are many inference only startups right now, and I know plenty of them are burning cash providing inference. I've done a lot of work fairly close to the inference layer and getting model serving happening with the requirements for regular business use is fairly tricky business and not as cheap as you seem to think.
  - furyofantares 1y ago
    > So you're assuming there's a world where these companies exist solely by providing inference? Yes, obviously? There is no world where the models and hardware just vanish.
    - HDThoreaun 1y ago

throwaway8xak92 1y ago

> None of them are doing that. Can you point us to the data?

necovek 1y ago

Sounds quite a bit like pyramid scheme "business model": how is it different? If a company stops training new models until they can fund it out of previous profits, do we only slow down or halt altogether? If they all do?

welcome_dragon 1y ago

Isn't that operating at a loss