4 ms·

This is surprising to me. Judging by what appears to be the common sentiment here on HN - which is that AI inference is already profitable, and OpenAI is fairly

This is surprising to me. Judging by what appears to be the common sentiment here on HN - which is that AI inference is already profitable, and OpenAI is fairly valued by private markets. Given that Oracle and Microsoft are major counterparties of OpenAI, it seems odd that their stocks have been performing so poorly recently. Can anyone square this circle for me?

jimbokun 3mo ago
HN has been split on this question, with both pro and con strongly and vigorously argued.
cmiles8 3mo ago
The general fallacy of the “but inference is profitable” argument is that it tends to ignore all the costs of building and training the model. Given the fact that 1) that’s not trivial, and 2) the arms race underway means one can’t stop training, then it ruins the financial picture. It’s like saying a new apartment building is “profitable” because the monthly income covers the monthly running costs, but ignoring the giant mortgage that covers the cost of building the building. That thinking is a good way to go bankrupt in real estate and a good way to go bankrupt in AI.
- an0malous 3mo ago
  > The general fallacy of the “but inference is profitable” argument is that it tends to ignore all the costs of building and training the model. Given the fact that 1) that’s not trivial, and 2) the arms race underway means one can’t stop training, then it ruins the financial picture. Or that it’s all hearsay and no one has released financials yet?
  - cmiles8 3mo ago
    Well there is clearly also a lot of non-GAAP style “trust us bro” things going on too which generally boil down to “if you ignore all the reasons why we’re not profitable then we’re profitable.” It’s WeWork’s “community adjusted EBITDA” messaging repackaged.
  - marcosdumay 3mo ago
    xAI financials are public, and OpenAI financials leaked a short while ago. That's the best possible interpretation of them. The other possible interpretation is that they are manipulating the numbers (that they have to show to investors) and inference isn't actually profitable either. If they are not manipulating the numbers right now, both companies have a serious case of uncontrolled operational costs that they have to solve too.
    - surgical_fire 3mo ago
      > That's the best possible interpretation of them. Correct. Because a less charitable interpretation will squint at their marketing spend and wonder if they are just sweeping a lot of their expenses on categories they don't belong to make their business look less bonkers to those that prefer to not ask the ugly questions.
      - marcosdumay 3mo ago
        That. I'm personally on the team that think it's honest. But then they have to explain why their marketing numbers are so weird.
        therobots927 3mo ago
        I don’t know how $5B+ in marketing for OpenAI is actually possible. To put it in perspective, that’s about the combined campaign budget of the Trump and Harris campaigns. I would expect to be seeing a lot more openAI ads than I’m seeing right now. And they would be everywhere.
        surgical_fire 3mo ago
        My charitable guess is that they lump the compute expense for free users as "Marketing". My less than charitable guess is that the leaked financials is a piece of fiction intended to make inference not look unprofitable.
- CamperBob2 3mo ago
  If the company who holds the mortgage wanted to own the building, they would have just bought it themselves. They don't, for whatever reason, so to some extent they have an incentive to help their customer succeed. That's why it's so hard to get a residential mortgage, for example. It's more of a partnership, with more mutual vulnerability, than most people think. Same thing seems to be true here.
lbrito 3mo ago
I think those are just the loud minority. I wouldn't be surprised if they're like 20-30% if a poll were made here
darkwi11ow 3mo ago
Inference might be profitable, but it does not mean the profits of AI datacenters will rise in future. Open weight models and local AI already put the pressure on the AI datacenter profit margins, and local AI is set to become much more efficient in the future.
twoodfin 3mo ago
Good question. Given what happened with xAI’s excess capacity lease to Anthropic, and Meta’s noises about doing the same, seems likely that the demand for inference will continue to slope upwards for a while. If I’m Oracle, I’m not worried about being able to utilize the data centers I’ve built for some price, almost certainly a profitable one. I’m guessing, though, that Oracle made their capital investments on assumptions of a higher price & return. Possibly because it wasn’t clear when these decisions were made how much competition OpenAI would have at the frontier. I don’t think this math is all that hard. Capital markets have everything they need to start to figure it out, most especially a year or two of history to project forward.
deleted 3mo ago
[deleted]
3848484894 3mo ago
That sentiment only seems to pop up in Anthropic / OAI threads, wonder why
chasd00 3mo ago
it is isn't enough for inference to be profitable, the whole organization has to be profitable enough to keep investors from looking elsewhere for a return.
- therobots927 3mo ago
  I agree but I don’t want to concede that point. There’s no evidence that inference is profitable which makes the rest of the conversation a non-starter. When dealing with sophists you have to pin them down on specific points and prevent them from shifting the conversation.

jimbokun 3mo ago

HN has been split on this question, with both pro and con strongly and vigorously argued.

cmiles8 3mo ago

The general fallacy of the “but inference is profitable” argument is that it tends to ignore all the costs of building and training the model. Given the fact that 1) that’s not trivial, and 2) the arms race underway means one can’t stop training, then it ruins the financial picture. It’s like saying a new apartment building is “profitable” because the monthly income covers the monthly running costs, but ignoring the giant mortgage that covers the cost of building the building. That thinking is a good way to go bankrupt in real estate and a good way to go bankrupt in AI.

an0malous 3mo ago
> The general fallacy of the “but inference is profitable” argument is that it tends to ignore all the costs of building and training the model. Given the fact that 1) that’s not trivial, and 2) the arms race underway means one can’t stop training, then it ruins the financial picture. Or that it’s all hearsay and no one has released financials yet?
- cmiles8 3mo ago
  Well there is clearly also a lot of non-GAAP style “trust us bro” things going on too which generally boil down to “if you ignore all the reasons why we’re not profitable then we’re profitable.” It’s WeWork’s “community adjusted EBITDA” messaging repackaged.
- marcosdumay 3mo ago
  xAI financials are public, and OpenAI financials leaked a short while ago. That's the best possible interpretation of them. The other possible interpretation is that they are manipulating the numbers (that they have to show to investors) and inference isn't actually profitable either. If they are not manipulating the numbers right now, both companies have a serious case of uncontrolled operational costs that they have to solve too.
  - surgical_fire 3mo ago
    > That's the best possible interpretation of them. Correct. Because a less charitable interpretation will squint at their marketing spend and wonder if they are just sweeping a lot of their expenses on categories they don't belong to make their business look less bonkers to those that prefer to not ask the ugly questions.
    - marcosdumay 3mo ago
      That. I'm personally on the team that think it's honest. But then they have to explain why their marketing numbers are so weird.
      - therobots927 3mo ago
        I don’t know how $5B+ in marketing for OpenAI is actually possible. To put it in perspective, that’s about the combined campaign budget of the Trump and Harris campaigns. I would expect to be seeing a lot more openAI ads than I’m seeing right now. And they would be everywhere.
        surgical_fire 3mo ago
        My charitable guess is that they lump the compute expense for free users as "Marketing". My less than charitable guess is that the leaked financials is a piece of fiction intended to make inference not look unprofitable.
CamperBob2 3mo ago
If the company who holds the mortgage wanted to own the building, they would have just bought it themselves. They don't, for whatever reason, so to some extent they have an incentive to help their customer succeed. That's why it's so hard to get a residential mortgage, for example. It's more of a partnership, with more mutual vulnerability, than most people think. Same thing seems to be true here.

lbrito 3mo ago

I think those are just the loud minority. I wouldn't be surprised if they're like 20-30% if a poll were made here

darkwi11ow 3mo ago

Inference might be profitable, but it does not mean the profits of AI datacenters will rise in future. Open weight models and local AI already put the pressure on the AI datacenter profit margins, and local AI is set to become much more efficient in the future.

twoodfin 3mo ago

Good question. Given what happened with xAI’s excess capacity lease to Anthropic, and Meta’s noises about doing the same, seems likely that the demand for inference will continue to slope upwards for a while. If I’m Oracle, I’m not worried about being able to utilize the data centers I’ve built for some price, almost certainly a profitable one. I’m guessing, though, that Oracle made their capital investments on assumptions of a higher price & return. Possibly because it wasn’t clear when these decisions were made how much competition OpenAI would have at the frontier. I don’t think this math is all that hard. Capital markets have everything they need to start to figure it out, most especially a year or two of history to project forward.

deleted 3mo ago

[deleted]

3848484894 3mo ago

That sentiment only seems to pop up in Anthropic / OAI threads, wonder why

chasd00 3mo ago

it is isn't enough for inference to be profitable, the whole organization has to be profitable enough to keep investors from looking elsewhere for a return.

therobots927 3mo ago
I agree but I don’t want to concede that point. There’s no evidence that inference is profitable which makes the rest of the conversation a non-starter. When dealing with sophists you have to pin them down on specific points and prevent them from shifting the conversation.