3 ms·

>inference is already being subsidized. Inference is not being subsidized and in fact has pretty high margins. Similar-sized open weight models on openrouter

>inference is already being subsidized. Inference is not being subsidized and in fact has pretty high margins. Similar-sized open weight models on openrouter are 15x cheaper per token than the big labs. This should reflect the isolated cost of inference, since 3rd party hosts have no reason to subsidize and no training costs to amortize. Only datacenter buildout costs are being subsidized.

dvt 20d ago
> Inference is not being subsidized and in fact has pretty high margins. I was referring to the "AI labs" here. Sam Altman himself conceded that OpenAI is losing money on the $200 subscription. Using open-weight/open-source models is indeed cheaper (and no reason for inference to be subsidized).
- Legend2440 20d ago
  That's not what I mean. If competitors can offer tokens 15x cheaper, the big labs must have high margins per token. (which they can use to amortize training costs) >Sam Altman himself conceded that OpenAI is losing money on the $200 subscription. They have since stopped offering the $200 subscription, probably for this reason. Subscription margins are harder to judge because it depends on usage; token costs are a better comparison.
reticulates 20d ago
The majority of revenue comes from API usage. The majority of usage comes from subscriptions. For any of the numbers to make any sense, subscriptions must be subsidized ergo the majority of usage is subsidized. A single $200 subscription can incur upwards of $10,000 in API equivalent usage (and even more when there are frequent resets). If it were true that Anthropic and OpenAI were profitable on all inference they wouldn’t need to constantly raise so much money. Anthropic regularly announce huge investments in infrastructure but it is all smoke and mirrors, data center build out costs aren’t being paid by OpenAI and Anthropic, they’re financed externally. Google, for example, are backstopping tens of billions of datacenter build outs that are being financed based on commitments but not investment from Anthropic. You are underestimating the insanity of subscription subsidization. Being profitable on API inference is meaningless when it is such a small proportion of usage and is only going to fall off a cliff as cheap open weight models become more capable. https://hraness.com/writing/my-girlfriend-asked-me-why-i-have https://hraness.com/writing/my-girlfriend-asked-me-why-i-hav... The absolute majority of tokens are being subsidized and as soon as the subsidies end usage will fall off a cliff, rendering all the data center buildout a terrible waste of money.
- delecti 20d ago
  > The majority of usage comes from subscriptions Do we know that? As I understand it, enterprise customers pay more. Do we know the usage breakdown between monthly subscribers vs enterprise accounts? I agree that it's inevitable that subsidized subscriptions are unlikely to last forever, but that's not the only assumption in your argument. Edit: I think "enterprise customers pay more" was poorly phrased. I mean that enterprise customers are charged per token, presumably with a profit margin, and thus are not subsidized. While personal accounts are (thought to be) highly subsidized if you consistently max out the quotas. We also don't know what proportion of personal accounts do that though, which is another big question mark.
- rovr138 20d ago
  I think you're also missing a quirk and that is, is everyone on a $200 plan using $10,000 worth of equivalent API spend? I know people that have the most expensive plan on all the platforms... because The other side to that is, what is 'cost'? Is cost just inference or are expenses also being taken into account? Because the expenses of these companies are huge to build the models.
- nl 20d ago
  > The majority of usage comes from subscriptions. This is untrue. You are way underestimating enterprise usage here. You can't get the $200 subscription on Teams plans at all, and Enterprise plans don't have any subsidized plans. Anthropic has 80% margins on inference: https://archive.is/BtEeN#selection-1575.0-1575.75 https://archive.is/BtEeN#selection-1575.0-1575.75
  - reticulates 20d ago
    The numbers in the article are forecasts but let’s take them as real. That’s $10bn of revenue, the majority from enterprise customers, let’s say 75% from enterprise API usage: $7.5 billion. If the margin on inference is 80% that means of the $7.5bn in enterprise revenue they’re spending $1.5bn on compute. Yet we know that they actually spend over $5bn per month on compute, which includes the $1.25bn per month to SpaceX. If $7.5bn is their enterprise revenue and it costs just $1.5bn to generate, that leaves $3.5bn in compute costs to account for. Dario previously said that training costs less than inference so training can’t explain it. If subscriptions aren’t the majority of usage and aren’t subsidized, where is the money going? Anthropic don’t spend money on data centre build out so that can’t be it either.
denverllc 20d ago
> Inference is not being subsidized and in fact has pretty high margins. 1. Companies are trying to decrease costs, not increase it, and are looking at alternatives 2. Competitors are catching up, and even if the frontier labs are "better" at some things (like writing plans or complicated analysis), the competitors can take a lot of the inference on routine tasks like implementing a well-defined plan 3. The frontier labs don't just need to have high margins right now. They have to pay back their massive liabilities.

dvt 20d ago

> Inference is not being subsidized and in fact has pretty high margins. I was referring to the "AI labs" here. Sam Altman himself conceded that OpenAI is losing money on the $200 subscription. Using open-weight/open-source models is indeed cheaper (and no reason for inference to be subsidized).

Legend2440 20d ago
That's not what I mean. If competitors can offer tokens 15x cheaper, the big labs must have high margins per token. (which they can use to amortize training costs) >Sam Altman himself conceded that OpenAI is losing money on the $200 subscription. They have since stopped offering the $200 subscription, probably for this reason. Subscription margins are harder to judge because it depends on usage; token costs are a better comparison.

reticulates 20d ago

The majority of revenue comes from API usage. The majority of usage comes from subscriptions. For any of the numbers to make any sense, subscriptions must be subsidized ergo the majority of usage is subsidized. A single $200 subscription can incur upwards of $10,000 in API equivalent usage (and even more when there are frequent resets). If it were true that Anthropic and OpenAI were profitable on all inference they wouldn’t need to constantly raise so much money. Anthropic regularly announce huge investments in infrastructure but it is all smoke and mirrors, data center build out costs aren’t being paid by OpenAI and Anthropic, they’re financed externally. Google, for example, are backstopping tens of billions of datacenter build outs that are being financed based on commitments but not investment from Anthropic. You are underestimating the insanity of subscription subsidization. Being profitable on API inference is meaningless when it is such a small proportion of usage and is only going to fall off a cliff as cheap open weight models become more capable. https://hraness.com/writing/my-girlfriend-asked-me-why-i-have https://hraness.com/writing/my-girlfriend-asked-me-why-i-hav... The absolute majority of tokens are being subsidized and as soon as the subsidies end usage will fall off a cliff, rendering all the data center buildout a terrible waste of money.

delecti 20d ago
> The majority of usage comes from subscriptions Do we know that? As I understand it, enterprise customers pay more. Do we know the usage breakdown between monthly subscribers vs enterprise accounts? I agree that it's inevitable that subsidized subscriptions are unlikely to last forever, but that's not the only assumption in your argument. Edit: I think "enterprise customers pay more" was poorly phrased. I mean that enterprise customers are charged per token, presumably with a profit margin, and thus are not subsidized. While personal accounts are (thought to be) highly subsidized if you consistently max out the quotas. We also don't know what proportion of personal accounts do that though, which is another big question mark.
rovr138 20d ago
I think you're also missing a quirk and that is, is everyone on a $200 plan using $10,000 worth of equivalent API spend? I know people that have the most expensive plan on all the platforms... because The other side to that is, what is 'cost'? Is cost just inference or are expenses also being taken into account? Because the expenses of these companies are huge to build the models.
nl 20d ago
> The majority of usage comes from subscriptions. This is untrue. You are way underestimating enterprise usage here. You can't get the $200 subscription on Teams plans at all, and Enterprise plans don't have any subsidized plans. Anthropic has 80% margins on inference: https://archive.is/BtEeN#selection-1575.0-1575.75 https://archive.is/BtEeN#selection-1575.0-1575.75
- reticulates 20d ago
  The numbers in the article are forecasts but let’s take them as real. That’s $10bn of revenue, the majority from enterprise customers, let’s say 75% from enterprise API usage: $7.5 billion. If the margin on inference is 80% that means of the $7.5bn in enterprise revenue they’re spending $1.5bn on compute. Yet we know that they actually spend over $5bn per month on compute, which includes the $1.25bn per month to SpaceX. If $7.5bn is their enterprise revenue and it costs just $1.5bn to generate, that leaves $3.5bn in compute costs to account for. Dario previously said that training costs less than inference so training can’t explain it. If subscriptions aren’t the majority of usage and aren’t subsidized, where is the money going? Anthropic don’t spend money on data centre build out so that can’t be it either.

denverllc 20d ago

> Inference is not being subsidized and in fact has pretty high margins. 1. Companies are trying to decrease costs, not increase it, and are looking at alternatives 2. Competitors are catching up, and even if the frontier labs are "better" at some things (like writing plans or complicated analysis), the competitors can take a lot of the inference on routine tasks like implementing a well-defined plan 3. The frontier labs don't just need to have high margins right now. They have to pay back their massive liabilities.