5 ms·

From the limited perspective of software development, today’s models are well-worth their per-token cost. This reads to me like Anthropic anticipating demand a

From the limited perspective of software development, today’s models are well-worth their per-token cost. This reads to me like Anthropic anticipating demand and making a commitment to acquire supply. Not unlike airlines committing to future jet fuel purchases, or Apple committing to future DRAM volume.

an0malous 6mo ago
> From the limited perspective of software development, today’s models are well-worth their per-token cost. At the current price or real price? Anthropic said a $200 subscription can cost them $5000 so the real price could be anywhere from 10-30x the current price.
- RealityVoid 6mo ago
  No, that is probably one of the worst cases they probably saw. Most likely the subscription inference cost is much lower than you expect. If you look at costs for similar open models they are much lower than what you get by buying from anthropic, so that is the real cost basis I expect. It's likely Amazon is making a fucking killing though.
  - SlinkyOnStairs 6mo ago
    While $5000 is a lot, the people who rack up close or just over a thousand "API equivalent cost" are pretty common. > Most likely the subscription inference cost is much lower than you expect. This is probably not true because they'd be screaming it off every rooftop were that the case. Same deal with the API inference. Even the "profitable on inference" claim is sourced back to hearsay of informal statements made by OpenAI/Anthropic staff. No formal announcements, nothing remotely of the "You can trust what I'm saying, because if I'm lying the SEC will have my head" sort. Yet making such statements would be invaluable. If Anthropic can demonstrate profitability before OpenAI, they could poach most of the funding. There's no reason to keep it a company secret. And API inference is only part of the total costs, not even bringing in training and ongoing fine-tuning. If they're not even profitable on inference, how could they hope to be profitable overall.
    - redsocksfan45 6mo ago
      [dead]
    - nielsole 6mo ago
      I don't know about SEC rules but the anthropic CEO said they have a 50%+ margin on API pricing.
      - stackskipton 6mo ago
        SEC rules means CEO cannot lie or deliberately hide the cost of something. 50%+ Margin statements have basically been "We are making 50% on delivering it." This does not include ANY of the costs of getting to this point, training, scraping, datacenters, people and so forth. They are basically saying "Oh yea, the cost of GAS in the car is only X so charging Y per mile is great margin" while ignoring maintenance, cost of acquiring the car and so forth.
        postflopclarity 6mo ago
        but comparing your margin of charging to drive a mile to the price of gas makes a lot of sense? that is the only variable cost in the equation. training / scraping / people are all pretty much fixed costs.
        stackskipton 6mo ago
        Maybe, maybe not. None of us know since these companies are not public, we are not getting clear accounting.
        paradoxyl 6mo ago
        That's a tad naive. CEOs can and have and often lied about everything: Sam Bankman-Fried, Elizabeth Holmes, Kenneth Lay - and hundreds if not thousands more. The SEC is a regulatory agency, not able to bring criminal charges. The above-named for the most part had to be prosecuted by the Department of Justice or sometimes state attorneys.
  - PunchyHamster 6mo ago
    The "worst case" is probably someone just using their $200 account limits. So yeah, real cost is probably close to that
- kiratp 6mo ago
  At the full current retail API price. Business buyers are paying API prices, not subscription Disclosure: Work at Microsoft on AI
  - an0malous 6mo ago
    Are your API prices profitable?
sandworm101 6mo ago
But that per-token cost is a total joke. All these companies are fighting to build market share in some future dominated by one or two AI ecosystems. It is musical chairs until someone creates the one ring to rule them all. So they are charging token amounts just to claim revenue as they burn through investor dollars. In short: per-token charges currently cover maybe 1% of the total costs in this field. To pay ongoing costs, and pay back investors, everyone will need to pay 100x or 1000x the current rates, likely for decades.
- twoodfin 6mo ago
  From the perspective of a deal like this, “total costs in the field” matter less than incremental cost per token served. The unit economics for today’s frontier models should be great, and this suggests Anthropic believes they’ll get better.
- deaux 6mo ago
  > In short: per-token charges currently cover maybe 1% of the total costs in this field There are plenty of seemingly informed people saying the exact opposite, so that's a lot of confidence you're talking with. I have a hard time believing it when we know what open weights models cost to run. And sure, there's training costs, but again many say inference costs are already above training costs.
- postalrat 6mo ago
  In a decade the cost of compute will be a tiny fraction of what it costs now. Specialized hardware will exist that will be cheap and efficient.
  - bitmasher9 6mo ago
    The difference in the cost of compute between 2026 and 2036 won’t be nearly as large as the difference in the cost of compute between 2016 and 2026. Even at 2016 the slowdown in improvements was noticeable. We might see a one time bump in inference when we move off GPUs onto more limited and efficient dedicated hardware, but the sustained fast pace of improvements are far behind us.
svnt 6mo ago
And receiving investment from their vendor in exchange? When this is done in established companies it is typically called a kickback and directed toward one person, but in this case the whole thing is so incestuous the kickback goes straight to the top.
- twoodfin 6mo ago
  Is it crazy to imagine Anthropic can leverage short term cash flow now to build the models and products that will let them resell $100B in AWS infra with nice margins tomorrow? If Amazon believes that story they’d be crazy not to invest.
  - svnt 6mo ago
    Yes I understand why the agreement exists, but that does not remove the circularity.

an0malous 6mo ago

> From the limited perspective of software development, today’s models are well-worth their per-token cost. At the current price or real price? Anthropic said a $200 subscription can cost them $5000 so the real price could be anywhere from 10-30x the current price.

RealityVoid 6mo ago
No, that is probably one of the worst cases they probably saw. Most likely the subscription inference cost is much lower than you expect. If you look at costs for similar open models they are much lower than what you get by buying from anthropic, so that is the real cost basis I expect. It's likely Amazon is making a fucking killing though.
- SlinkyOnStairs 6mo ago
  While $5000 is a lot, the people who rack up close or just over a thousand "API equivalent cost" are pretty common. > Most likely the subscription inference cost is much lower than you expect. This is probably not true because they'd be screaming it off every rooftop were that the case. Same deal with the API inference. Even the "profitable on inference" claim is sourced back to hearsay of informal statements made by OpenAI/Anthropic staff. No formal announcements, nothing remotely of the "You can trust what I'm saying, because if I'm lying the SEC will have my head" sort. Yet making such statements would be invaluable. If Anthropic can demonstrate profitability before OpenAI, they could poach most of the funding. There's no reason to keep it a company secret. And API inference is only part of the total costs, not even bringing in training and ongoing fine-tuning. If they're not even profitable on inference, how could they hope to be profitable overall.
  - redsocksfan45 6mo ago
    [dead]
  - nielsole 6mo ago
    I don't know about SEC rules but the anthropic CEO said they have a 50%+ margin on API pricing.
    - stackskipton 6mo ago
      SEC rules means CEO cannot lie or deliberately hide the cost of something. 50%+ Margin statements have basically been "We are making 50% on delivering it." This does not include ANY of the costs of getting to this point, training, scraping, datacenters, people and so forth. They are basically saying "Oh yea, the cost of GAS in the car is only X so charging Y per mile is great margin" while ignoring maintenance, cost of acquiring the car and so forth.
      - postflopclarity 6mo ago
        but comparing your margin of charging to drive a mile to the price of gas makes a lot of sense? that is the only variable cost in the equation. training / scraping / people are all pretty much fixed costs.
        stackskipton 6mo ago
        Maybe, maybe not. None of us know since these companies are not public, we are not getting clear accounting.
      - paradoxyl 6mo ago
        That's a tad naive. CEOs can and have and often lied about everything: Sam Bankman-Fried, Elizabeth Holmes, Kenneth Lay - and hundreds if not thousands more. The SEC is a regulatory agency, not able to bring criminal charges. The above-named for the most part had to be prosecuted by the Department of Justice or sometimes state attorneys.
- PunchyHamster 6mo ago
  The "worst case" is probably someone just using their $200 account limits. So yeah, real cost is probably close to that
kiratp 6mo ago
At the full current retail API price. Business buyers are paying API prices, not subscription Disclosure: Work at Microsoft on AI
- an0malous 6mo ago
  Are your API prices profitable?

sandworm101 6mo ago

But that per-token cost is a total joke. All these companies are fighting to build market share in some future dominated by one or two AI ecosystems. It is musical chairs until someone creates the one ring to rule them all. So they are charging token amounts just to claim revenue as they burn through investor dollars. In short: per-token charges currently cover maybe 1% of the total costs in this field. To pay ongoing costs, and pay back investors, everyone will need to pay 100x or 1000x the current rates, likely for decades.

twoodfin 6mo ago
From the perspective of a deal like this, “total costs in the field” matter less than incremental cost per token served. The unit economics for today’s frontier models should be great, and this suggests Anthropic believes they’ll get better.
deaux 6mo ago
> In short: per-token charges currently cover maybe 1% of the total costs in this field There are plenty of seemingly informed people saying the exact opposite, so that's a lot of confidence you're talking with. I have a hard time believing it when we know what open weights models cost to run. And sure, there's training costs, but again many say inference costs are already above training costs.
postalrat 6mo ago
In a decade the cost of compute will be a tiny fraction of what it costs now. Specialized hardware will exist that will be cheap and efficient.
- bitmasher9 6mo ago
  The difference in the cost of compute between 2026 and 2036 won’t be nearly as large as the difference in the cost of compute between 2016 and 2026. Even at 2016 the slowdown in improvements was noticeable. We might see a one time bump in inference when we move off GPUs onto more limited and efficient dedicated hardware, but the sustained fast pace of improvements are far behind us.

svnt 6mo ago

And receiving investment from their vendor in exchange? When this is done in established companies it is typically called a kickback and directed toward one person, but in this case the whole thing is so incestuous the kickback goes straight to the top.

twoodfin 6mo ago
Is it crazy to imagine Anthropic can leverage short term cash flow now to build the models and products that will let them resell $100B in AWS infra with nice margins tomorrow? If Amazon believes that story they’d be crazy not to invest.
- svnt 6mo ago
  Yes I understand why the agreement exists, but that does not remove the circularity.