3 ms·

The unit economics of these models and APIs are really ugly. Those saying they are not losing money on inference likely are only doing so when making up funky n

by JCM9 1y ago

The unit economics of these models and APIs are really ugly. Those saying they are not losing money on inference likely are only doing so when making up funky non-GAAP accounting thinking. It’s the old “we’re making money when you ignore all the places we’re spending money” argument. When you factor in the R&D costs required to make these models and the very limited lifespan of a model (and thus extremely high capital investment depreciation rate) the numbers are pretty nasty.

ACCount37 1y ago
R&D costs don't have to be sustainable. If the trend of staggering AI performance gains stops, you can afford to cut down on R&D and remain competitive. If it doesn't, you hit AGI and break the world economy - with a hope that it'll break in your favor.
- JCM9 1y ago
  If the performance gains stop then everything becomes a commodity and then a race to the bottom on pricing. It’s not a pretty picture.
  - ACCount37 1y ago
    And companies that already invested into massive amounts of compute for AI training? They're positioned to win that race. They get to convert that compute to inference compute, pivot their R&D towards "figure out how to make inference cheaper" and leverage all the economies of scale.
    - deleted 1y ago
      [deleted]
stego-tech 1y ago
Very well said. For what it’s worth, it’s the exact same “logic” public cloud providers also used to get customers off the hardware they already owned and onto hardware they’d rent forever. Some of the most successful businesses out there - tobacco companies, casinos, SaaS, fossil fuel providers, car companies, cloud providers, etc - are masterfully adept at weaving narratives that keep their customers focused on short or mid-term results over long-term costs, and AI is no different. Sure, if all you ever look at are the token costs, the inferencing costs at the edge, then the narrative that this will never skyrocket in price and the gates to the walled garden won’t ever close seems believable. Once you factor in the R&D, the training, the data acquisition, the datacenters, the electricity and water and real estate and lobbying and shareholder returns… It’ll be the most expensive product your company pays for, per seat, by miles, once the subsidy period ends and the real bills come due. Even open-weight models are likely to evaporate or shift to some sort of Folding@Home type distributed training model to keep costs low.

by JCM9 1y ago

ACCount37 1y ago

R&D costs don't have to be sustainable. If the trend of staggering AI performance gains stops, you can afford to cut down on R&D and remain competitive. If it doesn't, you hit AGI and break the world economy - with a hope that it'll break in your favor.

JCM9 1y ago
If the performance gains stop then everything becomes a commodity and then a race to the bottom on pricing. It’s not a pretty picture.
- ACCount37 1y ago
  And companies that already invested into massive amounts of compute for AI training? They're positioned to win that race. They get to convert that compute to inference compute, pivot their R&D towards "figure out how to make inference cheaper" and leverage all the economies of scale.
  - deleted 1y ago
    [deleted]

stego-tech 1y ago

Very well said. For what it’s worth, it’s the exact same “logic” public cloud providers also used to get customers off the hardware they already owned and onto hardware they’d rent forever. Some of the most successful businesses out there - tobacco companies, casinos, SaaS, fossil fuel providers, car companies, cloud providers, etc - are masterfully adept at weaving narratives that keep their customers focused on short or mid-term results over long-term costs, and AI is no different. Sure, if all you ever look at are the token costs, the inferencing costs at the edge, then the narrative that this will never skyrocket in price and the gates to the walled garden won’t ever close seems believable. Once you factor in the R&D, the training, the data acquisition, the datacenters, the electricity and water and real estate and lobbying and shareholder returns… It’ll be the most expensive product your company pays for, per seat, by miles, once the subsidy period ends and the real bills come due. Even open-weight models are likely to evaporate or shift to some sort of Folding@Home type distributed training model to keep costs low.