4 ms·

> But when you just look at it from an inference perspective, looking at these data centres like token factories makes sense. So if you ignore the majority of

> But when you just look at it from an inference perspective, looking at these data centres like token factories makes sense. So if you ignore the majority of the costs, then it makes sense. Opus 4.5 was released on November 25, 2025. That is less than 2 months ago. When they stop training new models, then we can forget about training costs.

sothatsit 9mo ago
Inference costs scale linearly with usage. R&D expenses do not. That's not to mention that Dario Amodei has said that their models actually have a good return, even when accounting for training costs [0]. [0] https://youtu.be/GcqQ1ebBqkc?si=Vs2R4taIhj3uwIyj&t=1088 https://youtu.be/GcqQ1ebBqkc?si=Vs2R4taIhj3uwIyj&t=1088
- Eisenstein 9mo ago
  > Inference costs scale linearly with usage. R&D expenses do not. Do we know this is true for AI?
  - sothatsit 9mo ago
    Yes. R&D is guaranteed to fall as a percentage of costs eventually. The only question is when, and there is also a question of who is still solvent when that time comes. It is competition and an innovation race that keeps it so high, and it won't stay so high forever. Either rising revenues or falling competition will bring R&D costs down as a percentage of revenue at some point.
    - saagarjha 9mo ago
      Yes, but eventually may be longer than the market can hold out. So far R&D expenses have skyrocketed and it does not look like that will be changing anytime soon.
      - sothatsit 9mo ago
        That's why it is a bet, and not a sure thing.
  - brookst 9mo ago
    It’s pretty much the definition of fixed costs versus variable costs. You spend the same amount on R&D whether you have one hobbyist user or 90% market share.
fc417fc802 9mo ago
I'm not taking a side here - I don't know enough - but it's an interesting line of reasoning. So I'll ask, how is that any different than fabs? From what I understand R&D is absurd and upgrading to a new node is even more absurd. The resulting chips sell for chump change on a per unit basis (analogous to tokens). But somehow it all works out. Well, sort of. The bleeding edge companies kept dropping out until you could count them on one hand at this point. At first glance it seems like the analogy might fit?
- short_sells_poo 9mo ago
  Someone else mentioned it elsewhere in this thread, and I believe this is the crux of the issue: this is all predicated in the actual end users finding enough benefit in LLM services to keep the gravy train going. It's irrelevant how scalable and profitable the shovel makes are, to keep this business afloat long term, the shovelers - ie the end users - have to make money using the shovesl. Those expectations are currently ridiculously inflated. Far beyond anything in the past. Invariably, there's going to be a collapse in the hype, the bubble will burst, and an investment deleveraging will remove a lot of money from the space in a short period of time. The bigger the bubble, the more painful and less survivable this event will be.

sothatsit 9mo ago

Inference costs scale linearly with usage. R&D expenses do not. That's not to mention that Dario Amodei has said that their models actually have a good return, even when accounting for training costs [0]. [0] https://youtu.be/GcqQ1ebBqkc?si=Vs2R4taIhj3uwIyj&t=1088 https://youtu.be/GcqQ1ebBqkc?si=Vs2R4taIhj3uwIyj&t=1088

Eisenstein 9mo ago
> Inference costs scale linearly with usage. R&D expenses do not. Do we know this is true for AI?
- sothatsit 9mo ago
  Yes. R&D is guaranteed to fall as a percentage of costs eventually. The only question is when, and there is also a question of who is still solvent when that time comes. It is competition and an innovation race that keeps it so high, and it won't stay so high forever. Either rising revenues or falling competition will bring R&D costs down as a percentage of revenue at some point.
  - saagarjha 9mo ago
    Yes, but eventually may be longer than the market can hold out. So far R&D expenses have skyrocketed and it does not look like that will be changing anytime soon.
    - sothatsit 9mo ago
      That's why it is a bet, and not a sure thing.
- brookst 9mo ago
  It’s pretty much the definition of fixed costs versus variable costs. You spend the same amount on R&D whether you have one hobbyist user or 90% market share.

fc417fc802 9mo ago

I'm not taking a side here - I don't know enough - but it's an interesting line of reasoning. So I'll ask, how is that any different than fabs? From what I understand R&D is absurd and upgrading to a new node is even more absurd. The resulting chips sell for chump change on a per unit basis (analogous to tokens). But somehow it all works out. Well, sort of. The bleeding edge companies kept dropping out until you could count them on one hand at this point. At first glance it seems like the analogy might fit?

short_sells_poo 9mo ago
Someone else mentioned it elsewhere in this thread, and I believe this is the crux of the issue: this is all predicated in the actual end users finding enough benefit in LLM services to keep the gravy train going. It's irrelevant how scalable and profitable the shovel makes are, to keep this business afloat long term, the shovelers - ie the end users - have to make money using the shovesl. Those expectations are currently ridiculously inflated. Far beyond anything in the past. Invariably, there's going to be a collapse in the hype, the bubble will burst, and an investment deleveraging will remove a lot of money from the space in a short period of time. The bigger the bubble, the more painful and less survivable this event will be.