4 ms·

If the hardware can be used more efficiently to do even more work, the value of the hardware will hold since demand will not reduce but actually increase much f

If the hardware can be used more efficiently to do even more work, the value of the hardware will hold since demand will not reduce but actually increase much faster than supply. Efficiency going up tends to increase demand by much more than the efficiency-induced supply increase. Assuming that the world is hungry for as much AI as it can get. Which I think is true, we're nowhere near the peak of leveraging AI. We barely got started.

mitthrowaway2 2y ago
Perhaps, but this is not guaranteed. For example, demand might shift from datacenter to on-site inference when high-performing models can run locally on consumer hardware. Kind of like how demand for desktop PCs went down in the 2010s as mobile phones, laptops, and ipads became more capable, even though desktops also became even more capable. People found that running apps on their phone was good enough. Now perhaps everyone will want to run inference on-site for security and privacy, and so demand might shift away from big datacenters into desktops and consumer-grade hardware, and those datacenters will be left bidding each other down looking for workloads.
- AYBABTME 2y ago
  Inference is not where the majority of this CAPEX is used. And even if, monetization will no doubt discourage developers from dispensing the secret sauce to user controlled devices. So I posit that data centres inference is safe for a good while.
  - littlestymaar 2y ago
    > Inference is not where the majority of this CAPEX is used That's what's baffling with Deepseek's results: they spent very little on training (at least that's what they claim). If true, then it's a complete paradigm shift. And even if it's false, the more wide AI usage is, the bigger the share of inference will be, and inference cost will be the main cost driver at some point anyway.
    - m3kw9 2y ago
      You are looking at one model and also you do realize it isn’t even multimodal, also it shifts training compute to inference compute. They are shifting the paradigm for this architecture for LLMs, but I don’t think this is really new either.
      - littlestymaar 2y ago
        > it shifts training compute to inference compute No, this is the change introduced by o1, what's different with R1 is that its use of RL is fundamentally different (and cheaper) that what OpenAI did.
jdietrich 2y ago
>Efficiency going up tends to increase demand by much more than the efficiency-induced supply increase. https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox
- littlestymaar 2y ago
  The mainframes market disagrees.
  - m3kw9 2y ago
    Like the cloud compute we all use right now to serve most of what you use online?
    - littlestymaar 2y ago
      Ran thanks to PC parts, that's the point. IBM is nowhere close to Amazon or Azure in terms of cloud, and I suspect most of their customers run on x86_64 anyway.

mitthrowaway2 2y ago

Perhaps, but this is not guaranteed. For example, demand might shift from datacenter to on-site inference when high-performing models can run locally on consumer hardware. Kind of like how demand for desktop PCs went down in the 2010s as mobile phones, laptops, and ipads became more capable, even though desktops also became even more capable. People found that running apps on their phone was good enough. Now perhaps everyone will want to run inference on-site for security and privacy, and so demand might shift away from big datacenters into desktops and consumer-grade hardware, and those datacenters will be left bidding each other down looking for workloads.

AYBABTME 2y ago
Inference is not where the majority of this CAPEX is used. And even if, monetization will no doubt discourage developers from dispensing the secret sauce to user controlled devices. So I posit that data centres inference is safe for a good while.
- littlestymaar 2y ago
  > Inference is not where the majority of this CAPEX is used That's what's baffling with Deepseek's results: they spent very little on training (at least that's what they claim). If true, then it's a complete paradigm shift. And even if it's false, the more wide AI usage is, the bigger the share of inference will be, and inference cost will be the main cost driver at some point anyway.
  - m3kw9 2y ago
    You are looking at one model and also you do realize it isn’t even multimodal, also it shifts training compute to inference compute. They are shifting the paradigm for this architecture for LLMs, but I don’t think this is really new either.
    - littlestymaar 2y ago
      > it shifts training compute to inference compute No, this is the change introduced by o1, what's different with R1 is that its use of RL is fundamentally different (and cheaper) that what OpenAI did.

jdietrich 2y ago

>Efficiency going up tends to increase demand by much more than the efficiency-induced supply increase. https://en.wikipedia.org/wiki/Jevons_paradox https://en.wikipedia.org/wiki/Jevons_paradox

littlestymaar 2y ago
The mainframes market disagrees.
- m3kw9 2y ago
  Like the cloud compute we all use right now to serve most of what you use online?
  - littlestymaar 2y ago
    Ran thanks to PC parts, that's the point. IBM is nowhere close to Amazon or Azure in terms of cloud, and I suspect most of their customers run on x86_64 anyway.