3 ms·

If the demand is flat and GPU performance doubles every 2 years, you don't need that many years before sales essentially go to zero. In just 4, you're down 75%.

If the demand is flat and GPU performance doubles every 2 years, you don't need that many years before sales essentially go to zero. In just 4, you're down 75%. Demand can't "not go anywhere". Demand has to continue to go through the roof for sales to stay where they're at. For sales to continue increasing at the rate they were, demand would have to be insane.

dartos 2y ago
> and GPU performance doubles every 2 years That’s a big assumption.
- onlyrealcuzzo 2y ago
  See Huang's Law: https://en.wikipedia.org/wiki/Huang%27s_law https://en.wikipedia.org/wiki/Huang%27s_law
  - dartos 2y ago
    From that same Wikipedia page > There has been criticism. Journalist Joel Hruska writing in ExtremeTech in 2020 said "there is no such thing as Huang's Law", calling it an "illusion" that rests on the gains made possible by Moore's law; and that it is too soon to determine a law exists.[9] The research nonprofit Epoch has found that, between 2006 and 2021, GPU price performance (in terms of FLOPS/$) has tended to double approximately every 2.5 years, much slower than predicted by Huang's law Regardless, there is a ceiling to how fast hardware can be. Can’t just double forever.
myrmidon 2y ago
Demand being independent of price/performance is a oversimplifying assumption that is very obviously flawed, and this does not even require continuously discovering novel applications -- there is already plenty of stuff that would get viable with lower costs/better availability of computing power right now.
danielmarkbruce 2y ago
CPU demand didn't drop to zero during it's heyday of doubling performance every two years. There is a decent chance we ramp to 10+ trillion $$ of demand annually for inference here in the next decade or so. That will drive a lot of GPU sales.
- onlyrealcuzzo 2y ago
  > CPU demand didn't drop to zero during it's heyday of doubling performance every two years. Because consumption "demand" didn't "not go anywhere". It went through the roof. My point is: you can't say that just because current usage is "here to stay" that sales at this level are "here to stay". Usage has to increase dramatically to keep sales flat.
  - danielmarkbruce 2y ago
    ok, i see, fair point.
janalsncm 2y ago
One of the reasons ML requires so many GPUs, besides the dataset sizes and the advantages of bigger models is the fact that ML is fundamentally unpredictable: it’s hard to debug, and hard to know if something you try will work before you try it. So if you told my company they could run 2x the number of experiments for the same price they would do it because it doubles the number of hyperparameter settings we can try simultaneously. If that means fine tuning BERT in a day vs 2, that’s a huge increase in iteration velocity. It means we can work on more backlog projects simultaneously. It means engineers aren’t elbowing each other for GPU time (ok that is optimistic).

dartos 2y ago

> and GPU performance doubles every 2 years That’s a big assumption.

onlyrealcuzzo 2y ago
See Huang's Law: https://en.wikipedia.org/wiki/Huang%27s_law https://en.wikipedia.org/wiki/Huang%27s_law
- dartos 2y ago
  From that same Wikipedia page > There has been criticism. Journalist Joel Hruska writing in ExtremeTech in 2020 said "there is no such thing as Huang's Law", calling it an "illusion" that rests on the gains made possible by Moore's law; and that it is too soon to determine a law exists.[9] The research nonprofit Epoch has found that, between 2006 and 2021, GPU price performance (in terms of FLOPS/$) has tended to double approximately every 2.5 years, much slower than predicted by Huang's law Regardless, there is a ceiling to how fast hardware can be. Can’t just double forever.

myrmidon 2y ago

Demand being independent of price/performance is a oversimplifying assumption that is very obviously flawed, and this does not even require continuously discovering novel applications -- there is already plenty of stuff that would get viable with lower costs/better availability of computing power right now.

danielmarkbruce 2y ago

CPU demand didn't drop to zero during it's heyday of doubling performance every two years. There is a decent chance we ramp to 10+ trillion $$ of demand annually for inference here in the next decade or so. That will drive a lot of GPU sales.

onlyrealcuzzo 2y ago
> CPU demand didn't drop to zero during it's heyday of doubling performance every two years. Because consumption "demand" didn't "not go anywhere". It went through the roof. My point is: you can't say that just because current usage is "here to stay" that sales at this level are "here to stay". Usage has to increase dramatically to keep sales flat.
- danielmarkbruce 2y ago
  ok, i see, fair point.

janalsncm 2y ago

One of the reasons ML requires so many GPUs, besides the dataset sizes and the advantages of bigger models is the fact that ML is fundamentally unpredictable: it’s hard to debug, and hard to know if something you try will work before you try it. So if you told my company they could run 2x the number of experiments for the same price they would do it because it doubles the number of hyperparameter settings we can try simultaneously. If that means fine tuning BERT in a day vs 2, that’s a huge increase in iteration velocity. It means we can work on more backlog projects simultaneously. It means engineers aren’t elbowing each other for GPU time (ok that is optimistic).