4 ms·

The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin,

The relatively slow depreciation of GPU value is an artifact of supply constraints. If you run fp4 inference and could choose freely between Hopper and a Rubin, the performance per watt would make the Hopper unattractive even if you paid zero for the hardware and only for the power. You can't get the Rubin, or even the Blackwell, so you will pay for the H100 but this won't last if fabs ramp up capacity.

tracker1 3mo ago
Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still. What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation. And how many facilities are years behind at this point already... all on various credit and dept swaps between all the involved companies... it's not just a balloon, it's a house of cards balanced on a balloon.
- cyanydeez 3mo ago
  supposedly, the next step is into fiber & optics. but I generally agree, people put a lot of faith in the exponential leaps vs the exponential space. You tell them we're not living on mars any time soon and they'll bring up christopher columbus.
  - wyre 3mo ago
    How would fiber and optics help if the bottleneck is computation efficiency and not data transfer?
    - cyanydeez 3mo ago
      the optics are doing the gates. literally, shine a light down a path and uts doing calcs.
- chuckadams 3mo ago
  The process node size in William the Conqueror's time was really off the charts.
  - rbanffy 3mo ago
    Plotting that backwards shows a transistor would be the size of a continent, at least.
    - lelanthran 2mo ago
      > Plotting that backwards shows a transistor would be the size of a continent, at least. Means that the computer would be the size of earth... maybe the mice really were onto something :-)
- rbanffy 3mo ago
  > since 2010 slower still The increase in PFLOPS/dollar has continued accelerating, a lot from process, but also a lot by simplifying the architecture- if you had placed an H100 worth of transistors on a CPU-like architecture, you wouldn’t reach the same peak performances.
- mickael-kerjean 3mo ago
  The infamous attention is all you need paper has this: "The Transformer ... reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs.", the p100 is now quite a bit under 100$ and that's from a time where nvidia was valued 100x less than it is today. Lets wait and see
- jbarberu 3mo ago
  Beating the GPU performance bump from 1080-1990 would be hard indeed...
  - tracker1 2mo ago
    lol, meant 1980....
hinkley 3mo ago
I wonder what shovels were worth after the Gold Rush faltered. Blacksmiths probably had all the scrap iron they could ever care for.
- NohatCoder 3mo ago
  Nah, "selling shovels" is mostly a metaphor, the amount of iron that went into mining equipment was insignificant, beyond some local demand peaks. The majority of the business was consumables. A key thing to understand about the gold rush is that it was not a major economic event, or at least nowhere near as big as the participants thought it would be, hence the tradegy. The AI gold rush is different in that there actually is a mountain of "shovels" large enough to flood the global market quite severely.
- oersted 3mo ago
  Railway tycoons were the real beneficiaries, and the Gold Rush led to broader organic economic development in the West, so railways remained relevant. Not sure what that means for the metaphor, but there you go.
  - hinkley 2mo ago
    Railways and Madams.
pianopatrick 3mo ago
Are fabs going to ramp up capacity? Wouldn't it make more business sense for them to just not ramp up capacity and enjoy the higher prices?
- fancyfredbot 3mo ago
  Yes they are going to ramp up capacity. It would only make business sense to do nothing if all your competitors were also doing nothing, which would probably require some level of illegal collision. If everyone ramps up then in the best case everyone has the same sized slice of a bigger pie. So in theory it makes business sense. But the more realistic possibility is that you end up with oversupply, crash the market and everyone loses. This is what normally seems to happen with DRAM.
  - deleted 3mo ago
    [deleted]
theptip 3mo ago
One does not simply “ramp up fab capacity”. This is all downstream of ASML who is putting out tens of EUV machines per year.

tracker1 3mo ago

Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still. What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation. And how many facilities are years behind at this point already... all on various credit and dept swaps between all the involved companies... it's not just a balloon, it's a house of cards balanced on a balloon.

cyanydeez 3mo ago
supposedly, the next step is into fiber & optics. but I generally agree, people put a lot of faith in the exponential leaps vs the exponential space. You tell them we're not living on mars any time soon and they'll bring up christopher columbus.
- wyre 3mo ago
  How would fiber and optics help if the bottleneck is computation efficiency and not data transfer?
  - cyanydeez 3mo ago
    the optics are doing the gates. literally, shine a light down a path and uts doing calcs.
chuckadams 3mo ago
The process node size in William the Conqueror's time was really off the charts.
- rbanffy 3mo ago
  Plotting that backwards shows a transistor would be the size of a continent, at least.
  - lelanthran 2mo ago
    > Plotting that backwards shows a transistor would be the size of a continent, at least. Means that the computer would be the size of earth... maybe the mice really were onto something :-)
rbanffy 3mo ago
> since 2010 slower still The increase in PFLOPS/dollar has continued accelerating, a lot from process, but also a lot by simplifying the architecture- if you had placed an H100 worth of transistors on a CPU-like architecture, you wouldn’t reach the same peak performances.
mickael-kerjean 3mo ago
The infamous attention is all you need paper has this: "The Transformer ... reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs.", the p100 is now quite a bit under 100$ and that's from a time where nvidia was valued 100x less than it is today. Lets wait and see
jbarberu 3mo ago
Beating the GPU performance bump from 1080-1990 would be hard indeed...
- tracker1 2mo ago
  lol, meant 1980....

hinkley 3mo ago

I wonder what shovels were worth after the Gold Rush faltered. Blacksmiths probably had all the scrap iron they could ever care for.

NohatCoder 3mo ago
Nah, "selling shovels" is mostly a metaphor, the amount of iron that went into mining equipment was insignificant, beyond some local demand peaks. The majority of the business was consumables. A key thing to understand about the gold rush is that it was not a major economic event, or at least nowhere near as big as the participants thought it would be, hence the tradegy. The AI gold rush is different in that there actually is a mountain of "shovels" large enough to flood the global market quite severely.
oersted 3mo ago
Railway tycoons were the real beneficiaries, and the Gold Rush led to broader organic economic development in the West, so railways remained relevant. Not sure what that means for the metaphor, but there you go.
- hinkley 2mo ago
  Railways and Madams.

pianopatrick 3mo ago

Are fabs going to ramp up capacity? Wouldn't it make more business sense for them to just not ramp up capacity and enjoy the higher prices?

fancyfredbot 3mo ago
Yes they are going to ramp up capacity. It would only make business sense to do nothing if all your competitors were also doing nothing, which would probably require some level of illegal collision. If everyone ramps up then in the best case everyone has the same sized slice of a bigger pie. So in theory it makes business sense. But the more realistic possibility is that you end up with oversupply, crash the market and everyone loses. This is what normally seems to happen with DRAM.
- deleted 3mo ago
  [deleted]

theptip 3mo ago

One does not simply “ramp up fab capacity”. This is all downstream of ASML who is putting out tens of EUV machines per year.