3 ms·

Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 108

Not to mention, physical limits to lithography are slowing down significantly... so tech will continue to evolve more slowly... it'll never be the jump from 1080-1990 again, for example, even though 1990-2000 was pretty close, 2000-2010 much slower and since 2010 slower still. What's as or more weird is how much hardware is backordered, and how much live hardware is allocated, but waiting on facilities for operation. And how many facilities are years behind at this point already... all on various credit and dept swaps between all the involved companies... it's not just a balloon, it's a house of cards balanced on a balloon.

cyanydeez 3mo ago
supposedly, the next step is into fiber & optics. but I generally agree, people put a lot of faith in the exponential leaps vs the exponential space. You tell them we're not living on mars any time soon and they'll bring up christopher columbus.
- wyre 3mo ago
  How would fiber and optics help if the bottleneck is computation efficiency and not data transfer?
  - cyanydeez 3mo ago
    the optics are doing the gates. literally, shine a light down a path and uts doing calcs.
chuckadams 3mo ago
The process node size in William the Conqueror's time was really off the charts.
- rbanffy 3mo ago
  Plotting that backwards shows a transistor would be the size of a continent, at least.
  - lelanthran 3mo ago
    > Plotting that backwards shows a transistor would be the size of a continent, at least. Means that the computer would be the size of earth... maybe the mice really were onto something :-)
rbanffy 3mo ago
> since 2010 slower still The increase in PFLOPS/dollar has continued accelerating, a lot from process, but also a lot by simplifying the architecture- if you had placed an H100 worth of transistors on a CPU-like architecture, you wouldn’t reach the same peak performances.
mickael-kerjean 3mo ago
The infamous attention is all you need paper has this: "The Transformer ... reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs.", the p100 is now quite a bit under 100$ and that's from a time where nvidia was valued 100x less than it is today. Lets wait and see
jbarberu 3mo ago
Beating the GPU performance bump from 1080-1990 would be hard indeed...
- tracker1 2mo ago
  lol, meant 1980....

cyanydeez 3mo ago

supposedly, the next step is into fiber & optics. but I generally agree, people put a lot of faith in the exponential leaps vs the exponential space. You tell them we're not living on mars any time soon and they'll bring up christopher columbus.

wyre 3mo ago
How would fiber and optics help if the bottleneck is computation efficiency and not data transfer?
- cyanydeez 3mo ago
  the optics are doing the gates. literally, shine a light down a path and uts doing calcs.

chuckadams 3mo ago

The process node size in William the Conqueror's time was really off the charts.

rbanffy 3mo ago
Plotting that backwards shows a transistor would be the size of a continent, at least.
- lelanthran 3mo ago
  > Plotting that backwards shows a transistor would be the size of a continent, at least. Means that the computer would be the size of earth... maybe the mice really were onto something :-)

rbanffy 3mo ago

> since 2010 slower still The increase in PFLOPS/dollar has continued accelerating, a lot from process, but also a lot by simplifying the architecture- if you had placed an H100 worth of transistors on a CPU-like architecture, you wouldn’t reach the same peak performances.

mickael-kerjean 3mo ago

The infamous attention is all you need paper has this: "The Transformer ... reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs.", the p100 is now quite a bit under 100$ and that's from a time where nvidia was valued 100x less than it is today. Lets wait and see

jbarberu 3mo ago

Beating the GPU performance bump from 1080-1990 would be hard indeed...

tracker1 2mo ago
lol, meant 1980....