4 ms·

> Unfortunately, Feynman’s “room at the bottom” is no longer plentiful. There's plenty of room at the bottom. Another million times the transistors is hardly i

> Unfortunately, Feynman’s “room at the bottom” is no longer plentiful. There's plenty of room at the bottom. Another million times the transistors is hardly implausible; a thousand times is practically a given. > Although other manufacturers continued to miniaturize—for example, with the Samsung Exynos 9825 (8) and the Apple A13 Bionic (9)—they also failed to meet the Moore cadence. This just factually isn't fair; transistor density has been on an unwavering straight line since 1970. https://docs.google.com/spreadsheets/d/1NNOqbJfcISFyMd0EsSrhppW7PT6GCfnrVGhxhLA5PVw https://docs.google.com/spreadsheets/d/1NNOqbJfcISFyMd0EsSrh...

deepnotderp 6y ago
To be fair, it's not necessarily clear what to do with those transistors. When people think of Moore's Law, they usually think of all of CMOS scaling, Dennard scaling, faster clock speeds, etc Power may soon become the bottleneck as it's not clear that energy per op is going down as fast as transistors are going up
- wongarsu 6y ago
  Power is the bottle neck right now. We have 16 core processors were each core is capable of 4.7GHz individually. But we can't run all of the cores simultaneously at those speeds because we have trouble with power delivery and especially with dealing with the waste heat that comes with using that much power. The only way we can really use all that transistor density today is to artificially limit clock speeds to conserve power.
  - pmoriarty 6y ago
    There's plenty of heat at the bottom.
ncmncm 6y ago
Imagine being a CPU architect and being told to find a good use for yet another billion transistors. It is hard to get much benefit from the N+1th multiplier when the ones you have are already starved for input, so it's not hard to see why now we mainly get more cores and cache. Cache is good because most of it is idle at every moment, so doesn't produce much heat. Cores are easy to count, and people will pay for them even if whenever they all get busy, you have to slow the clock after a moment. Often a moment is enough. Productive uses for a billion transistors that can be mostly turned off most of the time are hard to find.
- ladberg 6y ago
  I feel like more and more transistors will be dedicated to machine learning tasks. You'll eventually have phones and computers constantly running neural networks to accomplish advanced but relatively mundane tasks (e.g. text-to-speech, speech-to-text, facial recognition, human-seeming AI in videogames, neural rendering (see NeRF from Berkeley), and tons of other random stuff. All of it is parallel and benefits from a sheer increase in transistors and density. Maybe in a few generations we'll see chips that are 99% ML compute and 1% traditional CPU.
  - ncmncm 6y ago
    In those chips, I think practically all the transistors have to be active all the time, so the density must be quite limited unless you have really serious cooling capacity.
    - ladberg 6y ago
      I feel like there's a ton of unexplored area in the hardware space for extremely low power circuits used for ML. As we know, you can do ML with very low precision, which is why chips that can do fast INT8 and F16 are getting more popular. However, these lower power chips are still designed like classical computers and are deterministic. What if you had something like a few thousand analog multipliers in parallel computing matrix products? An analog multiply can use far less power than a digital one at the expense of accuracy and determinism, which might not matter for some ML tasks.
ksec 6y ago
Yes, because it only started levelling off a bit in the past few years. Which means they dont show up in the long view, and especially in log scale. TSMC 5nm, 3nm, 2nm, none of them will hit 2x density improvement compared to its previous node within 24 / 30 months period.
- Veedrac 6y ago
  I doubt we should read too much into the difference between 1.8x and 2x. If there's a long-term trend, it'll take more than one data point to see it. TSMC 3nm does sound less ambitious, but it's also supposedly coming early, and sounds too way early to call for sure, at least for anyone not under NDA.

deepnotderp 6y ago

To be fair, it's not necessarily clear what to do with those transistors. When people think of Moore's Law, they usually think of all of CMOS scaling, Dennard scaling, faster clock speeds, etc Power may soon become the bottleneck as it's not clear that energy per op is going down as fast as transistors are going up

wongarsu 6y ago
Power is the bottle neck right now. We have 16 core processors were each core is capable of 4.7GHz individually. But we can't run all of the cores simultaneously at those speeds because we have trouble with power delivery and especially with dealing with the waste heat that comes with using that much power. The only way we can really use all that transistor density today is to artificially limit clock speeds to conserve power.
- pmoriarty 6y ago
  There's plenty of heat at the bottom.

ncmncm 6y ago

Imagine being a CPU architect and being told to find a good use for yet another billion transistors. It is hard to get much benefit from the N+1th multiplier when the ones you have are already starved for input, so it's not hard to see why now we mainly get more cores and cache. Cache is good because most of it is idle at every moment, so doesn't produce much heat. Cores are easy to count, and people will pay for them even if whenever they all get busy, you have to slow the clock after a moment. Often a moment is enough. Productive uses for a billion transistors that can be mostly turned off most of the time are hard to find.

ladberg 6y ago
I feel like more and more transistors will be dedicated to machine learning tasks. You'll eventually have phones and computers constantly running neural networks to accomplish advanced but relatively mundane tasks (e.g. text-to-speech, speech-to-text, facial recognition, human-seeming AI in videogames, neural rendering (see NeRF from Berkeley), and tons of other random stuff. All of it is parallel and benefits from a sheer increase in transistors and density. Maybe in a few generations we'll see chips that are 99% ML compute and 1% traditional CPU.
- ncmncm 6y ago
  In those chips, I think practically all the transistors have to be active all the time, so the density must be quite limited unless you have really serious cooling capacity.
  - ladberg 6y ago
    I feel like there's a ton of unexplored area in the hardware space for extremely low power circuits used for ML. As we know, you can do ML with very low precision, which is why chips that can do fast INT8 and F16 are getting more popular. However, these lower power chips are still designed like classical computers and are deterministic. What if you had something like a few thousand analog multipliers in parallel computing matrix products? An analog multiply can use far less power than a digital one at the expense of accuracy and determinism, which might not matter for some ML tasks.

ksec 6y ago

Yes, because it only started levelling off a bit in the past few years. Which means they dont show up in the long view, and especially in log scale. TSMC 5nm, 3nm, 2nm, none of them will hit 2x density improvement compared to its previous node within 24 / 30 months period.

Veedrac 6y ago
I doubt we should read too much into the difference between 1.8x and 2x. If there's a long-term trend, it'll take more than one data point to see it. TSMC 3nm does sound less ambitious, but it's also supposedly coming early, and sounds too way early to call for sure, at least for anyone not under NDA.