5 ms·

3D makes every part of your heat story much worse. You don't want to just stack layers... modern chips already have tons of layers and each one is an opportuni

3D makes every part of your heat story much worse. You don't want to just stack layers... modern chips already have tons of layers and each one is an opportunity to screw something up. Stacking of dies has its own problems but people are shipping stacked dies and have been for some time and as noted the heat is a major problem. Things will improve but not like they used to. It doesn't take that many more scaling nodes before gates are a handful of atoms and quantum effects dominate. But really gate sizes stopped shrinking a while ago - scaling has been terribly uneven for some time.

coryfklein 3y ago
I am not in the chip industry at all, but I understand that chips offload so much heat in part due to cramming as much as performance as possible into a small 2D space. If you increase the number of layers by 10-20x you now have much more "surface area" to work with. Could we see chip designs that operate at a much lower voltage (thus minimizing heat) and are "slower" as measured by chip frequency, but have greater overall bandwidth? Maybe a chip for servers that has 256 cores on it, each with their own caches? From a Moore's law perspective this would continue the transistor count doubling trend too.
- Engineering-MD 3y ago
  It’s a larger surface area only if you increase the area without increasing the heat generation. That is a heat sink. If you want all those layers computing and therefore producing heat, then you might as well keep it flat and have more fluid moving heat away. In a 3d space that fluid is going to be started between different layers and reach its carrying capacity of heat quicker.
- xenadu02 3y ago
  No not really. Dynamic switching power consumption is proportional to frequency which is why things slowed down dramatically once we hit 1Ghz. Of course you must reduce voltages to reduce current and losses... but at a certain point your transistors no longer switch between clear on/off states because the voltage swing is too small. This is a quantum effect of all semiconductors (related to their valence band energy IIRC). The tiny leakage this represents only makes your heat dissipation story much worse. So you end up trapped between two walls: higher voltage is required to drive the transistor between full on-off states but lower voltage is required to keep the gate from burning itself up. No matter what material you pick (silicon, germanium, etc) there is some minimum voltage required. Also your insulating layers become a problem as they shrink: quantum tunneling allows electrons to jump the insulating layer. There's a tradeoff here too... alternative oxide layers reduce leakage at the gate but have worse barrier constants and so are prone to more quantum tunneling (related to conduction band energy). Adding more layers just makes more problems, as each layer you stack risks mangling the layer below it. Plus the risk of introducing a defect. The more defects you have the worse your yields and the more expensive the chip gets. The short version is: we are at the point where no matter which direction we go some quantum effect kicks in to cheat us out of further optimizations in that direction.
trashtester 3y ago
Imagine if you can produce 3D compute substrate as 1 liter modules that can be built as Lego's. With holes to run water-cooling pipes through. If you have, say, 100l of such substrate for less than $10,000, with a density similar today's tech, just in 3 dimensions, you can probably afford to run most of it at very low frequencies to conserve energy, with only non-parallelizable operations running at GHz-level frequencies. Such a machine could still probably provide as much compute as a small datacenter can today. Now I don't know if it will take 20 years or 200 to be able to do something like that, but I'm certain that we have a looong way to go before we reach the physical limit to how much compute we can stuff into a rack.

coryfklein 3y ago

I am not in the chip industry at all, but I understand that chips offload so much heat in part due to cramming as much as performance as possible into a small 2D space. If you increase the number of layers by 10-20x you now have much more "surface area" to work with. Could we see chip designs that operate at a much lower voltage (thus minimizing heat) and are "slower" as measured by chip frequency, but have greater overall bandwidth? Maybe a chip for servers that has 256 cores on it, each with their own caches? From a Moore's law perspective this would continue the transistor count doubling trend too.

Engineering-MD 3y ago
It’s a larger surface area only if you increase the area without increasing the heat generation. That is a heat sink. If you want all those layers computing and therefore producing heat, then you might as well keep it flat and have more fluid moving heat away. In a 3d space that fluid is going to be started between different layers and reach its carrying capacity of heat quicker.
xenadu02 3y ago
No not really. Dynamic switching power consumption is proportional to frequency which is why things slowed down dramatically once we hit 1Ghz. Of course you must reduce voltages to reduce current and losses... but at a certain point your transistors no longer switch between clear on/off states because the voltage swing is too small. This is a quantum effect of all semiconductors (related to their valence band energy IIRC). The tiny leakage this represents only makes your heat dissipation story much worse. So you end up trapped between two walls: higher voltage is required to drive the transistor between full on-off states but lower voltage is required to keep the gate from burning itself up. No matter what material you pick (silicon, germanium, etc) there is some minimum voltage required. Also your insulating layers become a problem as they shrink: quantum tunneling allows electrons to jump the insulating layer. There's a tradeoff here too... alternative oxide layers reduce leakage at the gate but have worse barrier constants and so are prone to more quantum tunneling (related to conduction band energy). Adding more layers just makes more problems, as each layer you stack risks mangling the layer below it. Plus the risk of introducing a defect. The more defects you have the worse your yields and the more expensive the chip gets. The short version is: we are at the point where no matter which direction we go some quantum effect kicks in to cheat us out of further optimizations in that direction.

trashtester 3y ago

Imagine if you can produce 3D compute substrate as 1 liter modules that can be built as Lego's. With holes to run water-cooling pipes through. If you have, say, 100l of such substrate for less than $10,000, with a density similar today's tech, just in 3 dimensions, you can probably afford to run most of it at very low frequencies to conserve energy, with only non-parallelizable operations running at GHz-level frequencies. Such a machine could still probably provide as much compute as a small datacenter can today. Now I don't know if it will take 20 years or 200 to be able to do something like that, but I'm certain that we have a looong way to go before we reach the physical limit to how much compute we can stuff into a rack.