4 ms·

He sort of implies that “just better hardware” will peter out in the 2030s. I think he’s calling it at least 50 years too soon. Here’s why: (1) I think logic de

He sort of implies that “just better hardware” will peter out in the 2030s. I think he’s calling it at least 50 years too soon. Here’s why: (1) I think logic designers are still faffing about in term of optimizing their designs; and (2), I think there’s a lot of smart people thinking “incrementally” through what we’d consider paradigm shifts in HW implementation. That is, our fabs will just naturally segue into 3D, spintronics, etc. I think he even mentions 3D circuits? One thing a lot of people miss is that layout of the design is materially different in 3D vs 2D: in 2D layout is NP-hard (complete) without efficient polynomial approximations; in 3D layout is low-order polynomial. The reduction in layout complexity will allow us to design things that are unthinkable right now, due to layout constraints & wire congestion.

hobofan 5y ago
Aren't "big" 3D circuits unfeasible due to temperature limitations, though?
- a1369209993 5y ago
  > Aren't "big" 3D circuits unfeasible due to temperature limitations, though? Less so than you'd think, since (as long as you can keep leakage current under control) heat is only generated when the circuits are active (barring leakage current, when bits are being flipped). So you can have arbitrarily large amounts of increasingly-rarely-used circuitry for various purposes. The naive-but-easy-to-understand example would be having separate, optimal circuitry for each machine instruction - the total number of gates is O(N*M), but the number of gates activated on each clock cycle (and thus the amount of waste heat generated) is only O(N), so you can keep adding new, perfectly-hardware-accelerated instructions up to the limits of physical space. In practice it's more complicated and less of a nonissue, but it's not "big circuits are useful in proportion to their surface area, not their volume", it's more "big circuits are less useful than their volume alone would suggest". (You do hit physical limits like the Bekenstein bound[0] eventually, but that's far enough out that we mostly don't care yet.) 0: https://en.wikipedia.org/wiki/Bekenstein_bound https://en.wikipedia.org/wiki/Bekenstein_bound
nradov 5y ago
How are we going to cool those 3D chips?
- AprilArcus 5y ago
  could it be fabbed around a lattice of thermally conductive elements, like a menger sponge?
- thechao 5y ago
  The commenter above is correct: just stop toggling HW. We already do this to a great extent; we’re limited in the number of custom implementations because we can’t wire everything together. 3D chips will have a lot more “dark” logic than current chips, but will be orders-of-magnitude more efficient (& thus powerful) due to deep customization. Also, remember the argument of my timeline is ~50–80 years out from now.
  - nradov 5y ago
    Dark logic isn't doing any useful work so what's the point? Sure you can include specialized circuitry for a bunch of rare cases, but that won't lift overall system performance much and will kill manufacturing yields.
    - thechao 5y ago
      The GPU in your computer consists of mostly-dark highly specialized circuits; it certainly seems to improve my computing experience.
  - ClumsyPilot 5y ago
    This is totally wishfull thinking. Customised logix means customised code. The GPU has an h265 and h264 hardware video encoder that has no support on Mac and ia a bitch to get working on linux. Most software doesn't support it most software does not support GPU compute either, and we had that for a decade now. Fuck, we had SIMD in every CPU for like 30 years, and out of 50 most popular programming languages how many even support it? 5?
yjftsjthsd-h 5y ago
> in 2D layout is NP-hard (complete) without efficient polynomial approximations; in 3D layout is low-order polynomial Any chance you could explain to a novice why 3D is easier? To my naive intuition, it would have seemed like the more room to maneuver is offset by having more stuff to route.
- thechao 5y ago
  Try laying out a square with all the vertices connected in a plane: it’s not possible. You must move one of the wires “up” a layer. Which wire? Great layout minimizes layer transitions while also bunching together related HW blocks. The NP-completeness proof is related to work Knuth’s student (Plass?) did on laying out images in TeX.
- blt 5y ago
  Intuitively, we feel that larger solution spaces should make a problem harder because there are more possible solutions to consider. And of course this is true if exhaustive search is the only algorithm. But CS is full of problems where the smaller space is NP-hard and the larger one isn't. Integer linear programming is a prominent example. To resolve the intuition, we can think of the larger space as an "unconstrained" problem where the solution space is somehow natural and nice for the problem. From this perspective, the smaller space looks like an added constraint. Adding constraints usually makes problems harder.
  - dragontamer 5y ago
    > Adding constraints usually makes problems harder. Adding constraints can also make a problem easier. Which is why you've carefully worded your sentence in this manner :-) You clearly know about those edge cases, but I felt it necessary to elaborate on this point: Additional constraints can make a problem easier OR harder, depending on the problem and the search space. Its very non-intuitive.

hobofan 5y ago

Aren't "big" 3D circuits unfeasible due to temperature limitations, though?

a1369209993 5y ago
> Aren't "big" 3D circuits unfeasible due to temperature limitations, though? Less so than you'd think, since (as long as you can keep leakage current under control) heat is only generated when the circuits are active (barring leakage current, when bits are being flipped). So you can have arbitrarily large amounts of increasingly-rarely-used circuitry for various purposes. The naive-but-easy-to-understand example would be having separate, optimal circuitry for each machine instruction - the total number of gates is O(N*M), but the number of gates activated on each clock cycle (and thus the amount of waste heat generated) is only O(N), so you can keep adding new, perfectly-hardware-accelerated instructions up to the limits of physical space. In practice it's more complicated and less of a nonissue, but it's not "big circuits are useful in proportion to their surface area, not their volume", it's more "big circuits are less useful than their volume alone would suggest". (You do hit physical limits like the Bekenstein bound[0] eventually, but that's far enough out that we mostly don't care yet.) 0: https://en.wikipedia.org/wiki/Bekenstein_bound https://en.wikipedia.org/wiki/Bekenstein_bound

nradov 5y ago

How are we going to cool those 3D chips?

AprilArcus 5y ago
could it be fabbed around a lattice of thermally conductive elements, like a menger sponge?
thechao 5y ago
The commenter above is correct: just stop toggling HW. We already do this to a great extent; we’re limited in the number of custom implementations because we can’t wire everything together. 3D chips will have a lot more “dark” logic than current chips, but will be orders-of-magnitude more efficient (& thus powerful) due to deep customization. Also, remember the argument of my timeline is ~50–80 years out from now.
- nradov 5y ago
  Dark logic isn't doing any useful work so what's the point? Sure you can include specialized circuitry for a bunch of rare cases, but that won't lift overall system performance much and will kill manufacturing yields.
  - thechao 5y ago
    The GPU in your computer consists of mostly-dark highly specialized circuits; it certainly seems to improve my computing experience.
- ClumsyPilot 5y ago
  This is totally wishfull thinking. Customised logix means customised code. The GPU has an h265 and h264 hardware video encoder that has no support on Mac and ia a bitch to get working on linux. Most software doesn't support it most software does not support GPU compute either, and we had that for a decade now. Fuck, we had SIMD in every CPU for like 30 years, and out of 50 most popular programming languages how many even support it? 5?

yjftsjthsd-h 5y ago

> in 2D layout is NP-hard (complete) without efficient polynomial approximations; in 3D layout is low-order polynomial Any chance you could explain to a novice why 3D is easier? To my naive intuition, it would have seemed like the more room to maneuver is offset by having more stuff to route.

thechao 5y ago
Try laying out a square with all the vertices connected in a plane: it’s not possible. You must move one of the wires “up” a layer. Which wire? Great layout minimizes layer transitions while also bunching together related HW blocks. The NP-completeness proof is related to work Knuth’s student (Plass?) did on laying out images in TeX.
blt 5y ago
Intuitively, we feel that larger solution spaces should make a problem harder because there are more possible solutions to consider. And of course this is true if exhaustive search is the only algorithm. But CS is full of problems where the smaller space is NP-hard and the larger one isn't. Integer linear programming is a prominent example. To resolve the intuition, we can think of the larger space as an "unconstrained" problem where the solution space is somehow natural and nice for the problem. From this perspective, the smaller space looks like an added constraint. Adding constraints usually makes problems harder.
- dragontamer 5y ago
  > Adding constraints usually makes problems harder. Adding constraints can also make a problem easier. Which is why you've carefully worded your sentence in this manner :-) You clearly know about those edge cases, but I felt it necessary to elaborate on this point: Additional constraints can make a problem easier OR harder, depending on the problem and the search space. Its very non-intuitive.