4 ms·

I've often pondered: why does this problem submit to sustained exponential improvement for such a long time? There doesn't seem to be any physical characteristi

I've often pondered: why does this problem submit to sustained exponential improvement for such a long time? There doesn't seem to be any physical characteristics of miniaturization that map directly to exponential functions, in the same way as many rate-of-change problems do when you first learn differential equations. I've largely concluded that it's a combination of three things. 1. As Fenyman said, there's plenty of room at the bottom. This is why so many orders of magnitude improvement have been available. 2. As Turing proved, computers are general purpose. A consequence of this is we have only a small number of problems to solve that apply to all software, and so as the industry has grown exponentially, we've been able to put concentrated resources into making hardware better. 3. Exponential growth is simply the best we can do as human innovators. Exponential growth isn't a property of the problem; it's a limit of our innovation. We are able to hit the limit due to the economic consequences of computers being general purpose, and able to sustain it due to there being plenty of room at the bottom.

chii 3y ago
computers are also used for their own future designs. I would imagine that faster computers could help design better computers in the future, thus there's some level of exponential growth there. However, i think computer aided designs are reaching the limits of what help it gives, and the bottleneck is human intelligence and material engineering difficulties (that cannot use the help of computers). that's why moores law was more exponential early on in the miniaturisation of chips, but it's slowed down more and more - till one day it's gonna be sublinear. Of course, this doesn't preclude a new breakthrough in a different medium (optical computers, rather than electronic?), or a different form like quantum.
hnaccount_rng 3y ago
I think there are two components that are required for exponential growth, with the second one being a corollary of the other: 1) There needs to be some feedback from the effect of the process back to the constraints on the process 2) There must not be a significant external limitation where 2) implies that the total (economic) effort expended towards the process is small compared to the total (economic) benefit derived from the process. As the sibling comment already states: We could not produce modern chips without (at least modernish) chips to execute the design tools. But I'm not sure how critical that actually is for the exponential growth (extrapolating from the inefficiency of all other software written today). The interesting question is, whether we are close to a limit yet. Which, I think, translates to the question, whether we are already close enough to the edge of what is physically possible to have some significant economic cost (with respect to the derived economic benefit) What makes me think right now is your 3) .. What would a (sustained) better-than-exponential growth require and how would we identify such a thing/opportunity
jakey_bakey 3y ago
I just realised, another big factor here: Innovations in transistor design help to shrink it on one dimension (e.g. the semiconductor size gap). This translates to a silicon wafer in two dimensions. Therefore, for every factor (n) decrease we get in transistor length, we can fit (n^2) more transistors on a chip. Therefore we don't need (as) exponential amounts of innovation to get exponential outcomes. Maybe I'm being crazy, Babbage you can correct me if this is way off :)
calf 3y ago
10 x 10 = 100 and 14 x 14 = 196. So in order to double the transistors we only need to shrink by 70% each time. So I wonder if the emphasis on exponentials is misleading. A geometric decrease in feature size is sufficient, there were lots of atoms to work with while making wires ever smaller, and up till 2010 or so the regime of Dennard scaling gave shrinking transistors the very nice property of not affecting power consumption. I know the article goes into more complex techno-economic reasons (and I guess that is what you are really discussing as well), but most IEEE papers cite Dennard scaling as a critical enabling driver. The end of Dennard is what made the IDRS roadmap identify an "end" to Moore's law.
- vanderZwan 3y ago
  > shrink by 70% each time Quick aside, I get what you are saying (that is: multiply the original size by 0.7), but as a non-native speaker of English: isn't "shrink by" the opposite of "shrink to", and wouldn't this wording technically imply shrinking to 30% of the original size?
  - calf 3y ago
    Yes correct! I should've written "shrink to 70%" or something like that. It was a thinko.
- furyofantares 3y ago
  If you're shrinking to a fixed(ish) percent at a fixed(ish) interval, I'd call that exponential. The difference between exponential and geometric is really just continuous vs discreet, isn't it?

chii 3y ago

computers are also used for their own future designs. I would imagine that faster computers could help design better computers in the future, thus there's some level of exponential growth there. However, i think computer aided designs are reaching the limits of what help it gives, and the bottleneck is human intelligence and material engineering difficulties (that cannot use the help of computers). that's why moores law was more exponential early on in the miniaturisation of chips, but it's slowed down more and more - till one day it's gonna be sublinear. Of course, this doesn't preclude a new breakthrough in a different medium (optical computers, rather than electronic?), or a different form like quantum.

hnaccount_rng 3y ago

I think there are two components that are required for exponential growth, with the second one being a corollary of the other: 1) There needs to be some feedback from the effect of the process back to the constraints on the process 2) There must not be a significant external limitation where 2) implies that the total (economic) effort expended towards the process is small compared to the total (economic) benefit derived from the process. As the sibling comment already states: We could not produce modern chips without (at least modernish) chips to execute the design tools. But I'm not sure how critical that actually is for the exponential growth (extrapolating from the inefficiency of all other software written today). The interesting question is, whether we are close to a limit yet. Which, I think, translates to the question, whether we are already close enough to the edge of what is physically possible to have some significant economic cost (with respect to the derived economic benefit) What makes me think right now is your 3) .. What would a (sustained) better-than-exponential growth require and how would we identify such a thing/opportunity

jakey_bakey 3y ago

I just realised, another big factor here: Innovations in transistor design help to shrink it on one dimension (e.g. the semiconductor size gap). This translates to a silicon wafer in two dimensions. Therefore, for every factor (n) decrease we get in transistor length, we can fit (n^2) more transistors on a chip. Therefore we don't need (as) exponential amounts of innovation to get exponential outcomes. Maybe I'm being crazy, Babbage you can correct me if this is way off :)

calf 3y ago

10 x 10 = 100 and 14 x 14 = 196. So in order to double the transistors we only need to shrink by 70% each time. So I wonder if the emphasis on exponentials is misleading. A geometric decrease in feature size is sufficient, there were lots of atoms to work with while making wires ever smaller, and up till 2010 or so the regime of Dennard scaling gave shrinking transistors the very nice property of not affecting power consumption. I know the article goes into more complex techno-economic reasons (and I guess that is what you are really discussing as well), but most IEEE papers cite Dennard scaling as a critical enabling driver. The end of Dennard is what made the IDRS roadmap identify an "end" to Moore's law.

vanderZwan 3y ago
> shrink by 70% each time Quick aside, I get what you are saying (that is: multiply the original size by 0.7), but as a non-native speaker of English: isn't "shrink by" the opposite of "shrink to", and wouldn't this wording technically imply shrinking to 30% of the original size?
- calf 3y ago
  Yes correct! I should've written "shrink to 70%" or something like that. It was a thinko.
furyofantares 3y ago
If you're shrinking to a fixed(ish) percent at a fixed(ish) interval, I'd call that exponential. The difference between exponential and geometric is really just continuous vs discreet, isn't it?