12 ms·

Moore on Moore – The past, present and uncertain future of Moore's Law

88 points by klelatti 3y ago

jdontillman 3y ago
Also see my article "The Mechanics of Moore's Law" for an analysis in terms of economic feedback loops with usual adaptive characteristics. https://till.com/articles/MooresLaw https://till.com/articles/MooresLaw
- an_aparallel 3y ago
  i read this comment...and thought "surely not thru-zero quadrature VCO donald tillman"? surely enough...you were :)
  - jdontillman 3y ago
    Hey ho!
gfody 3y ago
Jim Keller gave a talk at Berkeley titled "Moore's Law is not dead" (2019) where he summarized a bunch of up and coming technologies that will enable the continued scaling of transistor density - https://www.youtube.com/watch?v=oIG9ztQw2Gc https://www.youtube.com/watch?v=oIG9ztQw2Gc
- hinkley 3y ago
  How many of those are invalidated by Specter? We’ve been building chips that are literally impossibly fast for half of the last decade at least.
  - GuB-42 3y ago
    Spectre is just about timing attacks on branch prediction. It has nothing to do with transistor density, and it can be remediated with a reasonable impact on performance, that is if remediation is necessary. These vulnerabilities are also hard to exploit in real life and not always relevant. Rowhammer has to do with RAM density, but again, hard to exploit and there are mitigations.
  - renewiltord 3y ago
    Not that important depending on use-case. Lots of people have been running for years with `mitigations=off`. If someone could steal their crypto, they would have. https://news.ycombinator.com/item?id=32077583 https://news.ycombinator.com/item?id=32077583 I run with `mitigations=off` in offline cluster.
- xw390111 3y ago
  Moore's Law has been entirely coopted by the media and, as such, lost all meaning and predictive power, which is what made it famous in the first place. Once upon a time it was a useful tool to plan programs with. So outside of popular culture, it is dead...
Legend2440 3y ago
>the end of Dennard Scaling around 2006 has meant that the rate of increase in performance has slowed even as Moore’s Law has continued. This is why parallel compute (multi core/GPUs) has taken off since 2006 - heat puts a cap on clock speed, so the only option is to do more operations per cycle. Even some microcontrollers are dual-core now. I don't see this trend stopping. Massive parallelism is the future of computing.
- arein3 3y ago
  We need moore's nvidia law. Price increases 50% every 2 years
  - sigmoid10 3y ago
    That's just what monopolies get you. If they had serious competitors, they couldn't charge a 1000% markup.
    - p1necone 3y ago
      Intel's ARC gpu's keep improving with driver updates, if they ship another generation I can see them eventually properly disrupting the current duopoly.
      - sigmoid10 3y ago
        ARC is still very far away from even the lower end of what Nvidia offers. And on the enterprise side Intel is also well behind with their Gaudi chips. I'd love cheaper GPUs as much as everyone, but I don't see anyone who would could be considered a threat to Nvidia's position over the next years.
        p1necone 3y ago
        "very far away" isn't accurate - the A770 is pretty close to a 4060 - perfectly adequate for 1080p/1440p 60fps gaming on high settings in modern titles.
zamadatix 3y ago
Some discussion around applying Ngrams to your investigation: It's really easy to make ngrams say something wrong by accident. I don't think these ones turned out too poorly for what the conclusions were trying trying to convey but for modern terms there are two things you want to keep an eye out for or adapt for: 1) Enable case insensitivity, particularly for things like "transistor" which are rarely capitalized or "integrated circuit" but even for "should be capitalized things" like "Moore's Law" vs "Moore's law". 2) (Especially on things spanning less than a century) check the impact of smoothing. It can "smooth out" the introduction of term to appear to take off years early or hide the true peaks/variabilities because they only lasted a couple of years instead of decades like the default smoothing might be better suited for. 3) (Not as applicable here) make sure the terms are popular enough you're not just comparing noise. 4) Adjust your range so the data is properly scaled. If your term was coined around the 1970s (and comparatively uncommon prior to that)then the 1870s shouldn't be midway through your graph. 5) If various case insensitive spellings create a messy set of graph lines adding in a less popular term like "exaflops" will cause them to be merged and you can extract that line. 6) Be careful comparing things that can be pluralized or otherwise modified with things that aren't. E.g. "Moore's Law" vs "Integrated Circuit[s]?" Combining some of these, notes like "Here is a further ngram of the ‘End of Moore’s Law’ which has clearly been a live topic of discussion since the late 1990s." actually look to be a bit off when you switch the graph to clue in on the last 30 years, set smoothing to 0, and enable case sensitivity to catch "[Ee]nd of [Mm]oore's [Ll]aw". Case insensitivity changes the shape of the humps and 2019 ends up significantly higher than the peak in the early 2000s. Shrinking the x-axis and not smoothing out the peak to be wider than it is shows there is very little in the way of the end of Moore's Law in the late 90's, only really starting to take off in 99 and not peaking until 2003. A similar story repeats for the opening graph, the gap between variants of "Moore's Law" and variants of "Integrated Circuit" is more than double the frequency, adding in that variants of "Integrate Circuits" are even more than either and counted separately. Overall the points turn out right enough (a few years difference in when the phrase found its first wave doesn't impact the story, nor does how frequent the use of Moore's law is vs ICs as much as the general rise of one and decline of the other) but I always want to highlight how the friendly interface of Google's Ngram viewer can easily lead you to the wrong conclusion on the data. I'd also like to point out the above tips are neither exhaustive or universally applicable to any type of search. Sometimes you want more smoothing, sometimes you do want an exact version of a phrase, sometimes you do want to show the term was used long before the modern meaning, and sometimes there are additional things that can bite you. E.g. one I was worried about here was "Moores Law" typos excluding the ' but it turned out to not be common at all. You really don't know until you check each possibility though.
tambourine_man 3y ago
> Gargini, who is chairman of the IEEE International Roadmap for Devices and Systems (IRDS), proposed in April that the industry “return to reality" by adopting a three-number metric that combines contacted gate pitch (G), metal pitch (M), and, crucially for future chips, the number of layers, or tiers, of devices on the chip (T). It’s not my field of expertise, but couldn’t we adopt a measure of density of transistors? Billions per mm or something like it. A single number is also good for marketing as the MHz of the 90s have shown us
- xw390111 3y ago
  I suppose it depends on who the measure is for. If it's for engineers: In 2024 a single metric is going to be far too simplistic to realistically drive any decision. If it's for the general public: Any metric will do, might as well stick with nm.
- jasonwatkinspdx 3y ago
  > It’s not my field of expertise, but couldn’t we adopt a measure of density of transistors? Billions per mm or something like it. That's actually what the current "marketing nm" are meant to do. They're the notional extrapolation of simple planar transistor density to complex 3d transistor architectures. People like to stand on a soap box and stand impressed with themselves for pointing out these are "just marketing numbers not the real half pitch anymore" but they're missing the point. The industry continues to use the nm nomenclature because it's clear to them what everyone is referring to. Modern fab processes are so complex you're not going to characterize their actual technical details by any single number.
  - bmer 3y ago
    Is there a resource you'd recommend that would help one understand what `marketing nm` means technically, and why it's a quantity that the industry uses "because it's clear to them"?
    - brennanpeterson 3y ago
furyofantares 3y ago
I've often pondered: why does this problem submit to sustained exponential improvement for such a long time? There doesn't seem to be any physical characteristics of miniaturization that map directly to exponential functions, in the same way as many rate-of-change problems do when you first learn differential equations. I've largely concluded that it's a combination of three things. 1. As Fenyman said, there's plenty of room at the bottom. This is why so many orders of magnitude improvement have been available. 2. As Turing proved, computers are general purpose. A consequence of this is we have only a small number of problems to solve that apply to all software, and so as the industry has grown exponentially, we've been able to put concentrated resources into making hardware better. 3. Exponential growth is simply the best we can do as human innovators. Exponential growth isn't a property of the problem; it's a limit of our innovation. We are able to hit the limit due to the economic consequences of computers being general purpose, and able to sustain it due to there being plenty of room at the bottom.
- chii 3y ago
  computers are also used for their own future designs. I would imagine that faster computers could help design better computers in the future, thus there's some level of exponential growth there. However, i think computer aided designs are reaching the limits of what help it gives, and the bottleneck is human intelligence and material engineering difficulties (that cannot use the help of computers). that's why moores law was more exponential early on in the miniaturisation of chips, but it's slowed down more and more - till one day it's gonna be sublinear. Of course, this doesn't preclude a new breakthrough in a different medium (optical computers, rather than electronic?), or a different form like quantum.
- hnaccount_rng 3y ago
  I think there are two components that are required for exponential growth, with the second one being a corollary of the other: 1) There needs to be some feedback from the effect of the process back to the constraints on the process 2) There must not be a significant external limitation where 2) implies that the total (economic) effort expended towards the process is small compared to the total (economic) benefit derived from the process. As the sibling comment already states: We could not produce modern chips without (at least modernish) chips to execute the design tools. But I'm not sure how critical that actually is for the exponential growth (extrapolating from the inefficiency of all other software written today). The interesting question is, whether we are close to a limit yet. Which, I think, translates to the question, whether we are already close enough to the edge of what is physically possible to have some significant economic cost (with respect to the derived economic benefit) What makes me think right now is your 3) .. What would a (sustained) better-than-exponential growth require and how would we identify such a thing/opportunity
GauntletWizard 3y ago
I hold that Moore's law finally died in 2023, along with Moore himself. We'll need about five years to see if I'm right.
- lesuorac 3y ago
  Eh, it died in 1975 when he observed that it no longer doubled every year and instead doubled every 2 years. Either that or it's a Ship of Theseus moment and we can just redefine moore's law to be whatever the current rate becomes.
fullstackchris 3y ago
I suppose here is the best place to ask: Isn't Moore's law capped by fundamental physics (i.e. tunneling effects once you get around / below 1nm sizes)? EDIT: did some snooping around, Moore himself said it in 2005: "We have another 10 to 20 years before we reach a fundamental limit."
- xw390111 3y ago
  It depends on how one defines transistor but I believe there is no known fundamental limit. If transistor as switch is sufficient then single-atom transistors have been demonstrated. https://en.wikipedia.org/wiki/Single-atom_transistor https://en.wikipedia.org/wiki/Single-atom_transistor
  - RetroTechie 3y ago
    In that (extreme) example, each transistor requiring at least 1 atom, would be your fundamental limit. New semiconductor materials or entirely different approaches (like quantum computing, optical ICs, 3D stacking etc) may give us a few more orders of magnitude. But as the article states: most likely it's economic barriers. Market conditions must support the technical effort required. Otherwise that tech is a dead end no matter how far it could go in theory. FWIW: in a way I'd welcome an end to Moore's law. It's about time software bloat gets addressed more aggressively.

88 points by klelatti 3y ago

Also see my article "The Mechanics of Moore's Law" for an analysis in terms of economic feedback loops with usual adaptive characteristics. https://till.com/articles/MooresLaw https://till.com/articles/MooresLaw

an_aparallel 3y ago
i read this comment...and thought "surely not thru-zero quadrature VCO donald tillman"? surely enough...you were :)
- jdontillman 3y ago
  Hey ho!

gfody 3y ago

Jim Keller gave a talk at Berkeley titled "Moore's Law is not dead" (2019) where he summarized a bunch of up and coming technologies that will enable the continued scaling of transistor density - https://www.youtube.com/watch?v=oIG9ztQw2Gc https://www.youtube.com/watch?v=oIG9ztQw2Gc

hinkley 3y ago
How many of those are invalidated by Specter? We’ve been building chips that are literally impossibly fast for half of the last decade at least.
- GuB-42 3y ago
  Spectre is just about timing attacks on branch prediction. It has nothing to do with transistor density, and it can be remediated with a reasonable impact on performance, that is if remediation is necessary. These vulnerabilities are also hard to exploit in real life and not always relevant. Rowhammer has to do with RAM density, but again, hard to exploit and there are mitigations.
- renewiltord 3y ago
  Not that important depending on use-case. Lots of people have been running for years with `mitigations=off`. If someone could steal their crypto, they would have. https://news.ycombinator.com/item?id=32077583 https://news.ycombinator.com/item?id=32077583 I run with `mitigations=off` in offline cluster.
xw390111 3y ago
Moore's Law has been entirely coopted by the media and, as such, lost all meaning and predictive power, which is what made it famous in the first place. Once upon a time it was a useful tool to plan programs with. So outside of popular culture, it is dead...

Legend2440 3y ago

>the end of Dennard Scaling around 2006 has meant that the rate of increase in performance has slowed even as Moore’s Law has continued. This is why parallel compute (multi core/GPUs) has taken off since 2006 - heat puts a cap on clock speed, so the only option is to do more operations per cycle. Even some microcontrollers are dual-core now. I don't see this trend stopping. Massive parallelism is the future of computing.

arein3 3y ago
We need moore's nvidia law. Price increases 50% every 2 years
- sigmoid10 3y ago
  That's just what monopolies get you. If they had serious competitors, they couldn't charge a 1000% markup.
  - p1necone 3y ago
    Intel's ARC gpu's keep improving with driver updates, if they ship another generation I can see them eventually properly disrupting the current duopoly.
    - sigmoid10 3y ago
      ARC is still very far away from even the lower end of what Nvidia offers. And on the enterprise side Intel is also well behind with their Gaudi chips. I'd love cheaper GPUs as much as everyone, but I don't see anyone who would could be considered a threat to Nvidia's position over the next years.
      - p1necone 3y ago
        "very far away" isn't accurate - the A770 is pretty close to a 4060 - perfectly adequate for 1080p/1440p 60fps gaming on high settings in modern titles.

zamadatix 3y ago

Some discussion around applying Ngrams to your investigation: It's really easy to make ngrams say something wrong by accident. I don't think these ones turned out too poorly for what the conclusions were trying trying to convey but for modern terms there are two things you want to keep an eye out for or adapt for: 1) Enable case insensitivity, particularly for things like "transistor" which are rarely capitalized or "integrated circuit" but even for "should be capitalized things" like "Moore's Law" vs "Moore's law". 2) (Especially on things spanning less than a century) check the impact of smoothing. It can "smooth out" the introduction of term to appear to take off years early or hide the true peaks/variabilities because they only lasted a couple of years instead of decades like the default smoothing might be better suited for. 3) (Not as applicable here) make sure the terms are popular enough you're not just comparing noise. 4) Adjust your range so the data is properly scaled. If your term was coined around the 1970s (and comparatively uncommon prior to that)then the 1870s shouldn't be midway through your graph. 5) If various case insensitive spellings create a messy set of graph lines adding in a less popular term like "exaflops" will cause them to be merged and you can extract that line. 6) Be careful comparing things that can be pluralized or otherwise modified with things that aren't. E.g. "Moore's Law" vs "Integrated Circuit[s]?" Combining some of these, notes like "Here is a further ngram of the ‘End of Moore’s Law’ which has clearly been a live topic of discussion since the late 1990s." actually look to be a bit off when you switch the graph to clue in on the last 30 years, set smoothing to 0, and enable case sensitivity to catch "[Ee]nd of [Mm]oore's [Ll]aw". Case insensitivity changes the shape of the humps and 2019 ends up significantly higher than the peak in the early 2000s. Shrinking the x-axis and not smoothing out the peak to be wider than it is shows there is very little in the way of the end of Moore's Law in the late 90's, only really starting to take off in 99 and not peaking until 2003. A similar story repeats for the opening graph, the gap between variants of "Moore's Law" and variants of "Integrated Circuit" is more than double the frequency, adding in that variants of "Integrate Circuits" are even more than either and counted separately. Overall the points turn out right enough (a few years difference in when the phrase found its first wave doesn't impact the story, nor does how frequent the use of Moore's law is vs ICs as much as the general rise of one and decline of the other) but I always want to highlight how the friendly interface of Google's Ngram viewer can easily lead you to the wrong conclusion on the data. I'd also like to point out the above tips are neither exhaustive or universally applicable to any type of search. Sometimes you want more smoothing, sometimes you do want an exact version of a phrase, sometimes you do want to show the term was used long before the modern meaning, and sometimes there are additional things that can bite you. E.g. one I was worried about here was "Moores Law" typos excluding the ' but it turned out to not be common at all. You really don't know until you check each possibility though.

tambourine_man 3y ago

> Gargini, who is chairman of the IEEE International Roadmap for Devices and Systems (IRDS), proposed in April that the industry “return to reality" by adopting a three-number metric that combines contacted gate pitch (G), metal pitch (M), and, crucially for future chips, the number of layers, or tiers, of devices on the chip (T). It’s not my field of expertise, but couldn’t we adopt a measure of density of transistors? Billions per mm or something like it. A single number is also good for marketing as the MHz of the 90s have shown us

xw390111 3y ago
I suppose it depends on who the measure is for. If it's for engineers: In 2024 a single metric is going to be far too simplistic to realistically drive any decision. If it's for the general public: Any metric will do, might as well stick with nm.
jasonwatkinspdx 3y ago
> It’s not my field of expertise, but couldn’t we adopt a measure of density of transistors? Billions per mm or something like it. That's actually what the current "marketing nm" are meant to do. They're the notional extrapolation of simple planar transistor density to complex 3d transistor architectures. People like to stand on a soap box and stand impressed with themselves for pointing out these are "just marketing numbers not the real half pitch anymore" but they're missing the point. The industry continues to use the nm nomenclature because it's clear to them what everyone is referring to. Modern fab processes are so complex you're not going to characterize their actual technical details by any single number.
- bmer 3y ago
  Is there a resource you'd recommend that would help one understand what `marketing nm` means technically, and why it's a quantity that the industry uses "because it's clear to them"?
  - brennanpeterson 3y ago

furyofantares 3y ago

I've often pondered: why does this problem submit to sustained exponential improvement for such a long time? There doesn't seem to be any physical characteristics of miniaturization that map directly to exponential functions, in the same way as many rate-of-change problems do when you first learn differential equations. I've largely concluded that it's a combination of three things. 1. As Fenyman said, there's plenty of room at the bottom. This is why so many orders of magnitude improvement have been available. 2. As Turing proved, computers are general purpose. A consequence of this is we have only a small number of problems to solve that apply to all software, and so as the industry has grown exponentially, we've been able to put concentrated resources into making hardware better. 3. Exponential growth is simply the best we can do as human innovators. Exponential growth isn't a property of the problem; it's a limit of our innovation. We are able to hit the limit due to the economic consequences of computers being general purpose, and able to sustain it due to there being plenty of room at the bottom.

chii 3y ago
computers are also used for their own future designs. I would imagine that faster computers could help design better computers in the future, thus there's some level of exponential growth there. However, i think computer aided designs are reaching the limits of what help it gives, and the bottleneck is human intelligence and material engineering difficulties (that cannot use the help of computers). that's why moores law was more exponential early on in the miniaturisation of chips, but it's slowed down more and more - till one day it's gonna be sublinear. Of course, this doesn't preclude a new breakthrough in a different medium (optical computers, rather than electronic?), or a different form like quantum.
hnaccount_rng 3y ago
I think there are two components that are required for exponential growth, with the second one being a corollary of the other: 1) There needs to be some feedback from the effect of the process back to the constraints on the process 2) There must not be a significant external limitation where 2) implies that the total (economic) effort expended towards the process is small compared to the total (economic) benefit derived from the process. As the sibling comment already states: We could not produce modern chips without (at least modernish) chips to execute the design tools. But I'm not sure how critical that actually is for the exponential growth (extrapolating from the inefficiency of all other software written today). The interesting question is, whether we are close to a limit yet. Which, I think, translates to the question, whether we are already close enough to the edge of what is physically possible to have some significant economic cost (with respect to the derived economic benefit) What makes me think right now is your 3) .. What would a (sustained) better-than-exponential growth require and how would we identify such a thing/opportunity

GauntletWizard 3y ago

I hold that Moore's law finally died in 2023, along with Moore himself. We'll need about five years to see if I'm right.

lesuorac 3y ago
Eh, it died in 1975 when he observed that it no longer doubled every year and instead doubled every 2 years. Either that or it's a Ship of Theseus moment and we can just redefine moore's law to be whatever the current rate becomes.

fullstackchris 3y ago

I suppose here is the best place to ask: Isn't Moore's law capped by fundamental physics (i.e. tunneling effects once you get around / below 1nm sizes)? EDIT: did some snooping around, Moore himself said it in 2005: "We have another 10 to 20 years before we reach a fundamental limit."

xw390111 3y ago
It depends on how one defines transistor but I believe there is no known fundamental limit. If transistor as switch is sufficient then single-atom transistors have been demonstrated. https://en.wikipedia.org/wiki/Single-atom_transistor https://en.wikipedia.org/wiki/Single-atom_transistor
- RetroTechie 3y ago
  In that (extreme) example, each transistor requiring at least 1 atom, would be your fundamental limit. New semiconductor materials or entirely different approaches (like quantum computing, optical ICs, 3D stacking etc) may give us a few more orders of magnitude. But as the article states: most likely it's economic barriers. Market conditions must support the technical effort required. Otherwise that tech is a dead end no matter how far it could go in theory. FWIW: in a way I'd welcome an end to Moore's law. It's about time software bloat gets addressed more aggressively.