6 ms·

I've seen this interpretation before in HN comments and I think it is an unnecessary distortion of Moore's Law: Note the historical context the observation was

by tomxor 7y ago

I've seen this interpretation before in HN comments and I think it is an unnecessary distortion of Moore's Law: Note the historical context the observation was originally made in (1965) when doubling was achieved through process size reduction, the effect this had on speed was twofold: operating frequency would increase roughly in proportion (increasing about 40%); secondly and more uniquely to the time, since predecessors transistor count were so limited there was significant room for improvements in instruction implementation and specialization as available transistors increased. Although it's implied, Moore also explicitly stated this in his 1965 paper: > [...] In fact, shrinking dimensions on an integrated structure makes it possible to operate the structure at higher speed for the same power per unit area. [1] Later this effect was more explicitly defined as Dennard Scaling in 1974 [2] Transistor count increases in recent years has very little to do with dennard scaling or improving individual instruction performance and everything to do with improving some form of parallel compute by figuring out how to fit, route and schedule more transistors at the same process size, which does not have the same effect Moore was originally alluding to. [1] https://drive.google.com/file/d/0By83v5TWkGjvQkpBcXJKT1I1TTA/view https://drive.google.com/file/d/0By83v5TWkGjvQkpBcXJKT1I1TTA... [2] https://en.wikipedia.org/wiki/Dennard_scaling https://en.wikipedia.org/wiki/Dennard_scaling

ksec 7y ago
Tl;dr: Moore's Law is simply about doubling of transistor, we got performance and reduced cost CPU along with it for the first 30 - 40 years. But Somehow the world interrupt the performance and cost as Moore's Law.
- coldtea 7y ago
  What matters concerning what a term means is what the world interprets, not what something was etymologically, or what was in the mind of the person who coined it.
kurthr 7y ago
I'll link to the wikipedia graphic of Moore's Law: https://upload.wikimedia.org/wikipedia/commons/8/8b/Moore%27s_Law_Transistor_Count_1971-2018.png https://upload.wikimedia.org/wikipedia/commons/8/8b/Moore%27... It makes the point that Moore's law is dead. You can see the slope change in in 2006 with the Dual Core Itanium2. So it died after Dennard scaling toppled (and some of the fastest growth ever), and although transistor count continues to increase it is noticeably slower... I fear that it may go slower still since economic costs continue to increase, but profits don't (as fast). When the CFO figures out it's not worth investing in scaling, you just won't build new fabs (unless the gov just says so). To put some numbers to this (topline growth): 1971 TMS1000 (8k) to 1979 M68000 (70k) 2.6yr/double 1979 M68000 (70k) to 2003 Itanium M6 (400M) 1.9yr/double 2003 Itanium2 6M (400M) to 2006 Dual Itanium (1.6B) 1.5yr/double 2006 Dual Itanium2 (1.6B) to 2018 GC2 IPU (24B) 3.1yr/double Personally, I think that an era of backend improvements (wafer stacking etc) to combine processes (Memory/Logic/Flash) into locally interconnected 2.5D is possible, if there remains sufficient investment, and that in turn could drive some improvements to performance and pricing. However, that relies on ever more complex system design and integration... and heat dissipation issues. I don't see full 3D scaling or Quantum compute on the 3-5 year horizon where big new investments will be required. (edit formatting commentary and qualification)
- jsjohnst 7y ago
  I’m not sure I agree with your point about the Itanium2 specifically on this graph. It looks like the Itanium2 is an anomaly on the high side of the hockey stick to me, rather than being the rule.
  - kurthr 7y ago
    Since this chart doesn't show transistors/mm2 you kinda have to use the peaks and assume they are close to reticle/density limite. You could look at the lower ones or the "average", but it's not indicative of what is possible. In fact both die size and reticle size have been growing, which makes the plots pre-300mm wafer over performing... little of that happened in the last 15 years which has not helped continue the trend, and that's over since GPUs have been at max reticle for years now. I'm guessing that die size increase was originally about cooling. (edits corrected grammar and double negative added cooling comment)
- Veedrac 7y ago
  That looks like a pretty straight line to me, TBH. You're doing more subdivision than is fair to the data. I did my own graph of just density, which is also pretty clearly straight. https://docs.google.com/spreadsheets/d/1NNOqbJfcISFyMd0EsSrhppW7PT6GCfnrVGhxhLA5PVw/edit#gid=1105655921 https://docs.google.com/spreadsheets/d/1NNOqbJfcISFyMd0EsSrh...
  - kurthr 7y ago
    Yours is noticeably straighter than the Wikipedia page. I see that you are plotting density and not #transistors? Is that what Moores law says? Or does die size count? If you plot total #transistors your fitted slope will plummet in the later years and you'll see the change post 2006.
    - Veedrac 7y ago
      I think what Moore's law is depends on who you ask. Density makes sense and seems to be holding true, but so does cost/transistor, which hasn't.
karmakaze 7y ago
My key point about Moore's law is that it's about being able to 'throw new hardware' at a problem. If you're changing the game by saying, hey we've now got 1000 cpu chiplets that can compute way more and all you have to do is parallelize all your software--different game, different name.
- tomxor 7y ago
  > My key point about Moore's law is that it's about being able to 'throw new hardware' at a problem. If you're changing the game by saying, hey we've now got 1000 cpu chiplets [...] You're ignoring the physics. You can do all you want with new hardware designs, larger dies, more CPUs etc but ultimately it cannot make the same exponential gains that came with process shrinkage because power consumption and heat dissipation problems increase with every extra transistor if the process size remains the same. I'm not saying gains cannot be made without shrinking, but they will be incremental, at the most linear with a short window, but they will not be exponential and they will come with another cost (more power consumption). For Moore's Law to be revived, look to material science for answers.
  - coldtea 7y ago
    >You're ignoring the physics. You can do all you want with new hardware designs, larger dies, more CPUs etc but ultimately it cannot make the same exponential gains that came with process shrinkage because power consumption and heat dissipation problems increase with every extra transistor if the process size remains the same. He's not ignoring the physics, he's saying Moore's law (the "throw more hardware at the problem" version) is dead because of physics...
ChuckMcM 7y ago
The article, and the parent comment, share a common theme of "moving the goal posts" by re-defining (or at least attempting to re-contextualize) Gordon Moore's original conjecture. There isn't anything wrong with that, but for TSMC's Global Marketing guy to try to do that, it makes me think that TSMC doesn't get it. The argument "That isn't what Gordon meant, if you look at what he really said..." Attempts to dismiss four decades of what every engineer expected it to mean, and nobody was was correcting them because, well why should they because it was working. It wasn't until it started obviously failing to be true, that semiconductor companies started arguing in favor of an interpretation they could meet, rather than admit that Moore's law was dead, as pretty much any engineer actually building systems would tell you. Somewhere there must be a good play on the Monty Python Parrot sketch where Moore's law stands in for the parrot and a semiconductor marketing manager stands in for the hapless pet shop owner. It is really hard to make smaller and smaller transistors. And the laws of physics interferes. Further its really hard to get the heat out of a chip when you boost the frequency. Dennard, others, have characterized those limits more precisely and as we hit those limits, progress along that path slows to a crawl or stops. Amdahl pretty famously characterized the limits of parallelism, we are getting closer to that one too, even for things that are trivially parallelized like graphics or neural nets. The fear semiconductor companies have is clear, if the only way to improve performance is better software, then you don't need new chips, and an idle fab, ready to do a wafer start, costs nearly as much as an active one.
- AstralStorm 7y ago
  Unless they figure out how to make even more advanced nanotechnology. (MEMS comes to mind. It's hard too. As are analog systems.) Also less power intensive.

by tomxor 7y ago

ksec 7y ago

Tl;dr: Moore's Law is simply about doubling of transistor, we got performance and reduced cost CPU along with it for the first 30 - 40 years. But Somehow the world interrupt the performance and cost as Moore's Law.

coldtea 7y ago
What matters concerning what a term means is what the world interprets, not what something was etymologically, or what was in the mind of the person who coined it.

kurthr 7y ago

I'll link to the wikipedia graphic of Moore's Law: https://upload.wikimedia.org/wikipedia/commons/8/8b/Moore%27s_Law_Transistor_Count_1971-2018.png https://upload.wikimedia.org/wikipedia/commons/8/8b/Moore%27... It makes the point that Moore's law is dead. You can see the slope change in in 2006 with the Dual Core Itanium2. So it died after Dennard scaling toppled (and some of the fastest growth ever), and although transistor count continues to increase it is noticeably slower... I fear that it may go slower still since economic costs continue to increase, but profits don't (as fast). When the CFO figures out it's not worth investing in scaling, you just won't build new fabs (unless the gov just says so). To put some numbers to this (topline growth): 1971 TMS1000 (8k) to 1979 M68000 (70k) 2.6yr/double 1979 M68000 (70k) to 2003 Itanium M6 (400M) 1.9yr/double 2003 Itanium2 6M (400M) to 2006 Dual Itanium (1.6B) 1.5yr/double 2006 Dual Itanium2 (1.6B) to 2018 GC2 IPU (24B) 3.1yr/double Personally, I think that an era of backend improvements (wafer stacking etc) to combine processes (Memory/Logic/Flash) into locally interconnected 2.5D is possible, if there remains sufficient investment, and that in turn could drive some improvements to performance and pricing. However, that relies on ever more complex system design and integration... and heat dissipation issues. I don't see full 3D scaling or Quantum compute on the 3-5 year horizon where big new investments will be required. (edit formatting commentary and qualification)

jsjohnst 7y ago
I’m not sure I agree with your point about the Itanium2 specifically on this graph. It looks like the Itanium2 is an anomaly on the high side of the hockey stick to me, rather than being the rule.
- kurthr 7y ago
  Since this chart doesn't show transistors/mm2 you kinda have to use the peaks and assume they are close to reticle/density limite. You could look at the lower ones or the "average", but it's not indicative of what is possible. In fact both die size and reticle size have been growing, which makes the plots pre-300mm wafer over performing... little of that happened in the last 15 years which has not helped continue the trend, and that's over since GPUs have been at max reticle for years now. I'm guessing that die size increase was originally about cooling. (edits corrected grammar and double negative added cooling comment)
Veedrac 7y ago
That looks like a pretty straight line to me, TBH. You're doing more subdivision than is fair to the data. I did my own graph of just density, which is also pretty clearly straight. https://docs.google.com/spreadsheets/d/1NNOqbJfcISFyMd0EsSrhppW7PT6GCfnrVGhxhLA5PVw/edit#gid=1105655921 https://docs.google.com/spreadsheets/d/1NNOqbJfcISFyMd0EsSrh...
- kurthr 7y ago
  Yours is noticeably straighter than the Wikipedia page. I see that you are plotting density and not #transistors? Is that what Moores law says? Or does die size count? If you plot total #transistors your fitted slope will plummet in the later years and you'll see the change post 2006.
  - Veedrac 7y ago
    I think what Moore's law is depends on who you ask. Density makes sense and seems to be holding true, but so does cost/transistor, which hasn't.

karmakaze 7y ago

My key point about Moore's law is that it's about being able to 'throw new hardware' at a problem. If you're changing the game by saying, hey we've now got 1000 cpu chiplets that can compute way more and all you have to do is parallelize all your software--different game, different name.

tomxor 7y ago
> My key point about Moore's law is that it's about being able to 'throw new hardware' at a problem. If you're changing the game by saying, hey we've now got 1000 cpu chiplets [...] You're ignoring the physics. You can do all you want with new hardware designs, larger dies, more CPUs etc but ultimately it cannot make the same exponential gains that came with process shrinkage because power consumption and heat dissipation problems increase with every extra transistor if the process size remains the same. I'm not saying gains cannot be made without shrinking, but they will be incremental, at the most linear with a short window, but they will not be exponential and they will come with another cost (more power consumption). For Moore's Law to be revived, look to material science for answers.
- coldtea 7y ago
  >You're ignoring the physics. You can do all you want with new hardware designs, larger dies, more CPUs etc but ultimately it cannot make the same exponential gains that came with process shrinkage because power consumption and heat dissipation problems increase with every extra transistor if the process size remains the same. He's not ignoring the physics, he's saying Moore's law (the "throw more hardware at the problem" version) is dead because of physics...

ChuckMcM 7y ago

The article, and the parent comment, share a common theme of "moving the goal posts" by re-defining (or at least attempting to re-contextualize) Gordon Moore's original conjecture. There isn't anything wrong with that, but for TSMC's Global Marketing guy to try to do that, it makes me think that TSMC doesn't get it. The argument "That isn't what Gordon meant, if you look at what he really said..." Attempts to dismiss four decades of what every engineer expected it to mean, and nobody was was correcting them because, well why should they because it was working. It wasn't until it started obviously failing to be true, that semiconductor companies started arguing in favor of an interpretation they could meet, rather than admit that Moore's law was dead, as pretty much any engineer actually building systems would tell you. Somewhere there must be a good play on the Monty Python Parrot sketch where Moore's law stands in for the parrot and a semiconductor marketing manager stands in for the hapless pet shop owner. It is really hard to make smaller and smaller transistors. And the laws of physics interferes. Further its really hard to get the heat out of a chip when you boost the frequency. Dennard, others, have characterized those limits more precisely and as we hit those limits, progress along that path slows to a crawl or stops. Amdahl pretty famously characterized the limits of parallelism, we are getting closer to that one too, even for things that are trivially parallelized like graphics or neural nets. The fear semiconductor companies have is clear, if the only way to improve performance is better software, then you don't need new chips, and an idle fab, ready to do a wafer start, costs nearly as much as an active one.

AstralStorm 7y ago
Unless they figure out how to make even more advanced nanotechnology. (MEMS comes to mind. It's hard too. As are analog systems.) Also less power intensive.