5 ms·

> The M1 processor is a direct result of the death of Moore's law. I know most people misunderstand Moore's law, but this is HN, so I expect better: https://e

by slver 6y ago

> The M1 processor is a direct result of the death of Moore's law. I know most people misunderstand Moore's law, but this is HN, so I expect better: https://en.wikipedia.org/wiki/Moore%27s_law#/media/File:Moore's_Law_Transistor_Count_1970-2020.png https://en.wikipedia.org/wiki/Moore%27s_law#/media/File:Moor... Moore's law is quite alive and showing no signs of problems. > The performance gains from Moore's law have typically come from shrinking die size. Moore's Law is about number of transistors. Not about their size, and not about performance. And it's ESPECIALLY not about linear core performance. > That has ended, you can't juice more performance from general purpose CPUs. You don't need to, they're fast enough. Performance is expanding in other areas like GPU and ML. > The death of Moore's law has many consequences, the rise of ASICs and custom co-processor chips is just one of them. No, Moore's law is the very thing supporting them... You need extra transistors for those co-processors.

CivBase 6y ago
> You don't need to, they're fast enough. You had me with everything except this. Any time someone claims the current state of computing hardware is "enough" I'm reminded of that fake 640K quote. There is no indication we are running out of applications for more compute power.
- slver 6y ago
  I'm not saying we're running out of applications for more compute power. We're specifically running out of reasons to want faster linear (per core) general-purpose performance (in fact I'd say this happened some time ago). Everything else we get from here on in terms of smaller process etc. is just a bonus. We don't fundamentally need it to keep evolving our hardware for our ever-growing computation needs. And that's because as our problems multiply and grow, parallel execution and heterogenous cores tend to solve our problems much more efficiently on the watt, than asking for "more of the same, but faster". There's this Ford quote "if I had asked what people want, they'd have said faster horses". Fake or not, it reflects our tendency to stare at the wrong variables and miss the forest for the trees. The industry is full of languages utilizing parallel/heterogenous execution and you don't need a PhD to use one anymore. CPUs are effectively turning into "controllers" that prepare command queues for various other types of processing units. As we keep evolving our GPU/ML/etc. processing units, CPUs will have less to do, not more. In fact, I expect CPUs will get simpler and slower as our bottlenecks move to the specialized vector units.
  - maerF0x0 6y ago
    https://scicomp.stackexchange.com/a/1395 https://scicomp.stackexchange.com/a/1395 Explains it better than I could in a HN reply. Assuming our desired work to be done continues to grow, faster core speed* will eventually matter. * technically, net instructions per second after prediction/compute ahead etc
    - slver 6y ago
      Honestly that's a bit too abstract to make sense of. As a programmer to another, I'd rather ask... what's one example of a problem we have today that needs faster linear performance than our best chips (not in a nice-to-have way but in a must-have way). I'd rule out all casual computing like home PCs, smartphones, and so on, because honestly we've been there for years already. Also due to decades of bias we have serialized code in our programs that doesn't have to be serial, just because that's "normal" and because it's deemed easier. Also we have a huge untapped potential of better performance by being more data-oriented. None of this requires faster hardware. It doesn't even require new hardware. But anyway, I'm open to examples.
      - dilippkumar 6y ago
        > I'd rule out all casual computing like home PCs, smartphones, and so on, because honestly we've been there for years already. Casual computing can definitely be a lot better than where we are today[0][1]. The software business has moved to a place where it’s not really practical to program bare metal silicon in assembly to get screaming fast performance. We write software on several layers of abstraction, each of which consumes a ton of compute capacity. We have resigned to live with 100ms latencies in our daily computing. This is us giving up on the idea of fast computers. It should not be confused with actually having a computer where all interactions are sub 10ms (less than 1 frame refresh period at 90fps). [0]. https://danluu.com/input-lag/ https://danluu.com/input-lag/ [1]. https://www.extremetech.com/computing/261148-modern-computers-struggle-match-input-latency-apple-iie https://www.extremetech.com/computing/261148-modern-computer...
  - pshc 6y ago
pcurve 6y ago
Maybe this is a bit cheating, but there's chip with 2.5 trillion transistors. https://www.businesswire.com/news/home/20210420005955/en/Cerebras-Systems-Smashes-the-2.5-Trillion-Transistor-Mark-with-New-Second-Generation-Wafer-Scale-Engine https://www.businesswire.com/news/home/20210420005955/en/Cer... https://wccftech.com/cerebras-unveils-7nm-wafe-scale-engine-2-largest-ai-chip-ever-built/ https://wccftech.com/cerebras-unveils-7nm-wafe-scale-engine-...

by slver 6y ago

CivBase 6y ago

> You don't need to, they're fast enough. You had me with everything except this. Any time someone claims the current state of computing hardware is "enough" I'm reminded of that fake 640K quote. There is no indication we are running out of applications for more compute power.

slver 6y ago
I'm not saying we're running out of applications for more compute power. We're specifically running out of reasons to want faster linear (per core) general-purpose performance (in fact I'd say this happened some time ago). Everything else we get from here on in terms of smaller process etc. is just a bonus. We don't fundamentally need it to keep evolving our hardware for our ever-growing computation needs. And that's because as our problems multiply and grow, parallel execution and heterogenous cores tend to solve our problems much more efficiently on the watt, than asking for "more of the same, but faster". There's this Ford quote "if I had asked what people want, they'd have said faster horses". Fake or not, it reflects our tendency to stare at the wrong variables and miss the forest for the trees. The industry is full of languages utilizing parallel/heterogenous execution and you don't need a PhD to use one anymore. CPUs are effectively turning into "controllers" that prepare command queues for various other types of processing units. As we keep evolving our GPU/ML/etc. processing units, CPUs will have less to do, not more. In fact, I expect CPUs will get simpler and slower as our bottlenecks move to the specialized vector units.
- maerF0x0 6y ago
  https://scicomp.stackexchange.com/a/1395 https://scicomp.stackexchange.com/a/1395 Explains it better than I could in a HN reply. Assuming our desired work to be done continues to grow, faster core speed* will eventually matter. * technically, net instructions per second after prediction/compute ahead etc
  - slver 6y ago
    Honestly that's a bit too abstract to make sense of. As a programmer to another, I'd rather ask... what's one example of a problem we have today that needs faster linear performance than our best chips (not in a nice-to-have way but in a must-have way). I'd rule out all casual computing like home PCs, smartphones, and so on, because honestly we've been there for years already. Also due to decades of bias we have serialized code in our programs that doesn't have to be serial, just because that's "normal" and because it's deemed easier. Also we have a huge untapped potential of better performance by being more data-oriented. None of this requires faster hardware. It doesn't even require new hardware. But anyway, I'm open to examples.
    - dilippkumar 6y ago
      > I'd rule out all casual computing like home PCs, smartphones, and so on, because honestly we've been there for years already. Casual computing can definitely be a lot better than where we are today[0][1]. The software business has moved to a place where it’s not really practical to program bare metal silicon in assembly to get screaming fast performance. We write software on several layers of abstraction, each of which consumes a ton of compute capacity. We have resigned to live with 100ms latencies in our daily computing. This is us giving up on the idea of fast computers. It should not be confused with actually having a computer where all interactions are sub 10ms (less than 1 frame refresh period at 90fps). [0]. https://danluu.com/input-lag/ https://danluu.com/input-lag/ [1]. https://www.extremetech.com/computing/261148-modern-computers-struggle-match-input-latency-apple-iie https://www.extremetech.com/computing/261148-modern-computer...
- pshc 6y ago

pcurve 6y ago

Maybe this is a bit cheating, but there's chip with 2.5 trillion transistors. https://www.businesswire.com/news/home/20210420005955/en/Cerebras-Systems-Smashes-the-2.5-Trillion-Transistor-Mark-with-New-Second-Generation-Wafer-Scale-Engine https://www.businesswire.com/news/home/20210420005955/en/Cer... https://wccftech.com/cerebras-unveils-7nm-wafe-scale-engine-2-largest-ai-chip-ever-built/ https://wccftech.com/cerebras-unveils-7nm-wafe-scale-engine-...