11 ms·

The picture you're painting is far too simple. There are many metrics that play a role for different applications. A modern Nvidia card is magnitudes faster th

The picture you're painting is far too simple. There are many metrics that play a role for different applications. A modern Nvidia card is magnitudes faster than any CPU and the M1 GPUs. A Nvidia A100 card has HBM memory with ~2TB/s bandwidth. On the other hand, there are areas where a ~5GHz intel CPU is competitive, or even older server chips with AVX can compete. So let's see what happens when Intel and AMD step to smaller processes and switch to much faster memory (ddr5 as apple has done). Maybe the gap is large in energy consumption, but not in total performance.

qeternity 4y ago
> magnitudes faster No...
- fxtentacle 4y ago
  100+ vs <20 TFLOPS
  - danaris 4y ago
    ...which is one order of magnitude.
    - xhkkffbf 4y ago
      Not if you use base 2. And sometimes I like to use base 1.00001 for a grand effect.
- uniqueuid 4y ago
  https://en.wikipedia.org/wiki/FLOPS https://en.wikipedia.org/wiki/FLOPS Intel Rocket Lake FP64: 32 Flops per core per cycle at 8 cores and 5GHz Nvidia Ampere: 1/32 Flops per core per cycle at 2048 cores and ~2 GHz Even though there are more architectural differences, there IS a difference in orders of magnitude ... for the things that you can do on a GPU (SIMD).
  - bhouston 4y ago
    This is because FP64 is emulated on the NVIDIA Ampere as it is missing dedicated FP64 hardware. This is not a fair comparison. FP32 is 64x faster on NVIDIA Ampere than FP64 per: https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-architecture-whitepaper-v2.pdf https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-... Ampere has 60 TFLOPs of Performance on its top GPU, the Hopper per: https://en.wikipedia.org/wiki/Ampere_(microarchitecture) https://en.wikipedia.org/wiki/Ampere_(microarchitecture). Which is roughly 50% to 100% faster than Rocket Lake depending and that doesn't account for the parallelism/memory benefits that comes with the GPU architecture.
    - hajile 4y ago
      Let's analyze 3 real cores. A100 has 6912 cores (excluding tensor as they don't do 32-bit float math). Each core does 2 (32-bit) floats per cycle and runs at 1410MHz peak frequency. It is 826mm^2 and has an official 300w TDP. This yields 19491.84 GFLOPS. Threadripper (Zen 3) has 64 cores. Each core does 32 (32-bit) floats per cycle and runs at 4300MHz. It has 8 CPU chiplets (80.7mm^2 each) and one IO die (416mm^2) for a total area of 1061.6mm^2. This chip yields 8806.4 GFLOPS. Alder Lake has 8 P-cores and 8 E-cores. If you have an unlocked chip with AVX-512, that yields 64 (32-bit) floats per cycle running at up to 5.3GHz along with 32 (32-bit) floats per cycle running at up to 4GHz. Die area is around 215.25mm^2 (though much of that die area is taken up by the GPU) and it has a TDP of 150w (peak 241w). This chip yields 2713.6 GFLOPS for the P-cores and 1024 GFLOPS for the E-cores for a total of 3737.6 GFLOPS. The first thing of note is that orders of magnitude is a gross overstatement as there isn't even a single order of magnitude of difference between the slowest and fastest systems here. Dividing GFLOPS by die area gives 23.60 GFLOPS/mm^2 for the A100, 8.30 GFLOPS/mm^2 for the Threadripper, and 17.36 GFLOPS?mm&2 for the Alder Lake. Considering the GPU size of the Alder Lake, I suspect that GFLOPS per die area actually favors Alder Lake over the A100 (though this is also somewhat offset by the presence of tensor cores and other non-related stuff in A100). Also noteworthy is that power is a much weirder metric due to how turbos affect all the things. I suspect this is where the A100 has a major advantage not to mention real GFLOPS dropping off steeply in long-running workloads (though this will affect all of these systems to greater or lesser degrees). On the flip side, I should add that branchy code dramatically slows down actual performance on a GPU while only slowing down a CPU when a misprediction happens (in theory this should be only 1-5% of cases). A more interesting question would be Alpha EV9 with it's proposed 1024-bit SIMD unit running at around 2GHz. That chip would have delivered 256 GFLOPS of performance per core around 2005. Scaling to a modest 32-core chip at 4GHz gives a very impressive 16384 GFLOPS while retaining all those CPU advantages too. This leads to the observation that adding large SIMD/vector units to a CPU isn't really hard. A small, in-order core with very wide vector units running at a slower speed to conserve energy seems like an interesting idea and several startups are working in this direction (not to mention this being the basis of Intel's Larabee and later Knight's Corner designs).
daneel_w 4y ago
> So let's see what happens when Intel and AMD step to smaller processes and switch to much faster memory (ddr5 as apple has done). Maybe the gap is large in energy consumption, but not in total performance. There are 18 months worth of benchmarks from the Apple M1 that show that the gap really is large in total performance, and the "standard" M1 doesn't use DDR5, it uses the same DDR4 memory everything else uses. The observed performance gap is larger than what we can see happen between DDR4 and DDR5, and larger than what is expected to happen for AMD's and Intel's x86 between 7 and 5 nm.
- pclmulqdq 4y ago
  Apple uses LPDDR RAM which is a very different protocol. It is actually faster than DDR, and this may be a big part of the performance difference. For obvious reasons, desktop CPUs cannot use LPDDR RAM, because you can't put it in DIMM form.
  - daneel_w 4y ago
    Are you really sure about that? I thought LPDDR is all about low power consumption - running at half the frequency but making up for it by twice the bus width, ending up at same performance but with an emphasis on the LOW POWER part of its name. Add.: or maybe I got it the wrong way around; smaller bus width but same performance due often being clocked faster. Either way I cannot find anything suggesting LPDDR offers practically faster performance on memory-bound operations than the DDR equivalent, but I'd love to see some tech details and numbers explaining how it works.
    - wmf 4y ago
      In Apple's case, it's the same width (M1/2) or much wider (Pro/Max/Ultra) and also clocked faster (6400 vs. 4800).
    - pclmulqdq 4y ago
      LPDDR used to be about power consumption. It is not any more. DDR uses a lot of power for I/O and SERDES links to maintain a modular form factor, and LPDDR uses a lot less power by having worse signal integrity and lower swings. They run the memory cells and the I/O faster, and as of LPDDR4 it is now significantly faster than the DDR memory products available.
rob74 4y ago
Both Intel and AMD have the major disadvantage of having (or, let's say, wanting) to maintain x86 backward compatibility - which means not only staying compatible with the almost 45 year old CISC instruction set of the Intel 8086, but also with the whole line of CPUs that came after it. All this ballast makes it harder for them to build an energy-efficient CPU. By contrast, Apple was able to just create a new CPU based on the much more modern ARM architecture, and switch to it like they have already done two times before (from 68k to PowerPC and then to x86), because they control the whole Macintosh platform. Compare that to Microsoft's largely unsuccessful attempt of selling ARM-based Windows machines.
- saagarjha 4y ago
  Having to support x86 is definitely not a "major" disadvantage.
  - hajile 4y ago
    The X2 cores in 8cx gen3 get similar performance to Zen 2 U-series chips while using 7-10w instead of 15-50w. How many more ARM chips have to get equivalent performance at a tiny fraction of the power before people will finally start considering that either AMD and Intel hire the most incompetent people out there or that ISA actually matters?
    - saagarjha 4y ago
      The competent people say that ISA doesn’t matter, so there’s that.
      - hajile 4y ago
        Do their actions match their words? ARM could have extended their ARMv7 ISA to also be 64-bit without much trouble at all (plenty of space in the ISA). Instead, we got a ground up redesign that has more in common with MIPS64 than with ARMv7 (to the point that it's fascinating that ARM and some others bought MIPS just to get patent rights before immediately selling them off again). I'm sure the go-to individual here is Jim Keller. First, were his words free from constraint? If he said "x86 is crap and we should be using ARM or RISC-V" while working for those companies, how long would it take for him to be fired? Even if he were working at another company, saying that would burn a bridge that has kept him fed for decades. Would that be worth it? He later said that if he were designing a new chip from the ground up, he'd probably go with RISC-V. He then proceeded to do that when he went to work for TensTorrent even though licensing ARM cores and adding their custom units on the side would be cheaper and faster to market. Given his connections, working with the AMD custom division would also be possible and would not only save some time and money for the startup, but would also have a massive set of available and heavily-optimized software working from the very start which would save even more money. I'd say that his actions when free from corporate constraints show a definite opinion on the matter.
FireBeyond 4y ago
Very simple. There's also several articles discussing that how certain M2 configurations are actually lower performance than the M1, so this is hardly a done race.

qeternity 4y ago

> magnitudes faster No...

fxtentacle 4y ago
100+ vs <20 TFLOPS
- danaris 4y ago
  ...which is one order of magnitude.
  - xhkkffbf 4y ago
    Not if you use base 2. And sometimes I like to use base 1.00001 for a grand effect.
uniqueuid 4y ago
https://en.wikipedia.org/wiki/FLOPS https://en.wikipedia.org/wiki/FLOPS Intel Rocket Lake FP64: 32 Flops per core per cycle at 8 cores and 5GHz Nvidia Ampere: 1/32 Flops per core per cycle at 2048 cores and ~2 GHz Even though there are more architectural differences, there IS a difference in orders of magnitude ... for the things that you can do on a GPU (SIMD).
- bhouston 4y ago
  This is because FP64 is emulated on the NVIDIA Ampere as it is missing dedicated FP64 hardware. This is not a fair comparison. FP32 is 64x faster on NVIDIA Ampere than FP64 per: https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-architecture-whitepaper-v2.pdf https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-... Ampere has 60 TFLOPs of Performance on its top GPU, the Hopper per: https://en.wikipedia.org/wiki/Ampere_(microarchitecture) https://en.wikipedia.org/wiki/Ampere_(microarchitecture). Which is roughly 50% to 100% faster than Rocket Lake depending and that doesn't account for the parallelism/memory benefits that comes with the GPU architecture.
  - hajile 4y ago
    Let's analyze 3 real cores. A100 has 6912 cores (excluding tensor as they don't do 32-bit float math). Each core does 2 (32-bit) floats per cycle and runs at 1410MHz peak frequency. It is 826mm^2 and has an official 300w TDP. This yields 19491.84 GFLOPS. Threadripper (Zen 3) has 64 cores. Each core does 32 (32-bit) floats per cycle and runs at 4300MHz. It has 8 CPU chiplets (80.7mm^2 each) and one IO die (416mm^2) for a total area of 1061.6mm^2. This chip yields 8806.4 GFLOPS. Alder Lake has 8 P-cores and 8 E-cores. If you have an unlocked chip with AVX-512, that yields 64 (32-bit) floats per cycle running at up to 5.3GHz along with 32 (32-bit) floats per cycle running at up to 4GHz. Die area is around 215.25mm^2 (though much of that die area is taken up by the GPU) and it has a TDP of 150w (peak 241w). This chip yields 2713.6 GFLOPS for the P-cores and 1024 GFLOPS for the E-cores for a total of 3737.6 GFLOPS. The first thing of note is that orders of magnitude is a gross overstatement as there isn't even a single order of magnitude of difference between the slowest and fastest systems here. Dividing GFLOPS by die area gives 23.60 GFLOPS/mm^2 for the A100, 8.30 GFLOPS/mm^2 for the Threadripper, and 17.36 GFLOPS?mm&2 for the Alder Lake. Considering the GPU size of the Alder Lake, I suspect that GFLOPS per die area actually favors Alder Lake over the A100 (though this is also somewhat offset by the presence of tensor cores and other non-related stuff in A100). Also noteworthy is that power is a much weirder metric due to how turbos affect all the things. I suspect this is where the A100 has a major advantage not to mention real GFLOPS dropping off steeply in long-running workloads (though this will affect all of these systems to greater or lesser degrees). On the flip side, I should add that branchy code dramatically slows down actual performance on a GPU while only slowing down a CPU when a misprediction happens (in theory this should be only 1-5% of cases). A more interesting question would be Alpha EV9 with it's proposed 1024-bit SIMD unit running at around 2GHz. That chip would have delivered 256 GFLOPS of performance per core around 2005. Scaling to a modest 32-core chip at 4GHz gives a very impressive 16384 GFLOPS while retaining all those CPU advantages too. This leads to the observation that adding large SIMD/vector units to a CPU isn't really hard. A small, in-order core with very wide vector units running at a slower speed to conserve energy seems like an interesting idea and several startups are working in this direction (not to mention this being the basis of Intel's Larabee and later Knight's Corner designs).

daneel_w 4y ago

> So let's see what happens when Intel and AMD step to smaller processes and switch to much faster memory (ddr5 as apple has done). Maybe the gap is large in energy consumption, but not in total performance. There are 18 months worth of benchmarks from the Apple M1 that show that the gap really is large in total performance, and the "standard" M1 doesn't use DDR5, it uses the same DDR4 memory everything else uses. The observed performance gap is larger than what we can see happen between DDR4 and DDR5, and larger than what is expected to happen for AMD's and Intel's x86 between 7 and 5 nm.

pclmulqdq 4y ago
Apple uses LPDDR RAM which is a very different protocol. It is actually faster than DDR, and this may be a big part of the performance difference. For obvious reasons, desktop CPUs cannot use LPDDR RAM, because you can't put it in DIMM form.
- daneel_w 4y ago
  Are you really sure about that? I thought LPDDR is all about low power consumption - running at half the frequency but making up for it by twice the bus width, ending up at same performance but with an emphasis on the LOW POWER part of its name. Add.: or maybe I got it the wrong way around; smaller bus width but same performance due often being clocked faster. Either way I cannot find anything suggesting LPDDR offers practically faster performance on memory-bound operations than the DDR equivalent, but I'd love to see some tech details and numbers explaining how it works.
  - wmf 4y ago
    In Apple's case, it's the same width (M1/2) or much wider (Pro/Max/Ultra) and also clocked faster (6400 vs. 4800).
  - pclmulqdq 4y ago
    LPDDR used to be about power consumption. It is not any more. DDR uses a lot of power for I/O and SERDES links to maintain a modular form factor, and LPDDR uses a lot less power by having worse signal integrity and lower swings. They run the memory cells and the I/O faster, and as of LPDDR4 it is now significantly faster than the DDR memory products available.

rob74 4y ago

Both Intel and AMD have the major disadvantage of having (or, let's say, wanting) to maintain x86 backward compatibility - which means not only staying compatible with the almost 45 year old CISC instruction set of the Intel 8086, but also with the whole line of CPUs that came after it. All this ballast makes it harder for them to build an energy-efficient CPU. By contrast, Apple was able to just create a new CPU based on the much more modern ARM architecture, and switch to it like they have already done two times before (from 68k to PowerPC and then to x86), because they control the whole Macintosh platform. Compare that to Microsoft's largely unsuccessful attempt of selling ARM-based Windows machines.

saagarjha 4y ago
Having to support x86 is definitely not a "major" disadvantage.
- hajile 4y ago
  The X2 cores in 8cx gen3 get similar performance to Zen 2 U-series chips while using 7-10w instead of 15-50w. How many more ARM chips have to get equivalent performance at a tiny fraction of the power before people will finally start considering that either AMD and Intel hire the most incompetent people out there or that ISA actually matters?
  - saagarjha 4y ago
    The competent people say that ISA doesn’t matter, so there’s that.
    - hajile 4y ago
      Do their actions match their words? ARM could have extended their ARMv7 ISA to also be 64-bit without much trouble at all (plenty of space in the ISA). Instead, we got a ground up redesign that has more in common with MIPS64 than with ARMv7 (to the point that it's fascinating that ARM and some others bought MIPS just to get patent rights before immediately selling them off again). I'm sure the go-to individual here is Jim Keller. First, were his words free from constraint? If he said "x86 is crap and we should be using ARM or RISC-V" while working for those companies, how long would it take for him to be fired? Even if he were working at another company, saying that would burn a bridge that has kept him fed for decades. Would that be worth it? He later said that if he were designing a new chip from the ground up, he'd probably go with RISC-V. He then proceeded to do that when he went to work for TensTorrent even though licensing ARM cores and adding their custom units on the side would be cheaper and faster to market. Given his connections, working with the AMD custom division would also be possible and would not only save some time and money for the startup, but would also have a massive set of available and heavily-optimized software working from the very start which would save even more money. I'd say that his actions when free from corporate constraints show a definite opinion on the matter.

FireBeyond 4y ago

Very simple. There's also several articles discussing that how certain M2 configurations are actually lower performance than the M1, so this is hardly a done race.