3 ms·

The M1 parts actually have way worse memory latency (i.e. random accesses) than both AMD and Intel. A finely tuned Intel system has around 3x lower memory laten

The M1 parts actually have way worse memory latency (i.e. random accesses) than both AMD and Intel. A finely tuned Intel system has around 3x lower memory latency than an M1P/X. All of this is SDRAM, the S stands for synchronous and means that the memory is driven according to fixed timings in relation to a fixed bus clock. All LPDDR4X-4266 parts with the same timings perform exactly the same, whether they are soldered to the interposer or are 5 cm away on the board.

errantspark 5y ago
Interesting, I didn't know that the memory latency was worse. The bandwidth is still much higher right? In your estimation how much of the perf gains of the M1 chip are due to the increased memory bandwidth vs other optimizations/advancements?
- formerly_proven 5y ago
  I don't think you can single out one aspect when comparing designs which are so different, but I'd say most of the CPU performance is down to the core. There you have the M1 core which is (iirc) 10 pipes wide, much wider than both AMD and Intel, and a much fatter frontend as well to feed the beast. I think that's the main advantage the M1 has. It also has a larger L1 cache (twice as big as x86). These two are one of the few areas where x86 has an innate disadvantage because fattening up the frontend is much more complex on x86 compared to ARM, and you can't have a VIPT L1 cache larger than 64K with 4K pages, which is a hard-to-change default on x86, while the M1 by default uses larger pages (32K or something like that).
  - errantspark 5y ago
    "x86 has an innate disadvantage [...] fattening up the frontend" this is inherent largely because of ISA complexity and variable width instructions? "M1 core which is [...] much wider than both AMD and Intel" The core width you're referring to is the decoder width yeah? As another poster pointed out the M1 has a large reorder buffer as well. Combined with the ability to index a larger L1 cache what I'm getting here is that the M1 can be a lot better about scheduling instructions (and perhaps even running non-interfering instructions in parallel on a given core (is that a thing?)) because the frontend has more power and space to do so. I guess that efficiency is then a big part of the puzzle on why the increased bandwidth to ram makes such an impact?
- sliken 5y ago
  Many things don't push the memory bandwidth and are cache friendly. However GPUs are bandwidth limited and the M1 Max does quite well against any other integrated graphics from Intel or AMD. Even on the CPU side it can be a big win, in particular on SpecFPRate (a collection of heavy floating point real world codes, not microbenchmarks) Anand has this to say: The fp2017 suite has more workloads that are more memory-bound, and it’s here where the M1 Max is absolutely absurd. The workloads that put the most memory pressure and stress the DRAM the most, such as 503.bwaves, 519.lbm, 549.fotonik3d and 554.roms, have all multiple factors of performance advantages compared to the best Intel and AMD have to offer. To drive this home compare the Spec2017 FP Rate, the M1 Max gets 81.07, the Ryzen 5950x (high end desktop with twice as many fast cores and a 105 watt TDP) gets 62.27. So a low power M1 Max with half as many cores and much lower power is 30% faster than AMD's highest end desktop chip. Instead of a desktop size/volume/power, you can get it in a laptop that's 2/3rd of an inch thick.
sliken 5y ago
That's not at all what I'm seeing. Can you be specific as to what chips you are looking at and what benchmark you are using to draw this conclusion? I'll go first. Personally when talking about memory latency, I want to measure time to main memory, without TLB thrashing. Not that TLB latency isn't interesting, but it should be separately quantified. I've written a few microbenchmarks in this area, but I get very similar numbers to the Anandtech R per RV prange. Which puts the latency to main memory at around 35ns looking like it might flatten out at 40ns. Yes, full TLB thrashing is up at 110ns or so, but that's not a usual use case. If it is, at least under linux, you can switch to 1GB pages if that's important to you. R per RV prange numbers on the Ryzen 9 5950x is 65ns or so. So sure the TLB worst case is higher latency on the M1, but then you have to figure out how big the TLB is, and how often that's an issue if you want to know the real world performance impact. The i9-12900K with DDR5 gets 30ns on the R per RV prange, and 92ns on full random (tlb thrashing). Even assuming the worst case TLB behavior the m1 max is 111ns and the 5950x is 79ns or 1.4x higher. On the Intel side is 111ns vs 92n, 1.2x higher. Finding it hard to find any numbers that make the M1X look like 3x higher memory latency.
- formerly_proven 5y ago
  I feel like R per R isn't quite the perfect match for the pointer-chasy workloads I have in mind where you end up going through more-or-less random selections of relatively small (<< page size) objects, so R per R amortizes the randomization a little much. I still think having the overall totally random access for a system is a good number precisely because it contains all the penalties and latencies that can add up. Embarrassing as it may be, I also compared the wrong numbers across parts here (and in the past too, I think) - thanks for calling that out. I was kinda irritated how exactly they would be able to get the performance they were getting across all workloads with such poor latency but failed to double check.
  - sliken 5y ago
    Ah, sensible. Generally with the TLB numbers and the memory latency you can get a good idea of the performance by interoplating between the two number for any number of TLB misses (from 0% to 100%). The M1 Max also has a crazy number of memory channels, even the older M1 has 4 channels, pro has 8 channels, and max has 16. So you can have many more cache misses in flight, this is part of why the m1 max is 20% faster than the 5950x with twice as many cores.

errantspark 5y ago

Interesting, I didn't know that the memory latency was worse. The bandwidth is still much higher right? In your estimation how much of the perf gains of the M1 chip are due to the increased memory bandwidth vs other optimizations/advancements?

formerly_proven 5y ago
I don't think you can single out one aspect when comparing designs which are so different, but I'd say most of the CPU performance is down to the core. There you have the M1 core which is (iirc) 10 pipes wide, much wider than both AMD and Intel, and a much fatter frontend as well to feed the beast. I think that's the main advantage the M1 has. It also has a larger L1 cache (twice as big as x86). These two are one of the few areas where x86 has an innate disadvantage because fattening up the frontend is much more complex on x86 compared to ARM, and you can't have a VIPT L1 cache larger than 64K with 4K pages, which is a hard-to-change default on x86, while the M1 by default uses larger pages (32K or something like that).
- errantspark 5y ago
  "x86 has an innate disadvantage [...] fattening up the frontend" this is inherent largely because of ISA complexity and variable width instructions? "M1 core which is [...] much wider than both AMD and Intel" The core width you're referring to is the decoder width yeah? As another poster pointed out the M1 has a large reorder buffer as well. Combined with the ability to index a larger L1 cache what I'm getting here is that the M1 can be a lot better about scheduling instructions (and perhaps even running non-interfering instructions in parallel on a given core (is that a thing?)) because the frontend has more power and space to do so. I guess that efficiency is then a big part of the puzzle on why the increased bandwidth to ram makes such an impact?
sliken 5y ago
Many things don't push the memory bandwidth and are cache friendly. However GPUs are bandwidth limited and the M1 Max does quite well against any other integrated graphics from Intel or AMD. Even on the CPU side it can be a big win, in particular on SpecFPRate (a collection of heavy floating point real world codes, not microbenchmarks) Anand has this to say: The fp2017 suite has more workloads that are more memory-bound, and it’s here where the M1 Max is absolutely absurd. The workloads that put the most memory pressure and stress the DRAM the most, such as 503.bwaves, 519.lbm, 549.fotonik3d and 554.roms, have all multiple factors of performance advantages compared to the best Intel and AMD have to offer. To drive this home compare the Spec2017 FP Rate, the M1 Max gets 81.07, the Ryzen 5950x (high end desktop with twice as many fast cores and a 105 watt TDP) gets 62.27. So a low power M1 Max with half as many cores and much lower power is 30% faster than AMD's highest end desktop chip. Instead of a desktop size/volume/power, you can get it in a laptop that's 2/3rd of an inch thick.

sliken 5y ago

That's not at all what I'm seeing. Can you be specific as to what chips you are looking at and what benchmark you are using to draw this conclusion? I'll go first. Personally when talking about memory latency, I want to measure time to main memory, without TLB thrashing. Not that TLB latency isn't interesting, but it should be separately quantified. I've written a few microbenchmarks in this area, but I get very similar numbers to the Anandtech R per RV prange. Which puts the latency to main memory at around 35ns looking like it might flatten out at 40ns. Yes, full TLB thrashing is up at 110ns or so, but that's not a usual use case. If it is, at least under linux, you can switch to 1GB pages if that's important to you. R per RV prange numbers on the Ryzen 9 5950x is 65ns or so. So sure the TLB worst case is higher latency on the M1, but then you have to figure out how big the TLB is, and how often that's an issue if you want to know the real world performance impact. The i9-12900K with DDR5 gets 30ns on the R per RV prange, and 92ns on full random (tlb thrashing). Even assuming the worst case TLB behavior the m1 max is 111ns and the 5950x is 79ns or 1.4x higher. On the Intel side is 111ns vs 92n, 1.2x higher. Finding it hard to find any numbers that make the M1X look like 3x higher memory latency.

formerly_proven 5y ago
I feel like R per R isn't quite the perfect match for the pointer-chasy workloads I have in mind where you end up going through more-or-less random selections of relatively small (<< page size) objects, so R per R amortizes the randomization a little much. I still think having the overall totally random access for a system is a good number precisely because it contains all the penalties and latencies that can add up. Embarrassing as it may be, I also compared the wrong numbers across parts here (and in the past too, I think) - thanks for calling that out. I was kinda irritated how exactly they would be able to get the performance they were getting across all workloads with such poor latency but failed to double check.
- sliken 5y ago
  Ah, sensible. Generally with the TLB numbers and the memory latency you can get a good idea of the performance by interoplating between the two number for any number of TLB misses (from 0% to 100%). The M1 Max also has a crazy number of memory channels, even the older M1 has 4 channels, pro has 8 channels, and max has 16. So you can have many more cache misses in flight, this is part of why the m1 max is 20% faster than the 5950x with twice as many cores.