8 ms·

Is there actually noticably lower latency when going from sodimm/on MB RAM to memory integrated in SoC? I would expect bottlenecks to be somewhere else, not in

Is there actually noticably lower latency when going from sodimm/on MB RAM to memory integrated in SoC? I would expect bottlenecks to be somewhere else, not in those few extra centimeters of traces

AshamedCaptain 5y ago
Look at Apple's or HBM benchmarks. Also, it's not just the length, but the extra contact points, the "safety" margins for it all (to allow some extra routing distance for lazier/complicated mobo design), etc. all force you to a lower frequency, which also reduces bandwidth. (IMHO, I would not trade off modularity for that. But the public disagrees.)
- floatboth 5y ago
  Apple uses the exact same LPDDR4x that others route through the mainboard, in stock JEDEC spec, so they don’t really get performance advantages from memory on package.
  - AshamedCaptain 5y ago
    It's the chipset & CPU that have to slowdown to account for the extra cost to reach it, hidden in whatever stage you call it -- even if the actual DRAM chips may be the same. They're the dumbest part in this entire diagram. Also, they benefit from larger bus widths, lower power, etc. https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... ( despite this being DDR4 I have to go to DDR5-systems to find similar bandwidth & latency results ) , https://www.anandtech.com/show/17024/apple-m1-max-performance-review https://www.anandtech.com/show/17024/apple-m1-max-performanc... and https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc... ( I have to go quad-channel DDR5 or HBM to find similar results elsewhere)
    - floatboth 5y ago
      Time cost is dictated by the clock frequency and timings, not the physical distance itself. Lower distance may help with achieving better specs but Apple is not overclocking the memory, they run JEDEC spec. Maaaybe they train secondary and tertiary timings a little bit tighter, that’s about it. > despite this being DDR4 It is not! LPDDR IS NOT DDR!! It’s a very very different standard despite the similar name.
      - AshamedCaptain 5y ago
        > It is not! LPDDR IS NOT DDR!! And is that what explains the apparent increase in memory bandwidth?
        floatboth 5y ago
        Yes, LPDDR4X in particular is rated for 4266 Mbps (at least that's the only spec available from https://www.samsung.com/semiconductor/dram/lpddr4x/ https://www.samsung.com/semiconductor/dram/lpddr4x/ right now) while the top official JEDEC spec for DDR4 is 3200 Mbps. 4266 Mbps * 8 channels * 2 (bytes in a 16-bit channel) = 68.25GB/s theoretical maximum. (Or 4 channels * 4 bytes in a 32-bit channel, same deal.) See AIDA64 bandwidth benchmark results: https://storage-asset.msi.com/global/picture/news/2021/nb/lpddr-20210104-8.jpg https://storage-asset.msi.com/global/picture/news/2021/nb/lp... — these are from MSI's Intel Tigerlake laptops. There are AMD Renoir laptops with the same memory configuration too. This is not unique to Apple, this is what everyone has now. I guess the new big MBPs might have the largest version of this, but what the regular M1 has is not special whatsoever.
  - willyt 5y ago
    They get power savings from reduced capacitance by reducing the distance from the memory to the CPU from a few cm to a few mm.
Stevvo 5y ago
Yes, when you get to the very high frequencies of modern RAM, physical distance is significant. For years overclockers have been using mini-ITX motherboards to get the RAM closer to the CPU for higher frequencies.
- floatboth 5y ago
  ITX usage was never about trace length, it’s about having a 2-DIMM board for running 2 sticks – Unoccupied DIMM sockets are terrible! These days we have the luxury of overclocking boards that are 2-DIMM in ATX form factor.
formerly_proven 5y ago
> Is there actually noticably lower latency when going from sodimm/on MB RAM to memory integrated in SoC? No. But it's easier and lower power to run higher bus frequencies for higher bandwidths. Same goes for bus width.
phire 5y ago
There isn't any latency reduction on the M1, Apple use the exact same 4,266 MT/s LPDDR4X that can be found soldered onto the motherboards on a few window's laptops, and they are just using the stock timings. I think the main reason Apple are doing it is to simplify their motherboard PCB design and make the design more compact.

Is there actually noticably lower latency when going from sodimm/on MB RAM to memory integrated in SoC? I would expect bottlenecks to be somewhere else, not in those few extra centimeters of traces

AshamedCaptain 5y ago

Look at Apple's or HBM benchmarks. Also, it's not just the length, but the extra contact points, the "safety" margins for it all (to allow some extra routing distance for lazier/complicated mobo design), etc. all force you to a lower frequency, which also reduces bandwidth. (IMHO, I would not trade off modularity for that. But the public disagrees.)

floatboth 5y ago
Apple uses the exact same LPDDR4x that others route through the mainboard, in stock JEDEC spec, so they don’t really get performance advantages from memory on package.
- AshamedCaptain 5y ago
  It's the chipset & CPU that have to slowdown to account for the extra cost to reach it, hidden in whatever stage you call it -- even if the actual DRAM chips may be the same. They're the dumbest part in this entire diagram. Also, they benefit from larger bus widths, lower power, etc. https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... ( despite this being DDR4 I have to go to DDR5-systems to find similar bandwidth & latency results ) , https://www.anandtech.com/show/17024/apple-m1-max-performance-review https://www.anandtech.com/show/17024/apple-m1-max-performanc... and https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc... ( I have to go quad-channel DDR5 or HBM to find similar results elsewhere)
  - floatboth 5y ago
    Time cost is dictated by the clock frequency and timings, not the physical distance itself. Lower distance may help with achieving better specs but Apple is not overclocking the memory, they run JEDEC spec. Maaaybe they train secondary and tertiary timings a little bit tighter, that’s about it. > despite this being DDR4 It is not! LPDDR IS NOT DDR!! It’s a very very different standard despite the similar name.
    - AshamedCaptain 5y ago
      > It is not! LPDDR IS NOT DDR!! And is that what explains the apparent increase in memory bandwidth?
      - floatboth 5y ago
        Yes, LPDDR4X in particular is rated for 4266 Mbps (at least that's the only spec available from https://www.samsung.com/semiconductor/dram/lpddr4x/ https://www.samsung.com/semiconductor/dram/lpddr4x/ right now) while the top official JEDEC spec for DDR4 is 3200 Mbps. 4266 Mbps * 8 channels * 2 (bytes in a 16-bit channel) = 68.25GB/s theoretical maximum. (Or 4 channels * 4 bytes in a 32-bit channel, same deal.) See AIDA64 bandwidth benchmark results: https://storage-asset.msi.com/global/picture/news/2021/nb/lpddr-20210104-8.jpg https://storage-asset.msi.com/global/picture/news/2021/nb/lp... — these are from MSI's Intel Tigerlake laptops. There are AMD Renoir laptops with the same memory configuration too. This is not unique to Apple, this is what everyone has now. I guess the new big MBPs might have the largest version of this, but what the regular M1 has is not special whatsoever.
- willyt 5y ago
  They get power savings from reduced capacitance by reducing the distance from the memory to the CPU from a few cm to a few mm.

Stevvo 5y ago

Yes, when you get to the very high frequencies of modern RAM, physical distance is significant. For years overclockers have been using mini-ITX motherboards to get the RAM closer to the CPU for higher frequencies.

floatboth 5y ago
ITX usage was never about trace length, it’s about having a 2-DIMM board for running 2 sticks – Unoccupied DIMM sockets are terrible! These days we have the luxury of overclocking boards that are 2-DIMM in ATX form factor.

formerly_proven 5y ago

> Is there actually noticably lower latency when going from sodimm/on MB RAM to memory integrated in SoC? No. But it's easier and lower power to run higher bus frequencies for higher bandwidths. Same goes for bus width.

phire 5y ago

There isn't any latency reduction on the M1, Apple use the exact same 4,266 MT/s LPDDR4X that can be found soldered onto the motherboards on a few window's laptops, and they are just using the stock timings. I think the main reason Apple are doing it is to simplify their motherboard PCB design and make the design more compact.