5 ms·

Actually, the trend is to put the RAM inside the main chip itself so as to avoid those pesky latency and area-costly PCB traces. The performance improvements th

Actually, the trend is to put the RAM inside the main chip itself so as to avoid those pesky latency and area-costly PCB traces. The performance improvements this allows are hard to resist and some manufacturers are already falling in (e.g. Apple). And I guess the side effect of making upgrades impossible is nice, too.

mciancia 5y ago
Is there actually noticably lower latency when going from sodimm/on MB RAM to memory integrated in SoC? I would expect bottlenecks to be somewhere else, not in those few extra centimeters of traces
- AshamedCaptain 5y ago
  Look at Apple's or HBM benchmarks. Also, it's not just the length, but the extra contact points, the "safety" margins for it all (to allow some extra routing distance for lazier/complicated mobo design), etc. all force you to a lower frequency, which also reduces bandwidth. (IMHO, I would not trade off modularity for that. But the public disagrees.)
  - floatboth 5y ago
    Apple uses the exact same LPDDR4x that others route through the mainboard, in stock JEDEC spec, so they don’t really get performance advantages from memory on package.
    - AshamedCaptain 5y ago
      It's the chipset & CPU that have to slowdown to account for the extra cost to reach it, hidden in whatever stage you call it -- even if the actual DRAM chips may be the same. They're the dumbest part in this entire diagram. Also, they benefit from larger bus widths, lower power, etc. https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... ( despite this being DDR4 I have to go to DDR5-systems to find similar bandwidth & latency results ) , https://www.anandtech.com/show/17024/apple-m1-max-performance-review https://www.anandtech.com/show/17024/apple-m1-max-performanc... and https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc... ( I have to go quad-channel DDR5 or HBM to find similar results elsewhere)
      - floatboth 5y ago
        Time cost is dictated by the clock frequency and timings, not the physical distance itself. Lower distance may help with achieving better specs but Apple is not overclocking the memory, they run JEDEC spec. Maaaybe they train secondary and tertiary timings a little bit tighter, that’s about it. > despite this being DDR4 It is not! LPDDR IS NOT DDR!! It’s a very very different standard despite the similar name.
        AshamedCaptain 5y ago
        > It is not! LPDDR IS NOT DDR!! And is that what explains the apparent increase in memory bandwidth?
        floatboth 5y ago
        Yes, LPDDR4X in particular is rated for 4266 Mbps (at least that's the only spec available from https://www.samsung.com/semiconductor/dram/lpddr4x/ https://www.samsung.com/semiconductor/dram/lpddr4x/ right now) while the top official JEDEC spec for DDR4 is 3200 Mbps. 4266 Mbps * 8 channels * 2 (bytes in a 16-bit channel) = 68.25GB/s theoretical maximum. (Or 4 channels * 4 bytes in a 32-bit channel, same deal.) See AIDA64 bandwidth benchmark results: https://storage-asset.msi.com/global/picture/news/2021/nb/lpddr-20210104-8.jpg https://storage-asset.msi.com/global/picture/news/2021/nb/lp... — these are from MSI's Intel Tigerlake laptops. There are AMD Renoir laptops with the same memory configuration too. This is not unique to Apple, this is what everyone has now. I guess the new big MBPs might have the largest version of this, but what the regular M1 has is not special whatsoever.
    - willyt 5y ago
      They get power savings from reduced capacitance by reducing the distance from the memory to the CPU from a few cm to a few mm.
- Stevvo 5y ago
  Yes, when you get to the very high frequencies of modern RAM, physical distance is significant. For years overclockers have been using mini-ITX motherboards to get the RAM closer to the CPU for higher frequencies.
  - floatboth 5y ago
    ITX usage was never about trace length, it’s about having a 2-DIMM board for running 2 sticks – Unoccupied DIMM sockets are terrible! These days we have the luxury of overclocking boards that are 2-DIMM in ATX form factor.
- formerly_proven 5y ago
  > Is there actually noticably lower latency when going from sodimm/on MB RAM to memory integrated in SoC? No. But it's easier and lower power to run higher bus frequencies for higher bandwidths. Same goes for bus width.
- phire 5y ago
  There isn't any latency reduction on the M1, Apple use the exact same 4,266 MT/s LPDDR4X that can be found soldered onto the motherboards on a few window's laptops, and they are just using the stock timings. I think the main reason Apple are doing it is to simplify their motherboard PCB design and make the design more compact.
dehrmann 5y ago
Is Intel making SoCs like this yet? All the Core, Pentium, etc. CPUs I've seen are still just CPUs. True Apple is doing this, and ARM Chromebooks probably do it, too.
- AshamedCaptain 5y ago
  Intel also makes it, kinda: https://www.anandtech.com/show/16195/a-broadwell-retrospective-review-in-2020-is-edram-still-worth-it https://www.anandtech.com/show/16195/a-broadwell-retrospecti...
rasz 5y ago
You bought into Apple marketing. Speed of signal in copper ~20cm per 1ns. 5-7cm of extra track length to a slot would not hurt anything.
- AshamedCaptain 5y ago
  Oh my god, did you just discredit the entire field of EDA place & route in just one sentence? This is not your usual 20MHz bus. These are signals that switch at 2, 3, 4 GHz, and there are a shitton of effects that get in the way and limit how fast you can toggle a (longer) wire. Or at least significantly alter the power vs latency vs speed vs noise tradeoffs. (and no, I most certainly did not buy into apple marketing, since I'm pointing that as a negative)
  - rasz 5y ago
    Did you miss the part where Apple is clocking those chips at same speeds as normal DDR4? Closer placement did not result in lower latency nor higher speeds. Apple M1 is same situation as RPI foundation using POP ram. Requires fewer pcb layers, simplifies layout, lowers cost and as a bonus gives assured future sales due to forced obsolescence. I didnt dismiss anything, I simply know physics and have some experience in high speed layout.
    - AshamedCaptain 5y ago
      I can understand the claim the M1 has the same timings as off-the-shelf DDR4 (thought this was a nice way to explain the apparent increases in memory bandwidth shown in benchmarks), but here you are dismissing the concept of HBM entirely.
      - rasz 5y ago
        Afaik M1s dont use HBM (which is 1024bit per chip and are their own thing), but ordinary POP ram like RPI. Anything up to 512bits is just cost cutting. Look at Xbox - no problem fanning out 320bit bus in addition to pcie lanes, same for hiend GPUs. 4096bit is where you can claim someone actually uses HBM for performance, like on Radeon R9 Nano/Fury.

mciancia 5y ago

Is there actually noticably lower latency when going from sodimm/on MB RAM to memory integrated in SoC? I would expect bottlenecks to be somewhere else, not in those few extra centimeters of traces

AshamedCaptain 5y ago
Look at Apple's or HBM benchmarks. Also, it's not just the length, but the extra contact points, the "safety" margins for it all (to allow some extra routing distance for lazier/complicated mobo design), etc. all force you to a lower frequency, which also reduces bandwidth. (IMHO, I would not trade off modularity for that. But the public disagrees.)
- floatboth 5y ago
  Apple uses the exact same LPDDR4x that others route through the mainboard, in stock JEDEC spec, so they don’t really get performance advantages from memory on package.
  - AshamedCaptain 5y ago
    It's the chipset & CPU that have to slowdown to account for the extra cost to reach it, hidden in whatever stage you call it -- even if the actual DRAM chips may be the same. They're the dumbest part in this entire diagram. Also, they benefit from larger bus widths, lower power, etc. https://www.anandtech.com/show/16252/mac-mini-apple-m1-tested https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste... ( despite this being DDR4 I have to go to DDR5-systems to find similar bandwidth & latency results ) , https://www.anandtech.com/show/17024/apple-m1-max-performance-review https://www.anandtech.com/show/17024/apple-m1-max-performanc... and https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc... ( I have to go quad-channel DDR5 or HBM to find similar results elsewhere)
    - floatboth 5y ago
      Time cost is dictated by the clock frequency and timings, not the physical distance itself. Lower distance may help with achieving better specs but Apple is not overclocking the memory, they run JEDEC spec. Maaaybe they train secondary and tertiary timings a little bit tighter, that’s about it. > despite this being DDR4 It is not! LPDDR IS NOT DDR!! It’s a very very different standard despite the similar name.
      - AshamedCaptain 5y ago
        > It is not! LPDDR IS NOT DDR!! And is that what explains the apparent increase in memory bandwidth?
        floatboth 5y ago
        Yes, LPDDR4X in particular is rated for 4266 Mbps (at least that's the only spec available from https://www.samsung.com/semiconductor/dram/lpddr4x/ https://www.samsung.com/semiconductor/dram/lpddr4x/ right now) while the top official JEDEC spec for DDR4 is 3200 Mbps. 4266 Mbps * 8 channels * 2 (bytes in a 16-bit channel) = 68.25GB/s theoretical maximum. (Or 4 channels * 4 bytes in a 32-bit channel, same deal.) See AIDA64 bandwidth benchmark results: https://storage-asset.msi.com/global/picture/news/2021/nb/lpddr-20210104-8.jpg https://storage-asset.msi.com/global/picture/news/2021/nb/lp... — these are from MSI's Intel Tigerlake laptops. There are AMD Renoir laptops with the same memory configuration too. This is not unique to Apple, this is what everyone has now. I guess the new big MBPs might have the largest version of this, but what the regular M1 has is not special whatsoever.
  - willyt 5y ago
    They get power savings from reduced capacitance by reducing the distance from the memory to the CPU from a few cm to a few mm.
Stevvo 5y ago
Yes, when you get to the very high frequencies of modern RAM, physical distance is significant. For years overclockers have been using mini-ITX motherboards to get the RAM closer to the CPU for higher frequencies.
- floatboth 5y ago
  ITX usage was never about trace length, it’s about having a 2-DIMM board for running 2 sticks – Unoccupied DIMM sockets are terrible! These days we have the luxury of overclocking boards that are 2-DIMM in ATX form factor.
formerly_proven 5y ago
> Is there actually noticably lower latency when going from sodimm/on MB RAM to memory integrated in SoC? No. But it's easier and lower power to run higher bus frequencies for higher bandwidths. Same goes for bus width.
phire 5y ago
There isn't any latency reduction on the M1, Apple use the exact same 4,266 MT/s LPDDR4X that can be found soldered onto the motherboards on a few window's laptops, and they are just using the stock timings. I think the main reason Apple are doing it is to simplify their motherboard PCB design and make the design more compact.

dehrmann 5y ago

Is Intel making SoCs like this yet? All the Core, Pentium, etc. CPUs I've seen are still just CPUs. True Apple is doing this, and ARM Chromebooks probably do it, too.

AshamedCaptain 5y ago
Intel also makes it, kinda: https://www.anandtech.com/show/16195/a-broadwell-retrospective-review-in-2020-is-edram-still-worth-it https://www.anandtech.com/show/16195/a-broadwell-retrospecti...

rasz 5y ago

You bought into Apple marketing. Speed of signal in copper ~20cm per 1ns. 5-7cm of extra track length to a slot would not hurt anything.

AshamedCaptain 5y ago
Oh my god, did you just discredit the entire field of EDA place & route in just one sentence? This is not your usual 20MHz bus. These are signals that switch at 2, 3, 4 GHz, and there are a shitton of effects that get in the way and limit how fast you can toggle a (longer) wire. Or at least significantly alter the power vs latency vs speed vs noise tradeoffs. (and no, I most certainly did not buy into apple marketing, since I'm pointing that as a negative)
- rasz 5y ago
  Did you miss the part where Apple is clocking those chips at same speeds as normal DDR4? Closer placement did not result in lower latency nor higher speeds. Apple M1 is same situation as RPI foundation using POP ram. Requires fewer pcb layers, simplifies layout, lowers cost and as a bonus gives assured future sales due to forced obsolescence. I didnt dismiss anything, I simply know physics and have some experience in high speed layout.
  - AshamedCaptain 5y ago
    I can understand the claim the M1 has the same timings as off-the-shelf DDR4 (thought this was a nice way to explain the apparent increases in memory bandwidth shown in benchmarks), but here you are dismissing the concept of HBM entirely.
    - rasz 5y ago
      Afaik M1s dont use HBM (which is 1024bit per chip and are their own thing), but ordinary POP ram like RPI. Anything up to 512bits is just cost cutting. Look at Xbox - no problem fanning out 320bit bus in addition to pcie lanes, same for hiend GPUs. 4096bit is where you can claim someone actually uses HBM for performance, like on Radeon R9 Nano/Fury.