3 ms·

> I believe Apple was the first to ship something wider than 128-bit in a laptop Yes, that is now more obvious to me after your clarification and a short histo

> I believe Apple was the first to ship something wider than 128-bit in a laptop Yes, that is now more obvious to me after your clarification and a short history walk-through. Thanks! Given that Apple never discloses much about their design, I was never actually 100% sure if the difference would only be related to the number of memory channels and thus capability of a memory controller or there was something else that was novel and that nobody else did. I am much more familiar with the server grade CPUs and was aware, as you very neatly summarized, of the differences and high memory bandwidths available in that spectrum. I was checking just this morning how many channels there are on Epycs and last generation of Xeons! Since we have different 128-bit wide designs available, it is interesting that Apple M1 chose to have 128B cache line size instead what others have - 64B. I never thought about that but it makes me wonder if it is more challenging to design a memory subsystem with the cache line size larger than what your DIMM actually supports.

wtallis 3y ago
Memory bus widths are usually measured in bits while cache line sizes are bytes. The problem isn't with having a cache line that's too big, but rather too small: DDR3 and DDR4 both used burst sizes of eight transfers of 64 bits each, for a total burst of 64 bytes. DDR5 increased the burst size to 16 transfers, and split the DIMM into two subchannels to compensate—otherwise, it would be impossible to use the full bandwidth when fetching individual 64Byte cache lines. Transferring two bursts of data from RAM (not necessarily from the same channel) to fetch a single cache line worked fine in the DDR2 era.
- menaerus 3y ago
  I see that the burst length for LPDDR4/LPDDR5 is configurable as either 16 transfers (32B) or 32 transfers (64B) which means that fetching a single 128B cache line requires Apple M memory controller to issue 2 or 4 read bursts depending on the setting used. According to the page 78 from https://www.micron.com/-/media/client/global/documents/products/data-sheet/dram/mobile-dram/low-power-dram/lpddr4/z19m_embedded_lpddr4_lpddr4x.pdf https://www.micron.com/-/media/client/global/documents/produ..., burst read operation will take 16 clock cycles to clock in, and roughly 4 more cycles to get the data out. At 2133Mhz clock cycle is 468 ps or 0.46 ns so to get 64 bytes out it will take 20 cycles x 0.46 ns = 9.2 ns. This means that to populate the 128B cache line it will take at least two such bursts leading to a total latency of 18.4 ns. This latency translates to ~6GB/s of burst reads. I guess this is a number only for a single channel and it's relatively close to what Micron mentions in the PDF: "Up to 8.5 GB/s per die". What I don't understand is how the LPDDR4 data rate transfer of 3200 MT/s (~25GB/s) is achieved then? The biggest package Micron LPDDR4 module comes in is 2 channels, hence, per their numbers we get a maximum bandwidth of 2 x 8.5GB/s = 17GB/s.
  - menaerus 3y ago
    > The biggest package Micron LPDDR4 module comes in is 2 channels Okay, this is wrong, I couldn't find it before but there is a larger Micron LPDDR4 module equipped with 4 channels and thus this theoretical LPDDR4 limit can be hit. E.g. MT53D1024M32D4 from https://www.micron.com/-/media/client/global/documents/products/data-sheet/dram/mobile-dram/low-power-dram/lpddr4/200b_z11m_non_auto_lpddr4_lpddr4x.pdf?rev=6e4d27ad107040949c35ab17b491ad2a https://www.micron.com/-/media/client/global/documents/produ...

wtallis 3y ago

Memory bus widths are usually measured in bits while cache line sizes are bytes. The problem isn't with having a cache line that's too big, but rather too small: DDR3 and DDR4 both used burst sizes of eight transfers of 64 bits each, for a total burst of 64 bytes. DDR5 increased the burst size to 16 transfers, and split the DIMM into two subchannels to compensate—otherwise, it would be impossible to use the full bandwidth when fetching individual 64Byte cache lines. Transferring two bursts of data from RAM (not necessarily from the same channel) to fetch a single cache line worked fine in the DDR2 era.

menaerus 3y ago
I see that the burst length for LPDDR4/LPDDR5 is configurable as either 16 transfers (32B) or 32 transfers (64B) which means that fetching a single 128B cache line requires Apple M memory controller to issue 2 or 4 read bursts depending on the setting used. According to the page 78 from https://www.micron.com/-/media/client/global/documents/products/data-sheet/dram/mobile-dram/low-power-dram/lpddr4/z19m_embedded_lpddr4_lpddr4x.pdf https://www.micron.com/-/media/client/global/documents/produ..., burst read operation will take 16 clock cycles to clock in, and roughly 4 more cycles to get the data out. At 2133Mhz clock cycle is 468 ps or 0.46 ns so to get 64 bytes out it will take 20 cycles x 0.46 ns = 9.2 ns. This means that to populate the 128B cache line it will take at least two such bursts leading to a total latency of 18.4 ns. This latency translates to ~6GB/s of burst reads. I guess this is a number only for a single channel and it's relatively close to what Micron mentions in the PDF: "Up to 8.5 GB/s per die". What I don't understand is how the LPDDR4 data rate transfer of 3200 MT/s (~25GB/s) is achieved then? The biggest package Micron LPDDR4 module comes in is 2 channels, hence, per their numbers we get a maximum bandwidth of 2 x 8.5GB/s = 17GB/s.
- menaerus 3y ago
  > The biggest package Micron LPDDR4 module comes in is 2 channels Okay, this is wrong, I couldn't find it before but there is a larger Micron LPDDR4 module equipped with 4 channels and thus this theoretical LPDDR4 limit can be hit. E.g. MT53D1024M32D4 from https://www.micron.com/-/media/client/global/documents/products/data-sheet/dram/mobile-dram/low-power-dram/lpddr4/200b_z11m_non_auto_lpddr4_lpddr4x.pdf?rev=6e4d27ad107040949c35ab17b491ad2a https://www.micron.com/-/media/client/global/documents/produ...