3 ms·

For example the latest AMD Genoa has 12 memory channels per socket, at dual socket and with enough DIMMs that's a 75x speed advantage even if you compare with R

For example the latest AMD Genoa has 12 memory channels per socket, at dual socket and with enough DIMMs that's a 75x speed advantage even if you compare with RAID0 of high perfomance NVMe.

wtallis 3y ago
> even if you compare with RAID0 of high perfomance NVMe. Do you mean a RAID0 of just two or four NVMe SSDs? It's absolutely ridiculous to count aggregate DRAM bandwidth across two CPU sockets and not do the same for PCIe lanes. A fair comparison is that Genoa has about twice the DRAM bandwidth as it has PCIe bandwidth, though in a fully-loaded database server some portion of the PCIe bandwidth will be used for networking rather than storage.
- zamadatix 3y ago
  FWIW I think you have the aggregation backwards, dual socket Genoa would have 24 channels of DRAM but sacrifice some of the PCIe lanes for the interconnect. Your numbers work out right though as single socket is actually about 1:1 in RAM vs PCIe bandwidth so dual socket would still come out roughly 2:1. I think 10x is a fair rough number though, depending on your access pattern.
  - wtallis 3y ago
    Dual-socket Genoa would be 24 channels at DDR5-4800 (38.4 GB/s) for a total of ~921.6 GB/s. Typical PCIe configurations are 64 or 80 lanes per socket for dual-socket, so 128 or 160 lanes total, at PCIe 5.0 speed that's ~504 GB/s for 128 lanes total or ~630 GB/s for 160 lane configurations. Single-socket Genoa would be 12 channels of DRAM (~460.8 GB/s) and 128 lanes of PCIe 5.0 (~504 GB/s), but none of the previous comments were specifically about single-socket Genoa and I wasn't going to silently switch from considering dual-socket in one sentence to single-socket in the next sentence.
    - zamadatix 3y ago
      Ah yes, I forgot about the 48 lane 2 socket interconnect mode which does allow you to still aggregate some additional lanes.

3 ms·

For example the latest AMD Genoa has 12 memory channels per socket, at dual socket and with enough DIMMs that's a 75x speed advantage even if you compare with R

by semi-extrinsic 3y ago

For example the latest AMD Genoa has 12 memory channels per socket, at dual socket and with enough DIMMs that's a 75x speed advantage even if you compare with RAID0 of high perfomance NVMe.

wtallis 3y ago
> even if you compare with RAID0 of high perfomance NVMe. Do you mean a RAID0 of just two or four NVMe SSDs? It's absolutely ridiculous to count aggregate DRAM bandwidth across two CPU sockets and not do the same for PCIe lanes. A fair comparison is that Genoa has about twice the DRAM bandwidth as it has PCIe bandwidth, though in a fully-loaded database server some portion of the PCIe bandwidth will be used for networking rather than storage.
- zamadatix 3y ago
  FWIW I think you have the aggregation backwards, dual socket Genoa would have 24 channels of DRAM but sacrifice some of the PCIe lanes for the interconnect. Your numbers work out right though as single socket is actually about 1:1 in RAM vs PCIe bandwidth so dual socket would still come out roughly 2:1. I think 10x is a fair rough number though, depending on your access pattern.
  - wtallis 3y ago
    Dual-socket Genoa would be 24 channels at DDR5-4800 (38.4 GB/s) for a total of ~921.6 GB/s. Typical PCIe configurations are 64 or 80 lanes per socket for dual-socket, so 128 or 160 lanes total, at PCIe 5.0 speed that's ~504 GB/s for 128 lanes total or ~630 GB/s for 160 lane configurations. Single-socket Genoa would be 12 channels of DRAM (~460.8 GB/s) and 128 lanes of PCIe 5.0 (~504 GB/s), but none of the previous comments were specifically about single-socket Genoa and I wasn't going to silently switch from considering dual-socket in one sentence to single-socket in the next sentence.
    - zamadatix 3y ago
      Ah yes, I forgot about the 48 lane 2 socket interconnect mode which does allow you to still aggregate some additional lanes.