3 ms·
A singular CCD on Zen3 can max out the read bandwidth of two DDR4 channels at the latency-optimal 1:1 gearing between the IFOP clock and the DDR4 clock (i.e., 1
by namibj 5y ago
A singular CCD on Zen3 can max out the read bandwidth of two DDR4 channels at the latency-optimal 1:1 gearing between the IFOP clock and the DDR4 clock (i.e., 1600 MHz clock for DDR4-3200), but it's write bandwidth is only half as much (IFOP has asymmetric bandwidth).
I haven't looked into detailed limits of the server northbridge, but the consumer northbridge has heaps of PCIe bandwidth compared to the relatively low DDR4 bandwidth.
Even though PCIe4 is full-duplex, just the read bandwidth (after accounting for (in practice) unavoidable losses from DDR4's address bus nature) of the x16+x4 northbridge PCIe4 lanes plus the x4 PCIe4 lanes connecting the southbridge to the northbridge (which are re-emitted on X570 with the southbridge acting as a PCIe switch) exceeds the DDR4-2666 bandwidth.
So assuming that DDR4-2666 RAM with the infinity fabric geared 1:1 at 1333 MHz, a Ryzen9 5950X's two CCDs can (together/combined) read 42 GB/s from DRAM, read 42 GB/s from PCIe, and write 42 GB/s to PCIe.
This should break even with the peak L1$ bandwidth if the core is running at half the IFOP clock.
Granted, in reality DDR4-3800 runs fine on most R9-5950X's (some chips don't work at the 1800 MHz IFOP; those will still run at DDR4-3600 in almost all cases, given high-quality DRAM (16 Gbit Micron Rev.E dies are used in e.g. Kingston Server Premier 32GB 2Rx8 "DDR4-3200 CL22" ECC-UDIMMs and my two sticks work at 3600 CL18 normally, even reaching 3600 (with CL 30+; this was a signal integrity test) in the same channel suggesting DDR4-3600 should work with 128 GiB RAM)).
And that memory/IFOP speed is 42% above the 2666/1333 speed I equated to the available PCIe4x4 bandwidth.
It's still damming to see that for sheer full-duplex bandwidth, DDR4 quite easily wins, considering 16 lanes full-duplex match the practical full-duplex ceiling of 2 DDR4-3600 channels if you measure bandwidth at the CPU's L2$.