3 ms·
I'm surprised that 4NPS gives even a mild benefit to bandwidth (maybe 13% or so), given that the central IO die handles all communication anyway. You'd think th
by dragontamer 7y ago
I'm surprised that 4NPS gives even a mild benefit to bandwidth (maybe 13% or so), given that the central IO die handles all communication anyway. You'd think that with the architecture here, there'd be no real benefit to splitting the RAM channels.
- drewg123 7y agoI was surprised by this as well, but my results for a different memory bandwidth bound workload match what Dell is reporting: When testing a rome CPU at Netflix, I found I could serve 200+Gb/s of TLS encrypted video with 4nps, 192Gb/s with 2nps, and 184Gb/s with 1NPS. My best guess was that I didn't fully understand the architecture of the IO die. Eg, there was some benefit to being local that I didn't understand fully.
- loeg 7y agoPerhaps, as article sort of suggests, inside the IO chiplet there are four quadrants that each have two memory controllers associated with them, and the cross-quadrant bandwidth isn't sufficient for 75% non-local memory bandwidth.
- wmf 7y agoI made a video yesterday with some details on this topic: https://youtu.be/ghFx_jyP1U8?t=390 https://youtu.be/ghFx_jyP1U8?t=390
- dbancajas 7y agoBut there is concept of local and remote memory for NPS4 vs NPS1 right? NPS4 only accesses memory in 1 quadrant while NPS1 distributes it to all corners. The difference in latency + all other queuing delays in switches can add up to the difference.
- toast0 7y agoIf your stack is NUMA aware, and can manage that 4/8 partitions, enabling it would seem to reduce memory contention between chiplets; even if there isn't a latency penalty for going a bit farther to memory (which there likely is, but probably small)
- dragontamer 7y agoThe STREAM benchmark doesn't have any memory contention at all. Its basically a full tilt "write to memory / read from memory" kind of benchmark.
- toast0 7y agoI would expect memory bus contention in a benchmark like that.
- old-gregg 7y agoThe central I/O die does not have any cache on it. L3 caches are shared across just 4 cores in a CCX (not even CCD). IIRC the next generation of Ryzen will introduce CCD or even I/O die caches. I think one of Intel's Broadwell chips had L4, so it's the same idea.