3 ms·

Just checking my intuition: could we still get a speedup via pipelined execution and branch prediction? From what i see reading https://en.wikipedia.org/wiki/M

by znpy 3y ago

Just checking my intuition: could we still get a speedup via pipelined execution and branch prediction? From what i see reading https://en.wikipedia.org/wiki/Multi-channel_memory_architecture#Ganged_versus_unganged https://en.wikipedia.org/wiki/Multi-channel_memory_architect... different channels could, in theory, be used "autonomously of each other".

Lramseyer 3y ago
Controllers kind of do that. At the end of the day, it's what makes designing a memory controller so difficult (and I'm not even talking about the Phy, those things are straight up cursed!) We see these eye popping numbers for maximum potential bandwidths, but the reality is a bit more complicated. There's a lot that goes on behind the scenes with opening and closing memory banks, refreshes, and general read and write latencies. Unoptimized prediction algorithms (as they are programmable) can result in losing _half_ of your performance.
sliken 3y ago
Pipelining helps, usually. You can have 10 stages run at 3 GHz instead of 1 stage at 300 MHz. Up until you have a branch misprediction and have to dump the pipeline. Memory channels are independent, however generally all cores use all channels. There are two common designs. Intel has 4 dies, 2 memory channels per core, so 2 channels are closer/lower latency than the other 6. AMD has multiple chiplets, but a single memory controller with 12 channels. So All cores have the same latency to all channels. Generally Intel has lower latencies to 25% of the channels, but AMD has more throughput (bandwidth or random IOPs). One thing that surprised me is that for maximum throughput you want at cache misses queued to the memory controller, at least twice the number of memory channels. These days missing in L1/L2/L3 is often approximately half the total memory latency. So on an Intel Xeon at least 16 misses (per socket), AMD at least 24 misses (per socket.). So on Intel you could tune things (and the NUMA support helps) to prefer the local channels. Most OSs help, and C calls like numa_alloc_local() allows local control. For memory intensive codes I have found the best scaling when there's 2 cores per channel. Of course most codes are pretty friendly.
- znpy 3y ago
  thank you for the explanation!

by znpy 3y ago

Lramseyer 3y ago

Controllers kind of do that. At the end of the day, it's what makes designing a memory controller so difficult (and I'm not even talking about the Phy, those things are straight up cursed!) We see these eye popping numbers for maximum potential bandwidths, but the reality is a bit more complicated. There's a lot that goes on behind the scenes with opening and closing memory banks, refreshes, and general read and write latencies. Unoptimized prediction algorithms (as they are programmable) can result in losing _half_ of your performance.

sliken 3y ago

Pipelining helps, usually. You can have 10 stages run at 3 GHz instead of 1 stage at 300 MHz. Up until you have a branch misprediction and have to dump the pipeline. Memory channels are independent, however generally all cores use all channels. There are two common designs. Intel has 4 dies, 2 memory channels per core, so 2 channels are closer/lower latency than the other 6. AMD has multiple chiplets, but a single memory controller with 12 channels. So All cores have the same latency to all channels. Generally Intel has lower latencies to 25% of the channels, but AMD has more throughput (bandwidth or random IOPs). One thing that surprised me is that for maximum throughput you want at cache misses queued to the memory controller, at least twice the number of memory channels. These days missing in L1/L2/L3 is often approximately half the total memory latency. So on an Intel Xeon at least 16 misses (per socket), AMD at least 24 misses (per socket.). So on Intel you could tune things (and the NUMA support helps) to prefer the local channels. Most OSs help, and C calls like numa_alloc_local() allows local control. For memory intensive codes I have found the best scaling when there's 2 cores per channel. Of course most codes are pretty friendly.

znpy 3y ago
thank you for the explanation!