3 ms·

This is a dumb question. But as you've said, cores per die has a strong influence on how performant a system is and that's because of the processor interconnec

This is a dumb question. But as you've said, cores per die has a strong influence on how performant a system is and that's because of the processor interconnects. My question is, what are the processors saying to each other?

Nabati 6y ago
"knock knock" "who's there?"
tolien 6y ago
In the case of Zen/Zen2, some of the dies didn't have direct access to main memory but went through an adjacent die's memory controller and channel. This had a performance and latency effect, and required some extra work on the part of OS schedulers to make sure that tasks weren't being scheduled on related cores. It's (relatively) easy to have lots of cores on one die, it's not so easy to keep them fed.
MarioMan 6y ago
Modern CPUs like these are shared memory multiprocessors. Programs on these CPUs are often designed to run on multiple cores at the same time to improve performance, necessitating communication between cores, to coordinate and share work between each other. Since each core typically has a private cache, most communication in a system like this will typically involve cache coherence protocols to ensure cores have a consistent view of memory contents. Adding more cores increases the overall complexity of the system. The latency of these protocols grows with more cores as interconnections become more "distant", across processor interconnects. Minimizing latency and effectively scaling to more cores is a difficult problem to solve while staying within silicon and thermal budgets.
jcranmer 6y ago
It's not a dumb question; it's actually the defining characteristic of performance for large systems. In a shared-memory multiprocessor, all cores and all memory are logically connected along a massive shared bus, and both inter-core communication and regular core-to-main-memory communication takes up slots on the bus. In practice, of course, it isn't physically a single bus but a more complex topology (usually meshes, I think) with protocols that make it like a bus for cache coherency purposes. One of the readily apparent consequences of interconnect is NUMA, non-uniform memory accesses. Historically (my information on this matter is a few years out of date, so I don't know how much this is true today), this has been a bigger issue for AMD than Intel, as AMD had wildly divergent memory latencies (2-3× between worst/best case) even for a 4-core processor. So if the OS decided to resume execution of a sleeping program on a different core, your performance might suddenly drop by ½ or ⅓. For some benchmarks (such as classic SPEC, or make -j), multicore performance is approximated by running the same binary on every core and there is no sharing between different copies. The interconnect here matters only insomuch as the cross-chatter on the interconnect may impede performance. But HPC applications tend to be much more sensitive to the actual interconnect, since reading and writing from other processors' memory is more common (transferring halo data, or having global data structures that you need to read from and occasionally write).
dragontamer 6y ago
Its not so much "processors" talking to each other as caches. The basic theory is called "MESI": Modified / Exclusive / Shared / Invalid. When a remote core reads a cache-line, if no one else has that cache-line then its the Exclusive owner. But if multiple caches have the cache-line, then it is a Shared cache line instead. If a remote cache changes, then your current cache-line becomes Invalid, while the cache line that's valid is placed into the Modified state (cache is correct, RAM is incorrect). MESI isn't how things are done in practice: additional messages and states are piled on top in proprietary ways. But MESI will get you to the "basic textbook" concept at least. EDIT: Oh, and "snooping". Caches "snoop" on each other's communications, which helps keep the MESI state up to date. Now you can have directory-based snooping (kind of a centralized push), or rings, or whatever. Those details are proprietary and change from chip to chip. In practice, these details are almost never published, so your best bet to understanding is "Something MESI-like is going on..."

Nabati 6y ago

"knock knock" "who's there?"

tolien 6y ago

In the case of Zen/Zen2, some of the dies didn't have direct access to main memory but went through an adjacent die's memory controller and channel. This had a performance and latency effect, and required some extra work on the part of OS schedulers to make sure that tasks weren't being scheduled on related cores. It's (relatively) easy to have lots of cores on one die, it's not so easy to keep them fed.

MarioMan 6y ago

Modern CPUs like these are shared memory multiprocessors. Programs on these CPUs are often designed to run on multiple cores at the same time to improve performance, necessitating communication between cores, to coordinate and share work between each other. Since each core typically has a private cache, most communication in a system like this will typically involve cache coherence protocols to ensure cores have a consistent view of memory contents. Adding more cores increases the overall complexity of the system. The latency of these protocols grows with more cores as interconnections become more "distant", across processor interconnects. Minimizing latency and effectively scaling to more cores is a difficult problem to solve while staying within silicon and thermal budgets.

jcranmer 6y ago

It's not a dumb question; it's actually the defining characteristic of performance for large systems. In a shared-memory multiprocessor, all cores and all memory are logically connected along a massive shared bus, and both inter-core communication and regular core-to-main-memory communication takes up slots on the bus. In practice, of course, it isn't physically a single bus but a more complex topology (usually meshes, I think) with protocols that make it like a bus for cache coherency purposes. One of the readily apparent consequences of interconnect is NUMA, non-uniform memory accesses. Historically (my information on this matter is a few years out of date, so I don't know how much this is true today), this has been a bigger issue for AMD than Intel, as AMD had wildly divergent memory latencies (2-3× between worst/best case) even for a 4-core processor. So if the OS decided to resume execution of a sleeping program on a different core, your performance might suddenly drop by ½ or ⅓. For some benchmarks (such as classic SPEC, or make -j), multicore performance is approximated by running the same binary on every core and there is no sharing between different copies. The interconnect here matters only insomuch as the cross-chatter on the interconnect may impede performance. But HPC applications tend to be much more sensitive to the actual interconnect, since reading and writing from other processors' memory is more common (transferring halo data, or having global data structures that you need to read from and occasionally write).

dragontamer 6y ago

Its not so much "processors" talking to each other as caches. The basic theory is called "MESI": Modified / Exclusive / Shared / Invalid. When a remote core reads a cache-line, if no one else has that cache-line then its the Exclusive owner. But if multiple caches have the cache-line, then it is a Shared cache line instead. If a remote cache changes, then your current cache-line becomes Invalid, while the cache line that's valid is placed into the Modified state (cache is correct, RAM is incorrect). MESI isn't how things are done in practice: additional messages and states are piled on top in proprietary ways. But MESI will get you to the "basic textbook" concept at least. EDIT: Oh, and "snooping". Caches "snoop" on each other's communications, which helps keep the MESI state up to date. Now you can have directory-based snooping (kind of a centralized push), or rings, or whatever. Those details are proprietary and change from chip to chip. In practice, these details are almost never published, so your best bet to understanding is "Something MESI-like is going on..."