3 ms·
> Why can't I have fast, low instruction cost communication between any two CPUs, regardless of which core it's on? In other words, why do I have to do that boo
by dragontamer 7y ago
> Why can't I have fast, low instruction cost communication between any two CPUs, regardless of which core it's on? In other words, why do I have to do that bookkeeping to have super fast communication between threads/actors?
L1 cache is literally inside of the core of modern chips. You can't beat its speed because its physically the closest speed possible.
Moving cache further away (ex: L2 or L3) means more wires, more capacitance, and slower electricity. Electricity doesn't move instantly: it has to fight the capacitance of wires. The wider the wires, the more electrons you need to put onto the wire before its voltage changes. There's also inductance: the longer the wire is, the longer you need to hold a voltage before electrons actually decide to move.
Second: L1 caches do a LOT of book-keeping for you. All cores on modern systems (anything more modern than DEC Alpha from the 90s) will have cache coherence automatically.
That's downright huge, and modern multithreaded programming would be damn near impossible without the ordering guarantees of basic cache coherence. (yes: even ARM / MIPS and their "relaxed memory model" are more coherent than chips from earlier eras).
So first of all, be gracious about the level of memory ordering available to the modern programmer.
------------
So how are caches coherent? Most modern caches use some form of MESI to communicate with each other (invisibly: this is "below" assembly language and is happening at the hardware level).
When one cache claims "exclusive access" to the data, no other cache will write to it (although they may read stale versions of that data from their own cache). When that core / L1 cache is done, it will "invalidate" the data.
Modern cache coherence protocols can go faster: MESIF for Intel means that writes may be (F)orwarded to other caches automatically (which is slightly faster than (E)xclusive -> (I)nvalid -> (E)xclusive in the 2nd core).
Again, this is all done automatically, because programmers want to have their data look the same from all cores (for some definition of "looks the same").
This does NOT mean that the data is in the correct order however (!!). L1 caches can change the order of reads and writes before the data is shipped to other cores. Indeed, this is the very POINT of L1 cache. If one core is doing a while(true) iter++; loop, its more efficient to merge the writes together and "batch" them out slowly. Ex: Core#2 may see "1, 10, 33, 600" each time it reads the value of "iter".
Indeed, the most efficient implementation of this while(true) iter++; loop is to have "iter" be a register private to a core that has absolutely no communication to the outside world at all.
- tempguy9999 7y ago'Scusi, I said "they [each core] share the L1/L2 cache". Badly put but I thought on x86 each core had strictly private L1/L2, and L3 was shared across the socket. Am I wrong in thinking L2 is private to each core?
- dragontamer 7y agoI think L1 / L2 is private per core, but I'm not 100% sure. I "retreated" by stating L1 cache, because I'm 100% sure that l1 cache is private per core. I know L3 is external to the core, and L1 is private. L2 is that iffy spot in my memory that I'd have to read a few manuals to remember exactly where it is... lol.
- haneefmubarak 7y agoNot the parent, but yes on modern x86 L1/L2 is private. The key difference though is that the L1 is smaller and actually a physical part of the core, whereas the L2 is bigger and sits next to the core (read: further away from the execution engine).
- stcredzero 7y agoL1 caches do a LOT of book-keeping for you Again, this is all done automatically, because programmers want to have their data look the same from all cores (for some definition of "looks the same"). In terms of bookkeeping examples, we have to make sure we're not inadvertently causing False Sharing. https://en.wikipedia.org/wiki/False_sharing https://en.wikipedia.org/wiki/False_sharing This sort of thing is quite implicit as opposed to being explicit. This is because communicating with a caching+coherence mechanism is a bit of a kludge. When one cache claims "exclusive access" to the data, no other cache will write to it (although they may read stale versions of that data from their own cache). Get your head out of CPU caches and low level stuff for a moment. Imagine that someone proposes a high level communications protocol to you which acts like this. That would be ugly. That's the sort of mechanism you'd expect to see at a very low level of the networking stack. That's precisely the sort of thing a programmer wants abstracted away from them, so they can think of communication as communication, not as a series of details about the behavior of hardware.