3 ms·
Okay, I've given it some thought. Here's my theory: ARM's load/load, load/store, store/load, and store/store reordering is more flexible than Intel's store/loa
by dragontamer 8y ago
Okay, I've given it some thought. Here's my theory:
ARM's load/load, load/store, store/load, and store/store reordering is more flexible than Intel's store/load (only) reordering. As such, ARM is more likely to benefit from deeper load-buffering than Intel.
Remember that all of these systems are multi-core, which means we have to consider what these load-buffers do during a multi-core situation. Here's some example code for "Thread 1"
Thread 1:
register RAX=0; // Non-memory. No load/store associated
int a=0, b=0, c=0; // "True Memory", accessing these have a load/store
while(1){
RAX++;
memory barrier();
a = RAX;
RAX++;
memory barrier();
b = RAX;
RAX++;
memory barrier();
c = RAX;
}
In effect, this thread would create the memory commands:
memory barrier
store [A], 0
memory barrier
store [B], 1
memory barrier
store [C], 2
memory barrier
store [A], 3
memory barrier
store [B], 4
memory barrier
store [C], 5
(etc. etc)
For simplicity, I've added memory barriers to ensure that these writes happen in EXACTLY this order. No need to worry about ARM's store/store reordering in this example. (x86 does NOT need memory barriers, because store/store will always happen in order)
Now lets imagine a 2nd core reading from these variables. Lets say Core#2 has the following load buffer:
load rax, [A]
load rbx, [B]
load rcx, [C]
# Note: no memory barriers!
With this ordering, the Intel processor is guaranteed to have "RBX" a later value than RAX, and RCX to have the latest value.
This is NOT true in ARM. In the ARM processor, the following assignments are possible:
RAX = 3 # load RAX MAY execute last on ARM.
RBX = 1 # This example will NEVER happen on x86
RCX = 2
Why? Because in ARM, the load/load boundary can be crossed by the core. That is, the "load RBX, [B]" instruction MAY be executed first on ARM's architecture, despite coming in afterwards.
This means that load buffers are far, far cheaper on ARM. But they also come at a cost to the programmer's sanity. If you actually have important "ordering", you need to be sure to protect your code with memory barriers.
x86 has an implicit memory barrier on all stores and loads. This is slower and more costly. In effect, the load-buffer on x86 MUST be a proper queue: the first loads must execute first, while the last loads must execute last.
In ARM, this isn't the case. A load-buffer of size 40 can all execute simultaneously and out of order. Much to the confusion to multi-threaded programmers, I'm sure. This example also demonstrates why on weakly ordered systems, you need a memory-barrier on BOTH threads. (Fixing the store/store reordering with memory barriers only solves half the problem)
With that being said: modern programmers know that the only variables that need a defined ordering are synchronization structures, like spinlocks. Every spinlock needs a "release" barrier afterwards (Explicit barrier states: do NOT reorder anything past this store). Every spinlock needs an "acquire" barrier before reading (Command: do NOT reorder anything before this load).
Semaphores / Atomics / other synchronization structures also need a defined ordering. Accessing "normal" variables is well known to be "unordered" by modern programmers. So ARM has the advantage with regards to modern languages. x86's architecture is more strict than necessary.
- BeeOnRope 8y agoYes, this is a reasonable explanation of the ARM and Intel memory models, but I'm not following how this relates to miss buffers specifically. The load buffers which maintain the source/program ordering on a per-instruction basis as you describe above are called ... load buffers. These are different than the "miss buffers" which track outstanding misses. Intel chips have 70+ load buffers, so no there is no shortage there. Yes, x86 chips have to jump through a bunch of to keep ordering, which means their load buffers probably need to more sophisticated (Intel sometimes refers to a MOB - a "memory order buffer" which tracks this stuff, but as far as I can tell this is not a separate buffer but just another name for the load and store buffers and associated machinery, with a focus on the memory ordering aspect). The way it works in Intel is that everything will speculatively execute out of order, loads passing loads and all the things the memory model doesn't officially allow, but the load buffer will track the original order. As long as there is not indication of an actual problem with the ordering (i.e., a store on another core which causes a snoop that hits the load buffer on this core), the out-of-order nature of the execution isn't visible, so everything can retire fine. If something happens that violates the order, a "MOB machine clear" happens (with a cost similar to a branch misprediction) occurs and the chip starts over. None of this directly implicates the miss buffers though. For one thing, they are not 1:1 with instructions and they are not instruction ordered, since multiple misses will coalesce to the same buffer. That said, I can certainly see the stronger x86 memory model playing a role here, but it isn't clear to me how.