4 ms·

Hmm, I think the discussion is missing a few things needed for a complete picture of the situation. First, ARM and x86 coherency models differ, so a big discla

Hmm, I think the discussion is missing a few things needed for a complete picture of the situation. First, ARM and x86 coherency models differ, so a big disclaimer is needed regarding the protocol. Most ARM processors use the MOESI protocol instead of the MESI protocol. Second, synchronization isn't just because of register volatility and such. Synchronization is needed in general because without the appropriate lock/barrier instructions, compilers make assumptions about how loads and stores may be reordered with respect to one another.

chrismarlow9 3y ago
Any recommendations for something more thorough to read or watch?
- ninepoints 3y ago
  Honestly, the best resources I know are the ISA/architecture manuals directly from each vendor (Intel, AMD, Arm) for the various CPUs and/or GPUs of interest. Prior to that, my experience largely comes from doing (working with lower level code, benchmarks, looking at assembly, profiling hardware counters, etc.).
monocasa 3y ago
> First, ARM and x86 coherency models differ, so a big disclaimer is needed regarding the protocol. Most ARM processors use the MOESI protocol instead of the MESI protocol. MOESI is called out. > The above are just some of the possible scenarios that can occur. In reality, there are numerous variations of the above design, and no 2 implementations are the same. For example, some designs have an O/F state. However that's not an ARM v x86 thing; AMD has at least previously used MOESI. > Second, synchronization isn't just because of register volatility and such. Synchronization is needed in general because without the appropriate lock/barrier instructions, compilers make assumptions about how loads and stores may be reordered with respect to one another. Compiler barriers are different than hardware memory barriers. There are reasons to have one, but not the other.
- ComputerGuru 3y ago
  I didn't immediately find a first-party source but according to these COMP522 lecture notes from Rice, AMD still uses MOESI for Zen [0]. [0]: https://www.cs.rice.edu/~johnmc/comp522/lecture-notes/COMP522-2019-Lecture5-Cache-Coherence-I.pdf https://www.cs.rice.edu/~johnmc/comp522/lecture-notes/COMP52...
  - monocasa 3y ago
    Awesome, more than a source than I found. I knew previous designs were MOESI, but a quick search didn't give any results wrt Zen. Thanks for the citation!
- ninepoints 3y ago
  Thanks I missed that line regarding the O states. Still, a word about write reordering on ARM would probably be useful (unless I missed that also). I understand that synchronization in code vs hardware is different, but the blog explicitly moves out of hardware-land into source code land with references to Java volatile and such.
  - gpderetta 3y ago
    The blog mentions about java volatiles (but it would also apply to C++ atomic) to explicitly mention that volatile has no cache coherency implications on a typical MESI (and variants) machine. The fences required to maintain language level memory model guarantees act at a level above the L1 cache, once the data reaches L1 (i.e. the coherence point), the fences have done their job. [I'm ignoring remote fences which are a specialized and not yet mainstream feature]

chrismarlow9 3y ago

Any recommendations for something more thorough to read or watch?

ninepoints 3y ago
Honestly, the best resources I know are the ISA/architecture manuals directly from each vendor (Intel, AMD, Arm) for the various CPUs and/or GPUs of interest. Prior to that, my experience largely comes from doing (working with lower level code, benchmarks, looking at assembly, profiling hardware counters, etc.).

monocasa 3y ago

> First, ARM and x86 coherency models differ, so a big disclaimer is needed regarding the protocol. Most ARM processors use the MOESI protocol instead of the MESI protocol. MOESI is called out. > The above are just some of the possible scenarios that can occur. In reality, there are numerous variations of the above design, and no 2 implementations are the same. For example, some designs have an O/F state. However that's not an ARM v x86 thing; AMD has at least previously used MOESI. > Second, synchronization isn't just because of register volatility and such. Synchronization is needed in general because without the appropriate lock/barrier instructions, compilers make assumptions about how loads and stores may be reordered with respect to one another. Compiler barriers are different than hardware memory barriers. There are reasons to have one, but not the other.

ComputerGuru 3y ago
I didn't immediately find a first-party source but according to these COMP522 lecture notes from Rice, AMD still uses MOESI for Zen [0]. [0]: https://www.cs.rice.edu/~johnmc/comp522/lecture-notes/COMP522-2019-Lecture5-Cache-Coherence-I.pdf https://www.cs.rice.edu/~johnmc/comp522/lecture-notes/COMP52...
- monocasa 3y ago
  Awesome, more than a source than I found. I knew previous designs were MOESI, but a quick search didn't give any results wrt Zen. Thanks for the citation!
ninepoints 3y ago
Thanks I missed that line regarding the O states. Still, a word about write reordering on ARM would probably be useful (unless I missed that also). I understand that synchronization in code vs hardware is different, but the blog explicitly moves out of hardware-land into source code land with references to Java volatile and such.
- gpderetta 3y ago
  The blog mentions about java volatiles (but it would also apply to C++ atomic) to explicitly mention that volatile has no cache coherency implications on a typical MESI (and variants) machine. The fences required to maintain language level memory model guarantees act at a level above the L1 cache, once the data reaches L1 (i.e. the coherence point), the fences have done their job. [I'm ignoring remote fences which are a specialized and not yet mainstream feature]