4 ms·

Mixed memory order as in, a memory write stores the data in an order like 5, 7, 6, 4 instead of 4, 5, 6, 7?

notacoward 6y ago
Basically yeah, but it gets much more complicated. For example, when a write hits memory is actually not the same thing as when it's observable by some particular other processor. There might be particular rules for writes that have dependencies too. Here's some good reading material if you have a spare afternoon. https://www.kernel.org/doc/Documentation/memory-barriers.txt https://www.kernel.org/doc/Documentation/memory-barriers.txt Amusingly, it treats dependency issues as obsolete, but I worked on a processor as late as 2009 that had all of the weak-memory-ordering issues of the Alpha (because it was the same design team). We uncovered lots of missing memory barriers in the Linux kernel. Good times.
- loeg 6y ago
  The modern thing is using acquire/release semantics on the variables you actually care about intra-thread ordering on, rather than full barriers, which impose a higher cost.
  - notacoward 6y ago
    I think we're talking about different kinds of barriers. Acquire/release at the C++ level translates roughly into an rmb/wmb pair at the processor level, unless it's implemented using an actual interlocked instruction which is worse than barriers.
    - loeg 6y ago
      Nope. Acquire/release at C level translates into a compiler barrier and the exact machine implementation will depend on hardware. However, on x86, for example, with its strong memory semantics, ordinary stores already have release semantics and do not require an architectural barrier (wmb/sfence). No sfence, no lock prefix.
      - notacoward 6y ago
        First you said that the modern way is to use acquire/release instead of a barrier, then admitted that it is a barrier at the compiler level. The fact that it doesn't require a barrier instruction on the x86 is pretty much irrelevant. Not all the world's an x86, and even the x86 hasn't always been the same in this regard. So your rude "nope" is unwarranted. This attitude of "as it is (in my world) now, so it has ever been and ever will be (everywhere)" is exactly why these bugs occur.
        loeg 6y ago
        Compiler barriers and machine barriers are completely different things, despite having a similar name. In about the era you described, the Linux kernel community did go around throwing rmb()s and wmb()s everywhere, and those did translate to hardware fences. (And in general, it is not safe to elide a hardware fence on x86 despite relatively strong memory ordering.) This is what I believed you were talking about; maybe I misunderstood: apologies. Putting explicit compiler barriers in code is still not quite the modern model for relaxed atomic consumers. You use the abstract acquire/release load/store pseudo-functions, and they do the right thing depending on implementation. The compiler barriers in the relaxed atomics model are an internal implementation detail, not the API. x86 is just an example where the cheapest way to do a release-semantics store is a plain store. I never said all the world was x86 — if it were, abstract acquire/release relaxed atomics would be kind of pointless.

Basically yeah, but it gets much more complicated. For example, when a write hits memory is actually not the same thing as when it's observable by some particular other processor. There might be particular rules for writes that have dependencies too. Here's some good reading material if you have a spare afternoon. https://www.kernel.org/doc/Documentation/memory-barriers.txt https://www.kernel.org/doc/Documentation/memory-barriers.txt Amusingly, it treats dependency issues as obsolete, but I worked on a processor as late as 2009 that had all of the weak-memory-ordering issues of the Alpha (because it was the same design team). We uncovered lots of missing memory barriers in the Linux kernel. Good times.

loeg 6y ago
The modern thing is using acquire/release semantics on the variables you actually care about intra-thread ordering on, rather than full barriers, which impose a higher cost.
- notacoward 6y ago
  I think we're talking about different kinds of barriers. Acquire/release at the C++ level translates roughly into an rmb/wmb pair at the processor level, unless it's implemented using an actual interlocked instruction which is worse than barriers.
  - loeg 6y ago
    Nope. Acquire/release at C level translates into a compiler barrier and the exact machine implementation will depend on hardware. However, on x86, for example, with its strong memory semantics, ordinary stores already have release semantics and do not require an architectural barrier (wmb/sfence). No sfence, no lock prefix.
    - notacoward 6y ago
      First you said that the modern way is to use acquire/release instead of a barrier, then admitted that it is a barrier at the compiler level. The fact that it doesn't require a barrier instruction on the x86 is pretty much irrelevant. Not all the world's an x86, and even the x86 hasn't always been the same in this regard. So your rude "nope" is unwarranted. This attitude of "as it is (in my world) now, so it has ever been and ever will be (everywhere)" is exactly why these bugs occur.
      - loeg 6y ago
        Compiler barriers and machine barriers are completely different things, despite having a similar name. In about the era you described, the Linux kernel community did go around throwing rmb()s and wmb()s everywhere, and those did translate to hardware fences. (And in general, it is not safe to elide a hardware fence on x86 despite relatively strong memory ordering.) This is what I believed you were talking about; maybe I misunderstood: apologies. Putting explicit compiler barriers in code is still not quite the modern model for relaxed atomic consumers. You use the abstract acquire/release load/store pseudo-functions, and they do the right thing depending on implementation. The compiler barriers in the relaxed atomics model are an internal implementation detail, not the API. x86 is just an example where the cheapest way to do a release-semantics store is a plain store. I never said all the world was x86 — if it were, abstract acquire/release relaxed atomics would be kind of pointless.