7 ms·

What's up with this new memory_order_consume memory order?

67 points by luu 3y ago

tialaramex 3y ago
This seems like a rare mis-fire by Raymond Chen. Consume isn't new in C++ 20. It was part of the C++ 11 Memory Model, but in practice it didn't turn out to be viable, and the immediate cause for attempting this work died out - the DEC Alpha seems like it should have some sort of cheap fast atomics, but you can't cheaply do Acquire-Release so how about we invent something else ? Well if nobody cares about DEC Alpha then this ceases to be a priority. For a while the ISO documents claimed Consume was "Temporarily discouraged" and I believe now they just admit you'll get Acquire instead. Because Rust follows the C++ 11 Model (as revised) for lack of anything better, it deliberately does not mention Consume ordering either. https://doc.rust-lang.org/core/sync/atomic/enum.Ordering.html https://doc.rust-lang.org/core/sync/atomic/enum.Ordering.htm...
- saagarjha 3y ago
  Yeah, compilers were never really into it anyways. Most of them just lowered it to an acquire so there wasn't any point in using it.
- brickteacup 3y ago
  > the DEC Alpha seems like it should have some sort of cheap fast atomics, but you can't cheaply do Acquire-Release so how about we invent something else ? consume was not invented for Alpha. rather consume should have less overhead than acquire on any platform that needs additional memory barriers to implement acquire semantics (on most platforms consume requires no barriers at all since memory ordering is enforced as a side-effect of dependency ordering. Alpha is an exception to this, i.e. consume does require a barrier on Alpha). it's also worth noting that rcu_dereference in linux is based on the same idea as consume (using dependencies to enforce memory ordering) the the basic idea is definitely useful. it's more that the C++ standard's specification of consume turned out to be unimplementable in practice
  - ajross 3y ago
    > consume was not invented for Alpha. rather consume should have less overhead than acquire on any platform that [emphasis mine] needs additional memory barriers to implement acquire semantics Um... which platform?[1] Probably the biggest reason that memory ordering semantics are such a mess, and lockless algorithms such an infeasible disaster, is the propensity of people to argue about this stuff in generic terms, using language and concepts from egghead standards writers instead of examples of real machines. Out of order execution is an engineering technique for physical devices. We should talk about the machines first, and the abstractions later. It's like going to the mechanic with a leaky valve and having the explanation come back in terms of cycle efficiencies and isentropic losses. It's not actually helpful to understanding the problem unless you already understand the theory. And you won't ever learn the theory unless you see the problem it's solving. [1] It's a serious question, btw. I genuinely don't know what the target hardware is for "consume" myself, and suspect this is just a pet theory that snuck into the standard.
    - reitzensteinm 3y ago
      Are memory ordering semantics a mess? I implemented simulators for x86, ARM and C++ memory models and the documentation for all three was fantastic. What's part of it is a mess? I'm also not sure I understand your point about physical machines. Surely what we care about are the guarantees each platform gives. There are plenty of cases where the documented worst case is worse than you'll ever be able to replicate - for now. Sticking to the contract means you're safe on next year's machines, too.
      - ajross 3y ago
        To be glib, the part that answers the question "Who needs consume and why is it there?". As pointed out, Chen answers this incorrectly in the linked article. An upthread poster pointed out (correctly, though they skipped the justification) that Alpha needs this, but was then corrected (incorrectly, AFAICT, though I'm willing to be educated) that this feature was not for Alpha. To be more serious: anyone who's ever tried to implement a lockless algorithm knows how hard this stuff is even if you have a clear and unambiguous set of tools to use to do it. And the C++ memory model is, as mentioned, kinda ambiguous. At the end of the day, in practice these problems are solved on real hardware with real tools that look like "MFENCE" or "DSB" (either directly, or because you're reading the generated assembly trying to figure out how it's going wrong). That's exactly the opposite of the way a good abstraction is supposed to work.
- jlokier 3y ago
  If Consume is discouraged, what should you use instead for the read side of RCU style algorithms? Acquire is too heavy, and Relaxed is too relaxed, even if it often works on non-Alpha CPUs in practice, it assumes the compiler doesn't get too clever with speculative prefetch and reuse optimisations. Do we really still need to use Linux's compiler-specific `rcu_dereference` in performant code just because Consume isn't implemented?
  - bonzini 3y ago
    You can use relaxed followed by an acquire compiler fence. Not great but it works.
    - gpderetta 3y ago
      C++ even provides a portable compiler fence in the form atomic_signal_fence.
- j16sdiz 3y ago
  According to a comment in the blog post, the sementic had changed in C++20: > If your compiler predates C++20, however, the consume memory order is almost certainly treated as acquire (legal, just not faster). C++17 temporarily discouraged the use of the consume order because the semantics were impossible to use correctly, and thus all compilers were implementing it as acquire. > > The revisions in C++20 took some very smart people a lot of effort, and changed the definitions subtly so that you can use consume and get weaker memory ordering than acquire without immediately invoking our good friend UB.
  - tialaramex 3y ago
    That is what the comment says, but, it doesn't seem to match reality. "To verify comments I saw on Raymond Chen's blog" isn't enough justification for me to pay ISO's fees for the actual ISO C++ 20 standard document, but the draft the committee members work from still says of Consume: > Implementations have found it infeasible to provide performance better than that of memory_order :: acquire. I suspect that the comment is well-meant but is referring to some proposal that may have been made at some point in the last say five years, and which isn't in the ISO standard. Plenty of people have proposed that Consume could be fixed, but there are big gaps between proposing a fix, agreeing it among all interested parties and writing that fix into the standard, and actually shipping compilers with working Consume memory order.
jbandela1 3y ago
memory_order_consume has been around since C++11, but discouraged. If you want a good overview of what it is supposed to do, https://preshing.com/20140709/the-purpose-of-memory_order_consume-in-cpp11/ https://preshing.com/20140709/the-purpose-of-memory_order_co... goes into more detail with lots of examples and motivation.
bno1 3y ago
> Note that an acquire load of p would have prohibited this reordering, since acquire loads block all future memory access, even if unrelated to the value being acquired. I don't really get this part. The docs for acquire say: > memory_order_acquire A load operation with this memory order performs the acquire operation on the affected memory location: no reads or writes in the current thread can be reordered before this load. All writes in other threads that release the same atomic variable are visible in the current thread The prefetch is already ordered before the load in the same thread.
- brickteacup 3y ago
  `sample_consume_allowed` is an example of a transformation that the compiler is permitted to apply to the original function `sample_consume`. if the original function had used `acquire` instead of `consume` then the reordering of the v2 load before the p load wouldn't've been allowed
  - bno1 3y ago
    Thank you, makes sense now
ot 3y ago
> The consume memory order is not used much. That's because no compiler implements it, it just becomes an acquire in practice. Nobody has figured out how to maintain all the necessary invariants across all optimization passes.
EscapeFromNY 3y ago
So consume was added as a more surgical version of acquire. I see why that's useful. Why didn't they also add a surgical version of release? Wouldn't that also be useful in the same situations where consume is useful?
- gpderetta 3y ago
  consume was implemented because real architectures provide data dependency on the load path (i.e. a load of a pointer value is never reordered past the a load of the pointed value). No architecture provides such a guarantee on the store path, unless , like TSO, provides it for all stores, so no point in specifying it in the standard.
  - EscapeFromNY 3y ago
    Thanks, that makes sense
dragontamer 3y ago
I don't remember who said it. But some C++ Guru on some online talk (Herb Sutter, Pikus, etc etc.) was saying that 'consume' was pushed by ARM and PowerPC chipmakers as sufficient for C++11. And these chipmakers were somewhat biased by how POWER7 and ARMv7 worked. Alas, it seems like consume was too complicated for compiler writers to take full advantage of. So in practice, acquire is used instead. And then in ARMv8 and POWER9, acquire was added to those instruction sets. Acquire isn't the default, but we do have accelerated assembly language support for the acquire model. So moving forward, acquire seems to be the sweetspot model. I guess consume exists for a hypothetical future C++ compiler that tries to optimize multithreaded ARMv7 or something, a relatively small niche in practice as current C++ compilers don't support consume and future processors have moved on to the acquire model.
- gpderetta 3y ago
  Yes, I believe that consume will be consigned to the annals of history. Acq/rel is such a sweet spot that in retrospect it should have been the default (for load and stores respectively) instead of seq_cst.
  - tialaramex 3y ago
    All the (C++) defaults are wrong, of course. But in this case I believe the choice to have a default was itself the error. I have been persuaded that either you know which Ordering you need, and thus you should specify what you meant - or else you should not be using these APIs at all. If you aren't confident of the correct Ordering, you are not in fact confident your algorithm is correct, so don't do it.
    - gpderetta 3y ago
      You are not wrong; still an std::atomic variant that defaulted to mo_release on assignment, mo_acquire on load and mo_aq_rel on RMW would have covered 99% of my use cases. Then again, I usually target x86 where these defaults are optimal, in other architectures relaxing RMWs further has benefits.
  - 3y ago

67 points by luu 3y ago

This seems like a rare mis-fire by Raymond Chen. Consume isn't new in C++ 20. It was part of the C++ 11 Memory Model, but in practice it didn't turn out to be viable, and the immediate cause for attempting this work died out - the DEC Alpha seems like it should have some sort of cheap fast atomics, but you can't cheaply do Acquire-Release so how about we invent something else ? Well if nobody cares about DEC Alpha then this ceases to be a priority. For a while the ISO documents claimed Consume was "Temporarily discouraged" and I believe now they just admit you'll get Acquire instead. Because Rust follows the C++ 11 Model (as revised) for lack of anything better, it deliberately does not mention Consume ordering either. https://doc.rust-lang.org/core/sync/atomic/enum.Ordering.html https://doc.rust-lang.org/core/sync/atomic/enum.Ordering.htm...

saagarjha 3y ago
Yeah, compilers were never really into it anyways. Most of them just lowered it to an acquire so there wasn't any point in using it.
brickteacup 3y ago
> the DEC Alpha seems like it should have some sort of cheap fast atomics, but you can't cheaply do Acquire-Release so how about we invent something else ? consume was not invented for Alpha. rather consume should have less overhead than acquire on any platform that needs additional memory barriers to implement acquire semantics (on most platforms consume requires no barriers at all since memory ordering is enforced as a side-effect of dependency ordering. Alpha is an exception to this, i.e. consume does require a barrier on Alpha). it's also worth noting that rcu_dereference in linux is based on the same idea as consume (using dependencies to enforce memory ordering) the the basic idea is definitely useful. it's more that the C++ standard's specification of consume turned out to be unimplementable in practice
- ajross 3y ago
  > consume was not invented for Alpha. rather consume should have less overhead than acquire on any platform that [emphasis mine] needs additional memory barriers to implement acquire semantics Um... which platform?[1] Probably the biggest reason that memory ordering semantics are such a mess, and lockless algorithms such an infeasible disaster, is the propensity of people to argue about this stuff in generic terms, using language and concepts from egghead standards writers instead of examples of real machines. Out of order execution is an engineering technique for physical devices. We should talk about the machines first, and the abstractions later. It's like going to the mechanic with a leaky valve and having the explanation come back in terms of cycle efficiencies and isentropic losses. It's not actually helpful to understanding the problem unless you already understand the theory. And you won't ever learn the theory unless you see the problem it's solving. [1] It's a serious question, btw. I genuinely don't know what the target hardware is for "consume" myself, and suspect this is just a pet theory that snuck into the standard.
  - reitzensteinm 3y ago
    Are memory ordering semantics a mess? I implemented simulators for x86, ARM and C++ memory models and the documentation for all three was fantastic. What's part of it is a mess? I'm also not sure I understand your point about physical machines. Surely what we care about are the guarantees each platform gives. There are plenty of cases where the documented worst case is worse than you'll ever be able to replicate - for now. Sticking to the contract means you're safe on next year's machines, too.
    - ajross 3y ago
      To be glib, the part that answers the question "Who needs consume and why is it there?". As pointed out, Chen answers this incorrectly in the linked article. An upthread poster pointed out (correctly, though they skipped the justification) that Alpha needs this, but was then corrected (incorrectly, AFAICT, though I'm willing to be educated) that this feature was not for Alpha. To be more serious: anyone who's ever tried to implement a lockless algorithm knows how hard this stuff is even if you have a clear and unambiguous set of tools to use to do it. And the C++ memory model is, as mentioned, kinda ambiguous. At the end of the day, in practice these problems are solved on real hardware with real tools that look like "MFENCE" or "DSB" (either directly, or because you're reading the generated assembly trying to figure out how it's going wrong). That's exactly the opposite of the way a good abstraction is supposed to work.
jlokier 3y ago
If Consume is discouraged, what should you use instead for the read side of RCU style algorithms? Acquire is too heavy, and Relaxed is too relaxed, even if it often works on non-Alpha CPUs in practice, it assumes the compiler doesn't get too clever with speculative prefetch and reuse optimisations. Do we really still need to use Linux's compiler-specific `rcu_dereference` in performant code just because Consume isn't implemented?
- bonzini 3y ago
  You can use relaxed followed by an acquire compiler fence. Not great but it works.
  - gpderetta 3y ago
    C++ even provides a portable compiler fence in the form atomic_signal_fence.
j16sdiz 3y ago
According to a comment in the blog post, the sementic had changed in C++20: > If your compiler predates C++20, however, the consume memory order is almost certainly treated as acquire (legal, just not faster). C++17 temporarily discouraged the use of the consume order because the semantics were impossible to use correctly, and thus all compilers were implementing it as acquire. > > The revisions in C++20 took some very smart people a lot of effort, and changed the definitions subtly so that you can use consume and get weaker memory ordering than acquire without immediately invoking our good friend UB.
- tialaramex 3y ago
  That is what the comment says, but, it doesn't seem to match reality. "To verify comments I saw on Raymond Chen's blog" isn't enough justification for me to pay ISO's fees for the actual ISO C++ 20 standard document, but the draft the committee members work from still says of Consume: > Implementations have found it infeasible to provide performance better than that of memory_order :: acquire. I suspect that the comment is well-meant but is referring to some proposal that may have been made at some point in the last say five years, and which isn't in the ISO standard. Plenty of people have proposed that Consume could be fixed, but there are big gaps between proposing a fix, agreeing it among all interested parties and writing that fix into the standard, and actually shipping compilers with working Consume memory order.

jbandela1 3y ago

memory_order_consume has been around since C++11, but discouraged. If you want a good overview of what it is supposed to do, https://preshing.com/20140709/the-purpose-of-memory_order_consume-in-cpp11/ https://preshing.com/20140709/the-purpose-of-memory_order_co... goes into more detail with lots of examples and motivation.

bno1 3y ago

> Note that an acquire load of p would have prohibited this reordering, since acquire loads block all future memory access, even if unrelated to the value being acquired. I don't really get this part. The docs for acquire say: > memory_order_acquire A load operation with this memory order performs the acquire operation on the affected memory location: no reads or writes in the current thread can be reordered before this load. All writes in other threads that release the same atomic variable are visible in the current thread The prefetch is already ordered before the load in the same thread.

brickteacup 3y ago
`sample_consume_allowed` is an example of a transformation that the compiler is permitted to apply to the original function `sample_consume`. if the original function had used `acquire` instead of `consume` then the reordering of the v2 load before the p load wouldn't've been allowed
- bno1 3y ago
  Thank you, makes sense now

ot 3y ago

> The consume memory order is not used much. That's because no compiler implements it, it just becomes an acquire in practice. Nobody has figured out how to maintain all the necessary invariants across all optimization passes.

EscapeFromNY 3y ago

So consume was added as a more surgical version of acquire. I see why that's useful. Why didn't they also add a surgical version of release? Wouldn't that also be useful in the same situations where consume is useful?

gpderetta 3y ago
consume was implemented because real architectures provide data dependency on the load path (i.e. a load of a pointer value is never reordered past the a load of the pointed value). No architecture provides such a guarantee on the store path, unless , like TSO, provides it for all stores, so no point in specifying it in the standard.
- EscapeFromNY 3y ago
  Thanks, that makes sense

dragontamer 3y ago

I don't remember who said it. But some C++ Guru on some online talk (Herb Sutter, Pikus, etc etc.) was saying that 'consume' was pushed by ARM and PowerPC chipmakers as sufficient for C++11. And these chipmakers were somewhat biased by how POWER7 and ARMv7 worked. Alas, it seems like consume was too complicated for compiler writers to take full advantage of. So in practice, acquire is used instead. And then in ARMv8 and POWER9, acquire was added to those instruction sets. Acquire isn't the default, but we do have accelerated assembly language support for the acquire model. So moving forward, acquire seems to be the sweetspot model. I guess consume exists for a hypothetical future C++ compiler that tries to optimize multithreaded ARMv7 or something, a relatively small niche in practice as current C++ compilers don't support consume and future processors have moved on to the acquire model.

gpderetta 3y ago
Yes, I believe that consume will be consigned to the annals of history. Acq/rel is such a sweet spot that in retrospect it should have been the default (for load and stores respectively) instead of seq_cst.
- tialaramex 3y ago
  All the (C++) defaults are wrong, of course. But in this case I believe the choice to have a default was itself the error. I have been persuaded that either you know which Ordering you need, and thus you should specify what you meant - or else you should not be using these APIs at all. If you aren't confident of the correct Ordering, you are not in fact confident your algorithm is correct, so don't do it.
  - gpderetta 3y ago
    You are not wrong; still an std::atomic variant that defaulted to mo_release on assignment, mo_acquire on load and mo_aq_rel on RMW would have covered 99% of my use cases. Then again, I usually target x86 where these defaults are optimal, in other architectures relaxing RMWs further has benefits.
- 3y ago