4 ms·

Here's the example: int i = load(p, memory_order_consume); if (i == 3) { int y = x[i]; } If you replace the memory_order_consume with a regular load

Here's the example: int i = load(p, memory_order_consume); if (i == 3) { int y = x[i]; } If you replace the memory_order_consume with a regular load, or a memory_order_relaxed, your code will be broken, even on x86 or ARM. The compiler can, and in this case likely will (yay constant propagation), rewrite the subsequent load in such a way that it is no longer data-dependent on the atomic load. It is not a compiler bug; if you try to report it as such, the compiler implementers will take one look at your problem report, and tell you that it is your code that is broken, use memory_order_consume. You will similarly get no sympathy from the C or C++ standards committees, for this is precisely why memory_order_consume was invented. > Mostly: I don't believe you, I think it's you that are failing to understand the issue and not me As someone who works on a compiler, and who sits on language standards committees, when I am explaining to you how someone who works on a compiler and who sits on language standards committees will read the standard, it's not because I fail to understand how to do so. I do realize that the way the standards are read and interpreted does not come naturally to most, which is why I gave that long explanation. Perhaps you do not wish to engage with the specification on this level of understanding. That is fine--but a consequence is that you will not be able to properly understand the reason for certain features, like memory_order_consume. At the very least, I hope you can agree to stop spreading misinformation like "the purpose of memory_order_consume is to support the Alpha processor."

kazinator 3y ago
I would hope that this paradigm requires all participating accesses to be annotated: int i = load(p, memory_order_consume); if (i == 3) { int y = load(x[i], memory_order_consume); } it seems like bad design to require ordering between plain loads and annotated loads. We want to identify the two locations that are important, so the compiler can reorder the others however it sees fit, without regard as to their placement w.r.t. those two.
- gpderetta 3y ago
  No, loads from x need not be consume nor even need to be atomic. If the thread that last wrote to x and then i never touches x again, there is no possible data race. This is for example the case of for initialization guards.
  - kazinator 3y ago
    OK, so p is here like a lock, and x[] is protected data. Indeed; we don't want to have to do anything special with the data protected by the lock. But accesses to the data must not cross the point where the lock is manipulated; e.g. if i == 3 means we own the lock, so that we may access x, x must not be accessed before that check by an optimization.
    - gpderetta 3y ago
      If p is like by a lock, the compiler won't reorder the access outside of the lock but it can still take into account that i is a known constant to simplify the address computation of x[i] into x[3]. This is perfectly safe for a mutex protected variable as the mutex acquire will, in addition to the compiler barrier, provide the required hardware barriers. This is is catastrophic, if, instead of relying on an hardware barrier, we rely on a load-load dependency chain originating from i; rewriting x[i] into x[3] break the chain and now, while the compiler might still not reorder the access, the hardware very much can.
      - kazinator 3y ago
        > the hardware very much can. If its name is DEC Alpha? ;)
        gpderetta 3y ago
        No, alpha will reorder even if you have a dependency chain (so it will need a barrier no matter what). Power and ARM will reorder only if the chain is broken.
gpderetta 3y ago
For another empirical example, the linux kernel requires marking all address-dependent accesses via rcu_dereference or LOAD_ONCE. In particular rcu_dereference has an entire section on all the restrictions on what you can do with the result to prevent the compiler from messing with it. For example only deal with pointers, avoid cancellations, etc. Only the things that have been empirically been shown to be safe are allowed. Of course "do what gcc x.y does" is not a good spec so the standard tried to specify exactly the behavior of consume is, but it failed to be implementable.
- ajross 3y ago
  > For another empirical example, the linux kernel requires marking all address-dependent accesses via rcu_dereference Right. Solely so it can support DEC Alpha! I'm not saying data dependence has never been an ordering constraint, I'm saying it was only an ordering constraint on an ancient platform that thus doesn't belong in the standard. And by extension I'm saying that you people talking about standards instead of hardware are needlessly (frankly: deliberately and obfuscatorily) obscuring that clear truth.
  - gpderetta 3y ago
    Absolutely not solely for DEC Alpha; From the source [1]: You must use one of the rcu_dereference() family of primitives to load an RCU-protected pointer, otherwise CONFIG_PROVE_RCU will complain. Worse yet, your code can see random memory-corruption bugs due to games that compilers and DEC Alpha can play. Without one of the rcu_dereference() primitives, compilers can reload the value, and won't your code have fun with two different values for a single pointer! Without rcu_dereference(), DEC Alpha can load a pointer, dereference that pointer, and return data preceding initialization that preceded the store of the pointer. The whole document then goes in great details on the all the ways the compiler can screw you over. If the kernel were to drop Alpha support tomorrow, rcu_dereference and all the warnings would still be relevant. Similarly, if only x86, SPARC and Alpha existed, memory order relaxed, acquire/release and seq-cst would be sufficient. consume only exists to cater to architectures with memory models stronger than Alpha but weaker than TSO. [1]https://www.kernel.org/doc/Documentation/RCU/rcu_dereference.txt https://www.kernel.org/doc/Documentation/RCU/rcu_dereference...
ajross 3y ago
That doesn't show anything incorrect though! You're just stating a requirement that the dereference to x[] happen after the load through p, which is something that can be trivially expressed using 1980's syntax by marking *p volatile. You don't need consume for that. Show me something that explains why the standard needs consume. It doesn't need consume, and I think you agree but you won't admit it. You keep saying that somehow this isn't about alpha, but it's clear this is about alpha, because alpha needs a barrier there and volatile can't do barriers. You just... won't admit it. And per my point WAY upthread, I contend you won't admit it because you view the "standard" as the important thing and not the hardware.
- reitzensteinm 3y ago
  They did, elsewhere: https://news.ycombinator.com/item?id=36068685 https://news.ycombinator.com/item?id=36068685 Even if volatile was absolutely all you needed, why would the committee build a memory model with a nice API and then force you to sprinkle in volatile? It makes absolutely no sense.
  - kazinator 3y ago
    Volatile is what you would use when you don't care about the DEC Alpha, and ignore the API. When using the API, you wouldn't require volatile. You can build yourself an access API if you have volatile. E.g. #define volatile_lvalue(TYPE, LOC) (*(volatile TYPE *) &(LOC)) With typeof or _Generic, we can eliminate the TYPE argument. Then you can do: if (volatile_lvalue(int, i) == 3) { y = x[i]; // don't care if i replaced with 3 here } The user of volatile_lvalue doesn't have to declare anything volatile.
    - gpderetta 3y ago
      That's the crux of the matter: if i is replaced with 3 on the second line, what do you think prevents an ARM CPU from hoisting the x[3] above the i load first line? In practice the volatile cast might prevent current GCCs from performing constant propagation, so this exact code sequence might work, but it is extremely fragile and can break for some variations: if (int i2 = volatile_lvalue(int, i); i2 == 3) { y = x[i2]; // i2 is replaced, no data dependency } or if (volatile_lvalue(int, i)) { y = x[i-i]; // data dependency becomes a control dependency } You also have to be careful to what you do with y and with any value derived form it. As currently there is no way to efficiently implement consume you have to do with what you have, so for example the Linux kernel does use volatile casts and compilation barriers to implement LOAD_ONCE, but then has a large manual that documents what operations are currently considered safe and what are known to break. That's hardly a specification that can be put in a standard and you can't really can reason about the semantics of it. Linus himself has considered multiple time if it is worth the hassle and whether the kernel should just switch to acquire everywhere now that arm and power have relevant cheap barriers.

kazinator 3y ago

I would hope that this paradigm requires all participating accesses to be annotated: int i = load(p, memory_order_consume); if (i == 3) { int y = load(x[i], memory_order_consume); } it seems like bad design to require ordering between plain loads and annotated loads. We want to identify the two locations that are important, so the compiler can reorder the others however it sees fit, without regard as to their placement w.r.t. those two.

gpderetta 3y ago
No, loads from x need not be consume nor even need to be atomic. If the thread that last wrote to x and then i never touches x again, there is no possible data race. This is for example the case of for initialization guards.
- kazinator 3y ago
  OK, so p is here like a lock, and x[] is protected data. Indeed; we don't want to have to do anything special with the data protected by the lock. But accesses to the data must not cross the point where the lock is manipulated; e.g. if i == 3 means we own the lock, so that we may access x, x must not be accessed before that check by an optimization.
  - gpderetta 3y ago
    If p is like by a lock, the compiler won't reorder the access outside of the lock but it can still take into account that i is a known constant to simplify the address computation of x[i] into x[3]. This is perfectly safe for a mutex protected variable as the mutex acquire will, in addition to the compiler barrier, provide the required hardware barriers. This is is catastrophic, if, instead of relying on an hardware barrier, we rely on a load-load dependency chain originating from i; rewriting x[i] into x[3] break the chain and now, while the compiler might still not reorder the access, the hardware very much can.
    - kazinator 3y ago
      > the hardware very much can. If its name is DEC Alpha? ;)
      - gpderetta 3y ago
        No, alpha will reorder even if you have a dependency chain (so it will need a barrier no matter what). Power and ARM will reorder only if the chain is broken.

gpderetta 3y ago

For another empirical example, the linux kernel requires marking all address-dependent accesses via rcu_dereference or LOAD_ONCE. In particular rcu_dereference has an entire section on all the restrictions on what you can do with the result to prevent the compiler from messing with it. For example only deal with pointers, avoid cancellations, etc. Only the things that have been empirically been shown to be safe are allowed. Of course "do what gcc x.y does" is not a good spec so the standard tried to specify exactly the behavior of consume is, but it failed to be implementable.

ajross 3y ago
> For another empirical example, the linux kernel requires marking all address-dependent accesses via rcu_dereference Right. Solely so it can support DEC Alpha! I'm not saying data dependence has never been an ordering constraint, I'm saying it was only an ordering constraint on an ancient platform that thus doesn't belong in the standard. And by extension I'm saying that you people talking about standards instead of hardware are needlessly (frankly: deliberately and obfuscatorily) obscuring that clear truth.
- gpderetta 3y ago
  Absolutely not solely for DEC Alpha; From the source [1]: You must use one of the rcu_dereference() family of primitives to load an RCU-protected pointer, otherwise CONFIG_PROVE_RCU will complain. Worse yet, your code can see random memory-corruption bugs due to games that compilers and DEC Alpha can play. Without one of the rcu_dereference() primitives, compilers can reload the value, and won't your code have fun with two different values for a single pointer! Without rcu_dereference(), DEC Alpha can load a pointer, dereference that pointer, and return data preceding initialization that preceded the store of the pointer. The whole document then goes in great details on the all the ways the compiler can screw you over. If the kernel were to drop Alpha support tomorrow, rcu_dereference and all the warnings would still be relevant. Similarly, if only x86, SPARC and Alpha existed, memory order relaxed, acquire/release and seq-cst would be sufficient. consume only exists to cater to architectures with memory models stronger than Alpha but weaker than TSO. [1]https://www.kernel.org/doc/Documentation/RCU/rcu_dereference.txt https://www.kernel.org/doc/Documentation/RCU/rcu_dereference...

ajross 3y ago

That doesn't show anything incorrect though! You're just stating a requirement that the dereference to x[] happen after the load through p, which is something that can be trivially expressed using 1980's syntax by marking *p volatile. You don't need consume for that. Show me something that explains why the standard needs consume. It doesn't need consume, and I think you agree but you won't admit it. You keep saying that somehow this isn't about alpha, but it's clear this is about alpha, because alpha needs a barrier there and volatile can't do barriers. You just... won't admit it. And per my point WAY upthread, I contend you won't admit it because you view the "standard" as the important thing and not the hardware.

reitzensteinm 3y ago
They did, elsewhere: https://news.ycombinator.com/item?id=36068685 https://news.ycombinator.com/item?id=36068685 Even if volatile was absolutely all you needed, why would the committee build a memory model with a nice API and then force you to sprinkle in volatile? It makes absolutely no sense.
- kazinator 3y ago
  Volatile is what you would use when you don't care about the DEC Alpha, and ignore the API. When using the API, you wouldn't require volatile. You can build yourself an access API if you have volatile. E.g. #define volatile_lvalue(TYPE, LOC) (*(volatile TYPE *) &(LOC)) With typeof or _Generic, we can eliminate the TYPE argument. Then you can do: if (volatile_lvalue(int, i) == 3) { y = x[i]; // don't care if i replaced with 3 here } The user of volatile_lvalue doesn't have to declare anything volatile.
  - gpderetta 3y ago
    That's the crux of the matter: if i is replaced with 3 on the second line, what do you think prevents an ARM CPU from hoisting the x[3] above the i load first line? In practice the volatile cast might prevent current GCCs from performing constant propagation, so this exact code sequence might work, but it is extremely fragile and can break for some variations: if (int i2 = volatile_lvalue(int, i); i2 == 3) { y = x[i2]; // i2 is replaced, no data dependency } or if (volatile_lvalue(int, i)) { y = x[i-i]; // data dependency becomes a control dependency } You also have to be careful to what you do with y and with any value derived form it. As currently there is no way to efficiently implement consume you have to do with what you have, so for example the Linux kernel does use volatile casts and compilation barriers to implement LOAD_ONCE, but then has a large manual that documents what operations are currently considered safe and what are known to break. That's hardly a specification that can be put in a standard and you can't really can reason about the semantics of it. Linus himself has considered multiple time if it is worth the hassle and whether the kernel should just switch to acquire everywhere now that arm and power have relevant cheap barriers.