3 ms·

OOO is definitely observable when dealing with multithreading. That's why you have things like memory order constraints on atomic operations in C, and why you

OOO is definitely observable when dealing with multithreading. That's why you have things like memory order constraints on atomic operations in C, and why you need to be extremely careful with how you place things like memory fences when doing anything fine-grained between several threads.

titzer 4y ago
Fair point, but I'd qualify that OOO is only observable for racy programs. In languages with shared memory concurrency, but safety, races are either statically disallowed (Rust), or their results are constrained by a (weak) memory model. Java, e.g. only makes rough guarantees that you don't see out-of-thin air values and that you can't violate the type system. Java also is a little stricter around volatile memory access (they are sequentially consistent, iirc). Also worth noting, some reorderings that you see are not due to hardware, but due to compiler optimizations. Compilers have to obey the memory model, too. Generally, most programmers shouldn't need to be reasoning about OOO or memory order and be using fences. Rather, they should use libraries that are written by experts that use extra-lingual mechanisms (like compare-swap or fences) to guarantee a higher contract.
- zozbot234 4y ago
  Rust has atomic types that can observe races. Such "racy" code is useful because some concurrent code doesn't need the strictest guarantees (sequential consistency). Though, compared to C, Rust does not use the "Consume" memory ordering because so far it has turned out to be ill-defined.
  - titzer 4y ago
    AFAICT those can't observe OOO of CPUs, just nondeterminism from interleavings different threads. I.e. they will give you sequential consistency unless you have other, worse races.
    - zozbot234 4y ago
      Sequential consistency is absolutely optional, there are several other possible memory orderings that are far weaker. When programs involve multiple atomic locations concurrently, atomicity is only preserved at each separate location.
- toast0 4y ago
  > Fair point, but I'd qualify that OOO is only observable for racy programs. I mean, you're basically saying that it's only observable if you care to observe it.

titzer 4y ago

Fair point, but I'd qualify that OOO is only observable for racy programs. In languages with shared memory concurrency, but safety, races are either statically disallowed (Rust), or their results are constrained by a (weak) memory model. Java, e.g. only makes rough guarantees that you don't see out-of-thin air values and that you can't violate the type system. Java also is a little stricter around volatile memory access (they are sequentially consistent, iirc). Also worth noting, some reorderings that you see are not due to hardware, but due to compiler optimizations. Compilers have to obey the memory model, too. Generally, most programmers shouldn't need to be reasoning about OOO or memory order and be using fences. Rather, they should use libraries that are written by experts that use extra-lingual mechanisms (like compare-swap or fences) to guarantee a higher contract.

zozbot234 4y ago
Rust has atomic types that can observe races. Such "racy" code is useful because some concurrent code doesn't need the strictest guarantees (sequential consistency). Though, compared to C, Rust does not use the "Consume" memory ordering because so far it has turned out to be ill-defined.
- titzer 4y ago
  AFAICT those can't observe OOO of CPUs, just nondeterminism from interleavings different threads. I.e. they will give you sequential consistency unless you have other, worse races.
  - zozbot234 4y ago
    Sequential consistency is absolutely optional, there are several other possible memory orderings that are far weaker. When programs involve multiple atomic locations concurrently, atomicity is only preserved at each separate location.
toast0 4y ago
> Fair point, but I'd qualify that OOO is only observable for racy programs. I mean, you're basically saying that it's only observable if you care to observe it.