4 ms·

> lock-prefixed instructions Are not the only instructions used for atomic operations. > Citation needed EDIT: memory_order_acq_rel like I've shown bellow i

> lock-prefixed instructions Are not the only instructions used for atomic operations. > Citation needed EDIT: memory_order_acq_rel like I've shown bellow isn't actually a valid value to pass to store (noticed this when trying to spot sane disassembly in MSVC for yet another datapoint, which was turning into a noop!) memory_order_seq_cst is, however, which gets compiled to xchg in both GCC, Clang, and MSVC... but memory_order_release and memory_order_relaxed are both legal too, and get compiled down to vanilla movs in GCC, Clang, and MSVC as well: https://clang.godbolt.org/z/QfnxPJ https://clang.godbolt.org/z/QfnxPJ void foo(std::atomic<int> & i) { i.store(42, std::memory_order_acq_rel); } Gets compiled down into a no-lock-prefix mov by clang (gcc emits xchg which is implicitly lock and will be atomic even straddling cacheline boundaries): https://clang.godbolt.org/z/87x98C https://clang.godbolt.org/z/87x98C Mov can't even take a lock prefix (https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf https://software.intel.com/sites/default/files/managed/39/c5... page 1158): > [...] The LOCK prefix can be prepended only to the following instructions and only to those forms of the instructions where the destination operand is a memory operand: ADD, ADC, AND, BTC, BTR, BTS, CMPXCHG, CMPXCH8B, CMPXCHG16B, DEC, INC, NEG, NOT, OR, SBB, SUB, XOR, XADD, and XCHG [...] And mov is only atomic when within a single cacheline on modern processors: (https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf https://software.intel.com/sites/default/files/managed/39/c5... page 3052): > The Intel486 processor (and newer processors since) guarantees that the following basic memory operations will always be carried out atomically: > [...] > • Reading or writing a doubleword aligned on a 32-bit boundary > The Pentium processor (and newer processors since) guarantees that the following additional memory operations will always be carried out atomically: > [...] > • Unaligned 16-, 32-, and 64-bit accesses to cached memory that fit within a cache line > Accesses to cacheable memory that are split across cache lines and page boundaries are not guaranteed to be atomic by the Intel Core 2 Duo, Intel® Atom™, Intel Core Duo, Pentium M, Pentium 4, Intel Xeon, P6 family, Pentium, and Intel486 processors. The Intel Core 2 Duo, Intel Atom, Intel Core Duo, Pentium M, Pentium 4, Intel Xeon, and P6 family processors provide bus control signals that permit external memory subsystems to make split accesses atomic; however, nonaligned data accesses will seriously impact the performance of the processor and should be avoided. (PDF is the latest version of "Intel® 64 and IA-32 Architectures Software Developer’s Manual, Combined Volumes: 1, 2A, 2B, 2C, 2D, 3A, 3B, 3C, 3D and 4" I found at https://software.intel.com/en-us/download/intel-64-and-ia-32-architectures-sdm-combined-volumes-1-2a-2b-2c-2d-3a-3b-3c-3d-and-4 https://software.intel.com/en-us/download/intel-64-and-ia-32... )

MaulingMonkey 7y ago
...and just to give a concrete example of a program that can silently "misrun" on x64 (via cacheline straddling - which is undefined behavior due to the misalignment, obviously): #include <atomic> #include <thread> #include <iostream> #include <cassert> #include <cstdlib> int main() { constexpr size_t target_cacheline_bits = 6; constexpr size_t target_cacheline_size = (1 << target_cacheline_bits); // 64 bytes constexpr size_t target_cacheline_straddle_mask = (target_cacheline_size - 1); char buffer[target_cacheline_size + sizeof(std::atomic<short>)]; std::atomic<short> & straddling = *(std::atomic<short> *)((size_t)buffer | target_cacheline_straddle_mask); // UB due to misalignment straddling.store(1); constexpr size_t N = 1000000; std::thread t1([&](){ for (size_t i=0; i<N; ++i) { straddling.store(i&1 ? 0x0001 : 0x0100, std::memory_order_relaxed); } }); std::thread t2([&](){ for (size_t i=0; i<N; ++i) { const short value = straddling.load(std::memory_order_relaxed); if (value != 0x0001 && value != 0x0100) { std::cerr << "Got value: " << value << "\n"; std::exit(1); } } }); t1.join(); t2.join(); } https://wandbox.org/permlink/IK43Ou0NfAw9y7us https://wandbox.org/permlink/IK43Ou0NfAw9y7us Sometimes this will run to completion without triggering anything. Other times the values "0" or "257" (0x0101) will appear in stderr. Feel free to raise "N", or exclude "0" from the error triggering case, to see more values.
gmueckl 7y ago
I get the impression that you're blaming flaws in the implementation of std::atomic on the processor. Memory reads using MOV are not atomic. That's not as bad as you make it sound. Instructions that are prefixed with LOCK (or implicitly LOCKed) are always atomic on misaligned data as per the last paragraph you quoted. This paragraph is weaseling around as this feature technically depends on external bus management that is implemented on the mainboard and outside the scope of the processor manual. But third party chipsets are dead. The same is repeated explicitly in the description of the LOCK prefix (Volume 2A, chapter 3): > The integrity of the LOCK prefix is not affected by the alignment of the memory field. Memory locking is observed for arbitrarily misaligned fields. As for the example code that you posted: I postulate that at least GCC is essentially broken and violates the guarantees given in their own manual and the C++ standard. The following example demonstrates this: #define UNSAFE template<typename valueType> class MyAtomic { public: valueType load() { #ifdef UNSAFE return __atomic_load_n(&value, __ATOMIC_RELAXED); #else valueType temp = 0; __atomic_compare_exchange_n (&value, &temp, temp, false, __ATOMIC_RELAXED, __ATOMIC_RELAXED); return temp; #endif } void store(valueType newValue) { #ifdef UNSAFE __atomic_store_n(&value, newValue, __ATOMIC_RELAXED); #else __atomic_exchange_n(&value, newValue, __ATOMIC_RELAXED); #endif } private: valueType value; }; The UNSAFE version compiles to simple mov instructions. This is not correct as there is no alignment guarantee for value. Using LOCK CMPXCHG and XCHG for the read and write is possible and safe. This is the code you get when undefining UNSAFE.

MaulingMonkey 7y ago

...and just to give a concrete example of a program that can silently "misrun" on x64 (via cacheline straddling - which is undefined behavior due to the misalignment, obviously): #include <atomic> #include <thread> #include <iostream> #include <cassert> #include <cstdlib> int main() { constexpr size_t target_cacheline_bits = 6; constexpr size_t target_cacheline_size = (1 << target_cacheline_bits); // 64 bytes constexpr size_t target_cacheline_straddle_mask = (target_cacheline_size - 1); char buffer[target_cacheline_size + sizeof(std::atomic<short>)]; std::atomic<short> & straddling = *(std::atomic<short> *)((size_t)buffer | target_cacheline_straddle_mask); // UB due to misalignment straddling.store(1); constexpr size_t N = 1000000; std::thread t1([&](){ for (size_t i=0; i<N; ++i) { straddling.store(i&1 ? 0x0001 : 0x0100, std::memory_order_relaxed); } }); std::thread t2([&](){ for (size_t i=0; i<N; ++i) { const short value = straddling.load(std::memory_order_relaxed); if (value != 0x0001 && value != 0x0100) { std::cerr << "Got value: " << value << "\n"; std::exit(1); } } }); t1.join(); t2.join(); } https://wandbox.org/permlink/IK43Ou0NfAw9y7us https://wandbox.org/permlink/IK43Ou0NfAw9y7us Sometimes this will run to completion without triggering anything. Other times the values "0" or "257" (0x0101) will appear in stderr. Feel free to raise "N", or exclude "0" from the error triggering case, to see more values.

gmueckl 7y ago

I get the impression that you're blaming flaws in the implementation of std::atomic on the processor. Memory reads using MOV are not atomic. That's not as bad as you make it sound. Instructions that are prefixed with LOCK (or implicitly LOCKed) are always atomic on misaligned data as per the last paragraph you quoted. This paragraph is weaseling around as this feature technically depends on external bus management that is implemented on the mainboard and outside the scope of the processor manual. But third party chipsets are dead. The same is repeated explicitly in the description of the LOCK prefix (Volume 2A, chapter 3): > The integrity of the LOCK prefix is not affected by the alignment of the memory field. Memory locking is observed for arbitrarily misaligned fields. As for the example code that you posted: I postulate that at least GCC is essentially broken and violates the guarantees given in their own manual and the C++ standard. The following example demonstrates this: #define UNSAFE template<typename valueType> class MyAtomic { public: valueType load() { #ifdef UNSAFE return __atomic_load_n(&value, __ATOMIC_RELAXED); #else valueType temp = 0; __atomic_compare_exchange_n (&value, &temp, temp, false, __ATOMIC_RELAXED, __ATOMIC_RELAXED); return temp; #endif } void store(valueType newValue) { #ifdef UNSAFE __atomic_store_n(&value, newValue, __ATOMIC_RELAXED); #else __atomic_exchange_n(&value, newValue, __ATOMIC_RELAXED); #endif } private: valueType value; }; The UNSAFE version compiles to simple mov instructions. This is not correct as there is no alignment guarantee for value. Using LOCK CMPXCHG and XCHG for the read and write is possible and safe. This is the code you get when undefining UNSAFE.