5 ms·

I should add that my comment above is dependent on compiling with options that target a specific processor, when the performance on that processor is equally go

by nkurz 7y ago

I should add that my comment above is dependent on compiling with options that target a specific processor, when the performance on that processor is equally good with aligned and unaligned data. That is, when compiling with "-march=native" on a modern Intel processor, I strongly prefer a compiler that generates fast and working code over one that segfaults because the spec allows it to. I'm open to arguments that it should behave differently when asked to generate generic code that can work on generic processors. Are there x64 processors in common use today that are significantly slower with MOVDQU than MOVDQA?

gpderetta 7y ago
Movdqu vs movdqa is just one of the possible bad things that can happen when misaligned data is erroneously used. And it will segfault consistently which is a good thing which means that will be caught early in testing. You want compilers to produce binaries that will catch mistakes early in testing, not binaries that work 99.99% of the time and fail in obscure ways.
- MaulingMonkey 7y ago
  To expand on the 0.01% cases - atomic operations (be they used for lock-free algorithms, or to implement spinlocks) generally require alignment. If you're lucky, unaligned access will segfault. I have been so lucky, when unaligned global char[] buffers were recast to types containing pthread locks. Operations on said locks were kind enough to segfault when unit testing a port to ARM hardware. If you're unlucky, your "atomic" x64 instructions will silently revert to non-atomic behavior when straddling cachelines but otherwise "succeed". This will introduce one of the absolute nastiest kinds of heisenbugs into your codebase, in one of the hardest to understand and debug parts of your program. Speaking frankly - my coworkers won't figure it out. I won't figure it out. Instead, we'll ship unstable software, and maybe blame the hardware and suggest memtest. A movdqa segfault by comparison is a tame, easily understood, easily fixed, and may well help me catch and fix said accidentally unatomic behavior.
  - gmueckl 7y ago
    Citation needed that lock-prefixed instructions require alignment or must not span cache lines on x86. I don't see that in amy documentation.
    - MaulingMonkey 7y ago
      > lock-prefixed instructions Are not the only instructions used for atomic operations. > Citation needed EDIT: memory_order_acq_rel like I've shown bellow isn't actually a valid value to pass to store (noticed this when trying to spot sane disassembly in MSVC for yet another datapoint, which was turning into a noop!) memory_order_seq_cst is, however, which gets compiled to xchg in both GCC, Clang, and MSVC... but memory_order_release and memory_order_relaxed are both legal too, and get compiled down to vanilla movs in GCC, Clang, and MSVC as well: https://clang.godbolt.org/z/QfnxPJ https://clang.godbolt.org/z/QfnxPJ void foo(std::atomic<int> & i) { i.store(42, std::memory_order_acq_rel); } Gets compiled down into a no-lock-prefix mov by clang (gcc emits xchg which is implicitly lock and will be atomic even straddling cacheline boundaries): https://clang.godbolt.org/z/87x98C https://clang.godbolt.org/z/87x98C Mov can't even take a lock prefix (https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf https://software.intel.com/sites/default/files/managed/39/c5... page 1158): > [...] The LOCK prefix can be prepended only to the following instructions and only to those forms of the instructions where the destination operand is a memory operand: ADD, ADC, AND, BTC, BTR, BTS, CMPXCHG, CMPXCH8B, CMPXCHG16B, DEC, INC, NEG, NOT, OR, SBB, SUB, XOR, XADD, and XCHG [...] And mov is only atomic when within a single cacheline on modern processors: (https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf https://software.intel.com/sites/default/files/managed/39/c5... page 3052): > The Intel486 processor (and newer processors since) guarantees that the following basic memory operations will always be carried out atomically: > [...] > • Reading or writing a doubleword aligned on a 32-bit boundary > The Pentium processor (and newer processors since) guarantees that the following additional memory operations will always be carried out atomically: > [...] > • Unaligned 16-, 32-, and 64-bit accesses to cached memory that fit within a cache line > Accesses to cacheable memory that are split across cache lines and page boundaries are not guaranteed to be atomic by the Intel Core 2 Duo, Intel® Atom™, Intel Core Duo, Pentium M, Pentium 4, Intel Xeon, P6 family, Pentium, and Intel486 processors. The Intel Core 2 Duo, Intel Atom, Intel Core Duo, Pentium M, Pentium 4, Intel Xeon, and P6 family processors provide bus control signals that permit external memory subsystems to make split accesses atomic; however, nonaligned data accesses will seriously impact the performance of the processor and should be avoided. (PDF is the latest version of "Intel® 64 and IA-32 Architectures Software Developer’s Manual, Combined Volumes: 1, 2A, 2B, 2C, 2D, 3A, 3B, 3C, 3D and 4" I found at https://software.intel.com/en-us/download/intel-64-and-ia-32-architectures-sdm-combined-volumes-1-2a-2b-2c-2d-3a-3b-3c-3d-and-4 https://software.intel.com/en-us/download/intel-64-and-ia-32... )

by nkurz 7y ago

gpderetta 7y ago

Movdqu vs movdqa is just one of the possible bad things that can happen when misaligned data is erroneously used. And it will segfault consistently which is a good thing which means that will be caught early in testing. You want compilers to produce binaries that will catch mistakes early in testing, not binaries that work 99.99% of the time and fail in obscure ways.

MaulingMonkey 7y ago
To expand on the 0.01% cases - atomic operations (be they used for lock-free algorithms, or to implement spinlocks) generally require alignment. If you're lucky, unaligned access will segfault. I have been so lucky, when unaligned global char[] buffers were recast to types containing pthread locks. Operations on said locks were kind enough to segfault when unit testing a port to ARM hardware. If you're unlucky, your "atomic" x64 instructions will silently revert to non-atomic behavior when straddling cachelines but otherwise "succeed". This will introduce one of the absolute nastiest kinds of heisenbugs into your codebase, in one of the hardest to understand and debug parts of your program. Speaking frankly - my coworkers won't figure it out. I won't figure it out. Instead, we'll ship unstable software, and maybe blame the hardware and suggest memtest. A movdqa segfault by comparison is a tame, easily understood, easily fixed, and may well help me catch and fix said accidentally unatomic behavior.
- gmueckl 7y ago
  Citation needed that lock-prefixed instructions require alignment or must not span cache lines on x86. I don't see that in amy documentation.
  - MaulingMonkey 7y ago
    > lock-prefixed instructions Are not the only instructions used for atomic operations. > Citation needed EDIT: memory_order_acq_rel like I've shown bellow isn't actually a valid value to pass to store (noticed this when trying to spot sane disassembly in MSVC for yet another datapoint, which was turning into a noop!) memory_order_seq_cst is, however, which gets compiled to xchg in both GCC, Clang, and MSVC... but memory_order_release and memory_order_relaxed are both legal too, and get compiled down to vanilla movs in GCC, Clang, and MSVC as well: https://clang.godbolt.org/z/QfnxPJ https://clang.godbolt.org/z/QfnxPJ void foo(std::atomic<int> & i) { i.store(42, std::memory_order_acq_rel); } Gets compiled down into a no-lock-prefix mov by clang (gcc emits xchg which is implicitly lock and will be atomic even straddling cacheline boundaries): https://clang.godbolt.org/z/87x98C https://clang.godbolt.org/z/87x98C Mov can't even take a lock prefix (https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf https://software.intel.com/sites/default/files/managed/39/c5... page 1158): > [...] The LOCK prefix can be prepended only to the following instructions and only to those forms of the instructions where the destination operand is a memory operand: ADD, ADC, AND, BTC, BTR, BTS, CMPXCHG, CMPXCH8B, CMPXCHG16B, DEC, INC, NEG, NOT, OR, SBB, SUB, XOR, XADD, and XCHG [...] And mov is only atomic when within a single cacheline on modern processors: (https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf https://software.intel.com/sites/default/files/managed/39/c5... page 3052): > The Intel486 processor (and newer processors since) guarantees that the following basic memory operations will always be carried out atomically: > [...] > • Reading or writing a doubleword aligned on a 32-bit boundary > The Pentium processor (and newer processors since) guarantees that the following additional memory operations will always be carried out atomically: > [...] > • Unaligned 16-, 32-, and 64-bit accesses to cached memory that fit within a cache line > Accesses to cacheable memory that are split across cache lines and page boundaries are not guaranteed to be atomic by the Intel Core 2 Duo, Intel® Atom™, Intel Core Duo, Pentium M, Pentium 4, Intel Xeon, P6 family, Pentium, and Intel486 processors. The Intel Core 2 Duo, Intel Atom, Intel Core Duo, Pentium M, Pentium 4, Intel Xeon, and P6 family processors provide bus control signals that permit external memory subsystems to make split accesses atomic; however, nonaligned data accesses will seriously impact the performance of the processor and should be avoided. (PDF is the latest version of "Intel® 64 and IA-32 Architectures Software Developer’s Manual, Combined Volumes: 1, 2A, 2B, 2C, 2D, 3A, 3B, 3C, 3D and 4" I found at https://software.intel.com/en-us/download/intel-64-and-ia-32-architectures-sdm-combined-volumes-1-2a-2b-2c-2d-3a-3b-3c-3d-and-4 https://software.intel.com/en-us/download/intel-64-and-ia-32... )