3 ms·

Why is that? We are talking user space applications rather than drivers/etc. Properly coded applications will be using some form of sync primitives to shared da

Why is that? We are talking user space applications rather than drivers/etc. Properly coded applications will be using some form of sync primitives to shared data structures. So, when the emulator hits a LOCK xxx instruction it simply replaces it with a LDAXR/STLXR (load acquire exclusive, store release exclusive which have implied barriers) pair, or drops a barrier in. Generally though, you would expect that most well behaved application are using EnterCriticalSection()/InterlockedXXXX() or other library calls that could be calling ARM64 optimized code (likely one of those hybrid libraries) which would also assure proper barriers. So, an emulator like this isn't going to be perfect, and applications which have been getting lucky due to x86's ordering model and avoiding proper sync primitives will likely break, and there really isn't much that can be done in those cases but fixing the application although it wouldn't surprise me if there is a compatibility flag which falls back to some slow path emulation mode which orders individual loads/stores and runs at 1/10 the normal speeds.

pranith 9y ago
> Properly coded applications will be using some form of sync primitives to shared data structures. So, when the emulator hits a LOCK xxx instruction This is not always true. See [1] for an example. Because of the semantics of the x86 memory model a sequentially consistent load does not generate a lock'd instruction. When such loads are translated to ARM64, you need to introduce barriers or use a ldacq instruction. [1] https://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html https://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html
- StillBored 9y ago
  That doesn't matter because a properly consistent lock will need a sequentially consistent store somewhere in the sequence, and that store will generate the barrier.Something like a ticket lock works in this case because the store will eventually become visible and when it does the ordering of operations proceeding it will have completed.
vardump 9y ago
You're talking about atomic operations and synchronization. This is not about that. This is about memory order. How CPU is allowed to reorder normal unsynchronized loads and stores. X86 has strong order, but ARM CPUs can reorder loads and stores significantly more. Thus on ARM you very often need memory barriers on ARM where x86 needs none. If memory order is not handled according to specifications, programs executing simultaneously on more than one CPU core, relying on CPU memory model/ordering will not work correctly. You don't want to make every ordinary load and store atomic, because that would incur a rather high performance cost.
- StillBored 9y ago
  Yes it is (about atomics/sync). ARM like every other major core is "sequentially consistent" with respect to the core executing the code. The memory order only matters for external visibility (aka other threads or devices). This means that the load/store order from a single thread doesn't matter to the other threads except at synchronization points (aka locks/etc). Those sync primitives have implied barriers. If someone has a program which is depending on load/store order in userspace then they likely have bugs on x86 as well since threads can be migrated between cores and the compilers are fully allowed to reorder load/stores as well as long as visible side effects are maintained.. I could go into the finer points of how compilers have to create internal barriers (this has nothing to do with DMB/SFENCE/etc) across external function calls as well (which plays into why you should be using library locking calls rather than creating your own) but that is another whole subject. The latter case is something I don't think most people understand... Particularly as GCC and friends get more aggressive about determining side effects and tossing code. Also, volatile doesn't do what most people think it does and trying to create sync primitives simply by forcing load/store does nothing when the compiler is still free to reorder the operations. An emulator is also going to maintain this contract as well. That is why things like qemu work just fine to run x86 binaries on random ARMs today without having to modify the hardware memory model.
  - vardump 9y ago
    > If someone has a program which is depending on load/store order in userspace then they likely have bugs on x86 as well Incorrect. On x86, if you write to memory n times, other cores are guaranteed to see the writes in same order. Second write is never going to be visible to other cores before first write. It's correct to rely on x86 memory model in x86 software. On ARM, those stores can become visible to other cores in any order. > since threads can be migrated between cores Irrelevant. This is about code executing concurrently on multiple cores. Operating system and threads are irrelevant. This is about hardware behavior, CPU core load/store system and instruction reordering, not software. > ... and the compilers are fully allowed to reorder load/stores... This has nothing to do with compilers. This has everything to do how CPU cores reorder reads and writes. > Particularly as GCC and friends get more aggressive about determining side effects and tossing code If GCC has bugs, please report them. Undefined behavior can give that impression, but again, this topic has nothing to do with compilers. > An emulator is also going to maintain this contract as well. That is why things like qemu work just fine to run x86 binaries on random ARMs today without having to modify the hardware memory model. This is not true. See: http://wiki.qemu.org/Features/tcg-multithread#Memory_consistency http://wiki.qemu.org/Features/tcg-multithread#Memory_consist... Remaining Case: strong on weak, ex. emulating x86 memory model on ARM systems I recommend you read this: https://en.wikipedia.org/wiki/Memory_ordering https://en.wikipedia.org/wiki/Memory_ordering.

pranith 9y ago

> Properly coded applications will be using some form of sync primitives to shared data structures. So, when the emulator hits a LOCK xxx instruction This is not always true. See [1] for an example. Because of the semantics of the x86 memory model a sequentially consistent load does not generate a lock'd instruction. When such loads are translated to ARM64, you need to introduce barriers or use a ldacq instruction. [1] https://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html https://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html

StillBored 9y ago
That doesn't matter because a properly consistent lock will need a sequentially consistent store somewhere in the sequence, and that store will generate the barrier.Something like a ticket lock works in this case because the store will eventually become visible and when it does the ordering of operations proceeding it will have completed.

vardump 9y ago

You're talking about atomic operations and synchronization. This is not about that. This is about memory order. How CPU is allowed to reorder normal unsynchronized loads and stores. X86 has strong order, but ARM CPUs can reorder loads and stores significantly more. Thus on ARM you very often need memory barriers on ARM where x86 needs none. If memory order is not handled according to specifications, programs executing simultaneously on more than one CPU core, relying on CPU memory model/ordering will not work correctly. You don't want to make every ordinary load and store atomic, because that would incur a rather high performance cost.

StillBored 9y ago
Yes it is (about atomics/sync). ARM like every other major core is "sequentially consistent" with respect to the core executing the code. The memory order only matters for external visibility (aka other threads or devices). This means that the load/store order from a single thread doesn't matter to the other threads except at synchronization points (aka locks/etc). Those sync primitives have implied barriers. If someone has a program which is depending on load/store order in userspace then they likely have bugs on x86 as well since threads can be migrated between cores and the compilers are fully allowed to reorder load/stores as well as long as visible side effects are maintained.. I could go into the finer points of how compilers have to create internal barriers (this has nothing to do with DMB/SFENCE/etc) across external function calls as well (which plays into why you should be using library locking calls rather than creating your own) but that is another whole subject. The latter case is something I don't think most people understand... Particularly as GCC and friends get more aggressive about determining side effects and tossing code. Also, volatile doesn't do what most people think it does and trying to create sync primitives simply by forcing load/store does nothing when the compiler is still free to reorder the operations. An emulator is also going to maintain this contract as well. That is why things like qemu work just fine to run x86 binaries on random ARMs today without having to modify the hardware memory model.
- vardump 9y ago
  > If someone has a program which is depending on load/store order in userspace then they likely have bugs on x86 as well Incorrect. On x86, if you write to memory n times, other cores are guaranteed to see the writes in same order. Second write is never going to be visible to other cores before first write. It's correct to rely on x86 memory model in x86 software. On ARM, those stores can become visible to other cores in any order. > since threads can be migrated between cores Irrelevant. This is about code executing concurrently on multiple cores. Operating system and threads are irrelevant. This is about hardware behavior, CPU core load/store system and instruction reordering, not software. > ... and the compilers are fully allowed to reorder load/stores... This has nothing to do with compilers. This has everything to do how CPU cores reorder reads and writes. > Particularly as GCC and friends get more aggressive about determining side effects and tossing code If GCC has bugs, please report them. Undefined behavior can give that impression, but again, this topic has nothing to do with compilers. > An emulator is also going to maintain this contract as well. That is why things like qemu work just fine to run x86 binaries on random ARMs today without having to modify the hardware memory model. This is not true. See: http://wiki.qemu.org/Features/tcg-multithread#Memory_consistency http://wiki.qemu.org/Features/tcg-multithread#Memory_consist... Remaining Case: strong on weak, ex. emulating x86 memory model on ARM systems I recommend you read this: https://en.wikipedia.org/wiki/Memory_ordering https://en.wikipedia.org/wiki/Memory_ordering.