5 ms·

There's another option: 4) Translate x86 loads and stores to acquire load and release stores, to align with the x86 semantics. These already exist in the ARM I

There's another option: 4) Translate x86 loads and stores to acquire load and release stores, to align with the x86 semantics. These already exist in the ARM ISA, so it's not much of a stretch at all. This is the one I'm betting on.

monocasa 6y ago
Expect you'd need to do that with every single store since the x86 instruction stream doesn't have those semantics embedded in it. That'd kill your memory perf by at least an order of magnitude, and kill perf for other cores as well. It'd be cheaper to just say "you only get one core in x86 mode". Essentially you'd be marking every store as an L1 and store buffer flush and only operating out of L2.
- my123 6y ago
  What I can say is that they aren't pinning to only a single core, so the answer is elsewhere.
- phire 6y ago
  ARMv8.1 adds a bunch of improved atomic instructions that basically implement the same functionality as x86. Because x86 has atomic read-modify-write instructions by default; You need emulate those too. The ARMv8.1 extensions look like they have been explicitly designed to allow emulation of x86. The implication is that implementations of ARMv8.1 can (and perhaps should) implement cache/coherency subsystems with high preformance atomic operations. And I'm willing to bet Apple has made sure their implementation is good at atomics.
  - monocasa 6y ago
    So the ARMV8.1 extensions aren't for emulating x86, they're a reflection of how concurrency hardware has changed over the years. It used to be that (and you can see this in the RISC ISAs from the 80s/90s, but AIUI this is what happend in x86 microcode as well) * the CPU would read a line and lock it for modifications in cache. * the CPU core would do the modification * the CPU would write the new value down to L2 with an unlock, and L2 would now allow this cache line to be accessed So you're looking at ~15 cycles of contention from L2 through the CPU core and back down to L2 of a lock for that line. If this all looks really close to a subset of hardware transnational memory when you realize that the store can fail and the CPU has to do it again if some resource limit exceeded, you're not alone. Then somebody figured out that you can just stick an ALU directly in L2, and send the ALU ops down to it in the memory requests instead of locking lines. That reduces the contention time to around a couple cycles. These ALU ops can also be easily included in the coherency protocol, allowing remote NUMA RMW atomics without thrashing cache lines like you'd normally need to. This is why you see both in the RISC-V A extension as well. The underlying hardware has implementations in both models. I've heard rumors that earlier ARM tried to do macro op fusion to build atomic RMWs out of common sequences, but there were enough versions of that in the wild that it didn't give the benefits they were expecting. However, all that being said, atomics are orthogonal to what I'm talking about. It's x86's TSO model, and how it _lacks_ barriers in places that ARM requires them, with no context just from the instruction stream about where they're necessary that's the problem here, not emulating the explicit atomic sequences.
    - brigade 6y ago
      On the other hand, the ARMv8.5 flag manipulation instructions almost certainly were added specifically for x86 emulation.
- BeeOnRope 6y ago
  Yes, you need it for every store. It absolutely does not imply that it would kill your memory perf by an order of magnitude! Remember that x86 does this as their normal store behavior and they don't take an order of magnitude hit. The easiest approach that leads to fast release stores is just to have a stronger store pipeline that doesn't allow store-store or load-store reordring. The latter basically comes for free and most weak pipelines already preserve it. The store-store ordering, on the other hand, does have a cost: primarily in requiring stores to drain order, and handling misses in order. Nothing close to an order of magnitude. A higher performance design would allow store store reordering but not around release stores. Allowing most ARM code to the full benefit while allowing fast release stores for x86. I think you are mixing up release stores with more expensive atomics like seq-cst stores or atomic RMWs. There is no need for a store buffer drain, ever, for release stores.
gpderetta 6y ago
That would be too expensive, unless the emulation in general is ao high not to matter. If store releases and load aquires were very cheap already why make them distinct from normal one anyway? I suspect that either the CPU is already TSO in practice or has some TSO mode.
- saagarjha 6y ago
  Then why not expose that to ARM applications running on it?
  - gpderetta 6y ago
    Apple might not want to promote this to an architectural guarantee. It is only needed for the few years required to transition away from x86, but if applications start relying on it it will need to maintain it forever.
    - saagarjha 6y ago
      If they run with it for long enough it will all but become an architectural guarantee as people unintentionally write incorrect programs that happen to still work right.
- BeeOnRope 6y ago
  > If store releases and load aquires were very cheap already why make them distinct from normal one anyway? Perhaps if the ARM ISA was designed today, they would, I'm not sure. My impression is that ARM added them when (a) it became obvious the way the wind was blowing with respect to modern memory models, like Java (sort of), C and C++, where acquire and release are the dominant paradigm (and I doubt any other langue will stray very far), and (b) they could see that these operations implementations can be implemented at a relatively low complexity cost and reasonable performance, compared to the existing barrier approach. That said, there is a lot of room between "very cheap" and "too expensive", and it is probably hardware dependent. On some simple microcontroller-like design that doesn't want to do any extra work to support these, they might be very expensive, just relying on something like a full barrier which they have to support anyways. However, on bigger designs, it is not necessarily very expensive to implement efficient release stores. They will still be slower than plain stores, but may not all that much slower. It's nothing like sequentially consistent stores, or barrier types that require a full store buffer drain. Mostly all you need to need is to ensure that the stores drain in order. Actually the requirement is even weaker: you can't let release stores pass older stores. So it depends on your store buffer design and prevents some optimizations like merging and out-of-order commit to L1D, but those are far from critical optimizations (and you can still do them for plain stores at the cost of a bit more complexity). If you are designing a core where release stores only occur in concurrent code, e.g,. as a result of a release store in C++, or as a lock release or whatever, you don't need to make them that fast. If they are 10x slower than regular stores, it's probably OK. However, if you are designing a chip where you know you are going to be running a lot of emulated x86 code where every store is a store release, then yeah you are doing to do a bit more work to make these fast. What are the other options? (1) and (2) in the GPs list seem very unlikely to me. (1) would almost certainly be much worse than release stores for multithreaded apps and destroy performance in popular creator apps. (3) is certainly plausible and one variant of what I'm suggesting here: it means making plain stores and release stores the same thing (and I guess for loads). Definitely possible but seems less likely to me than (4). Another possibility is a TSO mode, as you suggest. Perhaps this is somewhat easier than dynamically handling release stores, I'm not sure.

monocasa 6y ago

Expect you'd need to do that with every single store since the x86 instruction stream doesn't have those semantics embedded in it. That'd kill your memory perf by at least an order of magnitude, and kill perf for other cores as well. It'd be cheaper to just say "you only get one core in x86 mode". Essentially you'd be marking every store as an L1 and store buffer flush and only operating out of L2.

my123 6y ago
What I can say is that they aren't pinning to only a single core, so the answer is elsewhere.
phire 6y ago
ARMv8.1 adds a bunch of improved atomic instructions that basically implement the same functionality as x86. Because x86 has atomic read-modify-write instructions by default; You need emulate those too. The ARMv8.1 extensions look like they have been explicitly designed to allow emulation of x86. The implication is that implementations of ARMv8.1 can (and perhaps should) implement cache/coherency subsystems with high preformance atomic operations. And I'm willing to bet Apple has made sure their implementation is good at atomics.
- monocasa 6y ago
  So the ARMV8.1 extensions aren't for emulating x86, they're a reflection of how concurrency hardware has changed over the years. It used to be that (and you can see this in the RISC ISAs from the 80s/90s, but AIUI this is what happend in x86 microcode as well) * the CPU would read a line and lock it for modifications in cache. * the CPU core would do the modification * the CPU would write the new value down to L2 with an unlock, and L2 would now allow this cache line to be accessed So you're looking at ~15 cycles of contention from L2 through the CPU core and back down to L2 of a lock for that line. If this all looks really close to a subset of hardware transnational memory when you realize that the store can fail and the CPU has to do it again if some resource limit exceeded, you're not alone. Then somebody figured out that you can just stick an ALU directly in L2, and send the ALU ops down to it in the memory requests instead of locking lines. That reduces the contention time to around a couple cycles. These ALU ops can also be easily included in the coherency protocol, allowing remote NUMA RMW atomics without thrashing cache lines like you'd normally need to. This is why you see both in the RISC-V A extension as well. The underlying hardware has implementations in both models. I've heard rumors that earlier ARM tried to do macro op fusion to build atomic RMWs out of common sequences, but there were enough versions of that in the wild that it didn't give the benefits they were expecting. However, all that being said, atomics are orthogonal to what I'm talking about. It's x86's TSO model, and how it _lacks_ barriers in places that ARM requires them, with no context just from the instruction stream about where they're necessary that's the problem here, not emulating the explicit atomic sequences.
  - brigade 6y ago
    On the other hand, the ARMv8.5 flag manipulation instructions almost certainly were added specifically for x86 emulation.
BeeOnRope 6y ago
Yes, you need it for every store. It absolutely does not imply that it would kill your memory perf by an order of magnitude! Remember that x86 does this as their normal store behavior and they don't take an order of magnitude hit. The easiest approach that leads to fast release stores is just to have a stronger store pipeline that doesn't allow store-store or load-store reordring. The latter basically comes for free and most weak pipelines already preserve it. The store-store ordering, on the other hand, does have a cost: primarily in requiring stores to drain order, and handling misses in order. Nothing close to an order of magnitude. A higher performance design would allow store store reordering but not around release stores. Allowing most ARM code to the full benefit while allowing fast release stores for x86. I think you are mixing up release stores with more expensive atomics like seq-cst stores or atomic RMWs. There is no need for a store buffer drain, ever, for release stores.

gpderetta 6y ago

That would be too expensive, unless the emulation in general is ao high not to matter. If store releases and load aquires were very cheap already why make them distinct from normal one anyway? I suspect that either the CPU is already TSO in practice or has some TSO mode.

saagarjha 6y ago
Then why not expose that to ARM applications running on it?
- gpderetta 6y ago
  Apple might not want to promote this to an architectural guarantee. It is only needed for the few years required to transition away from x86, but if applications start relying on it it will need to maintain it forever.
  - saagarjha 6y ago
    If they run with it for long enough it will all but become an architectural guarantee as people unintentionally write incorrect programs that happen to still work right.
BeeOnRope 6y ago
> If store releases and load aquires were very cheap already why make them distinct from normal one anyway? Perhaps if the ARM ISA was designed today, they would, I'm not sure. My impression is that ARM added them when (a) it became obvious the way the wind was blowing with respect to modern memory models, like Java (sort of), C and C++, where acquire and release are the dominant paradigm (and I doubt any other langue will stray very far), and (b) they could see that these operations implementations can be implemented at a relatively low complexity cost and reasonable performance, compared to the existing barrier approach. That said, there is a lot of room between "very cheap" and "too expensive", and it is probably hardware dependent. On some simple microcontroller-like design that doesn't want to do any extra work to support these, they might be very expensive, just relying on something like a full barrier which they have to support anyways. However, on bigger designs, it is not necessarily very expensive to implement efficient release stores. They will still be slower than plain stores, but may not all that much slower. It's nothing like sequentially consistent stores, or barrier types that require a full store buffer drain. Mostly all you need to need is to ensure that the stores drain in order. Actually the requirement is even weaker: you can't let release stores pass older stores. So it depends on your store buffer design and prevents some optimizations like merging and out-of-order commit to L1D, but those are far from critical optimizations (and you can still do them for plain stores at the cost of a bit more complexity). If you are designing a core where release stores only occur in concurrent code, e.g,. as a result of a release store in C++, or as a lock release or whatever, you don't need to make them that fast. If they are 10x slower than regular stores, it's probably OK. However, if you are designing a chip where you know you are going to be running a lot of emulated x86 code where every store is a store release, then yeah you are doing to do a bit more work to make these fast. What are the other options? (1) and (2) in the GPs list seem very unlikely to me. (1) would almost certainly be much worse than release stores for multithreaded apps and destroy performance in popular creator apps. (3) is certainly plausible and one variant of what I'm suggesting here: it means making plain stores and release stores the same thing (and I guess for loads). Definitely possible but seems less likely to me than (4). Another possibility is a TSO mode, as you suggest. Perhaps this is somewhat easier than dynamically handling release stores, I'm not sure.