4 ms·

I'm a little bit surprised that Apple hasn't moved beyond 128-bit NEON, either with a custom 256 bit extension, or with an early implementation of SVE. This see

by addaon 6y ago

I'm a little bit surprised that Apple hasn't moved beyond 128-bit NEON, either with a custom 256 bit extension, or with an early implementation of SVE. This seems to be an area where vertical integration of software makes transitions easier -- getting Accelerate.framework over to something wider would have immediate pay-off, and especially at higher power budgets it seems clear from looking at the rest of the industry that somewhat wider vectors is a good perf to area trade-off.

wmf 6y ago
You mean like the AMX instructions?
api 6y ago
They probably don’t want to go beyond the standard. It could lead to compatibility and support headaches and accusations of “embrace extend extinguish” later. Not sure but I seem to recall the ARM licensing terms discouraging this.
- saagarjha 6y ago
  Too late, they have a number of proprietary extensions already.
  - api 6y ago
    But those don’t conflict with the spec. They can be ignored and it’s just an ARM64. An ahead of release implementation of new vector instructions could result in actual incompatibility with official ARM64.
addaon 6y ago
(As a side-note, can someone who has explored SVE in more detail than I comment on whether it's possible for a big.LITTLE-style implementation to use different vector widths on the big and little cores? One obvious challenge of going to wider vectors on something like the M1 is that you either pay the cost (area and power) for the capability on the small cores, or you lock threads that use those capabilities to big cores, neither of which is ideal. If you could have 128-bit SVE registers on little and 512-bit on big, it would help with that; but it's not clear to me how complex it would be to allow an SVE instruction to be interrupted and restart it on a core with a different vector width. I guess in extremis one could emulate the remainder of the instruction before restart, but eww.)
- thecompilr 6y ago
  Much easier would be to break the 512b operation into 4 128b uops on the small cores.
  - addaon 6y ago
    Right. But if you take an interrupt after the two of those three operations, and you want to restart on the big core, what do you do? Disallow interrupts in split operations? Unwind the partially-performed op? Allow the big core to run 128-bit ops to complete this one after restart? Something else?
    - ithkuil 6y ago
      Micro-ops, while require intermediate microarchitectural state, doesn't produce inconsistent architectural state when interrupted. The instruction either retired (produced effects) or didn't, from the POV of the ISA (and thus the kernel and user code)
      - addaon 6y ago
        But I thought that the SVE extension allowed interrupt and restart of an instruction? The alternative seems like it would lead to potentially very high interrupt latency?
        ithkuil 6y ago
        I don't know about SVE, just some general principles/thought: you either postpone the serving of the interrupt until the current instruction is done or you throw away progress made in the current instruction. My educated guess is that the latter is most frequently used as a technique. Also traps are caused by the very instruction being executed so you cannot complete the current instruction in all circumstances before vectoring to the trap/interrupt handler.
qeternity 6y ago
Unlikely that there would be any big gains on the mobile side, and this being their first desktop chip, they probably didn’t want to rock the boat. Apple nailed this, unbelievably. I have an M1 MacBook and it’s truly black magic. It’s already the fastest computer I’ve ever used. I’d rather have stability during the Rosetta 2 stage than added complexity and bugs by introducing a new bleeding edge instruction set.
- addaon 6y ago
  I suppose, but couldn't the same be said for the early 64-bit transition on Arm for Apple? They don't seem scared about biting off complexity today when they know they'll need it eventually. The other side of this is that we're now in a window where performance-sensitive developers are motivated to start looking at NEON and adding NEON support to existing SIMD code. Which is fine... but if SVE is the long-term answer and SVE will (as its goal) support better performance scaling of SIMD code with future processors, it seems like there's a real motivation for Apple to push developers into doing things "the right way" as early as possible. Assuming (!) that SVE is indeed the future, missing it for the first few generations is shades of 32-bit-Intel-for-Apple -- a stop-gap with some remarkably long-tailed support costs, compared to jumping straight to x64.
- specialist 6y ago
  I'm eager for someone to run the Dan Luu style keyboard latency tests on their new M1 MacBooks. It's the kind of fit & finish that I would expect Apple to obsess over. http://danluu.com/keyboard-latency/ http://danluu.com/keyboard-latency/
brigade 6y ago
M1’s big cores already match Intel and AMD consumer SKUs in flops/cycle... x86 pretty much had to go with wider ALUs because it was cheaper than widening instruction decode. Anyway there’s already AMX
dirtypersian 6y ago
Do you have a source that they don't already support some sort of SVE? I don't think they would make this public information if they don't have to. As far as I can tell they wouldn't need to recompile any previous ARM binaries so this could remain under the hood and they would never need to tell you whether or not they're doing it. So I'd be interested if you have any source on this.
- saagarjha 6y ago
  I don't think anyone has found any SVE being used by Apple's binaries. Sure, it could be there, hidden, but that would in practice mean that applications can't use it regardless.

by addaon 6y ago

wmf 6y ago

You mean like the AMX instructions?

api 6y ago

They probably don’t want to go beyond the standard. It could lead to compatibility and support headaches and accusations of “embrace extend extinguish” later. Not sure but I seem to recall the ARM licensing terms discouraging this.

saagarjha 6y ago
Too late, they have a number of proprietary extensions already.
- api 6y ago
  But those don’t conflict with the spec. They can be ignored and it’s just an ARM64. An ahead of release implementation of new vector instructions could result in actual incompatibility with official ARM64.

addaon 6y ago

(As a side-note, can someone who has explored SVE in more detail than I comment on whether it's possible for a big.LITTLE-style implementation to use different vector widths on the big and little cores? One obvious challenge of going to wider vectors on something like the M1 is that you either pay the cost (area and power) for the capability on the small cores, or you lock threads that use those capabilities to big cores, neither of which is ideal. If you could have 128-bit SVE registers on little and 512-bit on big, it would help with that; but it's not clear to me how complex it would be to allow an SVE instruction to be interrupted and restart it on a core with a different vector width. I guess in extremis one could emulate the remainder of the instruction before restart, but eww.)

thecompilr 6y ago
Much easier would be to break the 512b operation into 4 128b uops on the small cores.
- addaon 6y ago
  Right. But if you take an interrupt after the two of those three operations, and you want to restart on the big core, what do you do? Disallow interrupts in split operations? Unwind the partially-performed op? Allow the big core to run 128-bit ops to complete this one after restart? Something else?
  - ithkuil 6y ago
    Micro-ops, while require intermediate microarchitectural state, doesn't produce inconsistent architectural state when interrupted. The instruction either retired (produced effects) or didn't, from the POV of the ISA (and thus the kernel and user code)
    - addaon 6y ago
      But I thought that the SVE extension allowed interrupt and restart of an instruction? The alternative seems like it would lead to potentially very high interrupt latency?
      - ithkuil 6y ago
        I don't know about SVE, just some general principles/thought: you either postpone the serving of the interrupt until the current instruction is done or you throw away progress made in the current instruction. My educated guess is that the latter is most frequently used as a technique. Also traps are caused by the very instruction being executed so you cannot complete the current instruction in all circumstances before vectoring to the trap/interrupt handler.

qeternity 6y ago

Unlikely that there would be any big gains on the mobile side, and this being their first desktop chip, they probably didn’t want to rock the boat. Apple nailed this, unbelievably. I have an M1 MacBook and it’s truly black magic. It’s already the fastest computer I’ve ever used. I’d rather have stability during the Rosetta 2 stage than added complexity and bugs by introducing a new bleeding edge instruction set.

addaon 6y ago
I suppose, but couldn't the same be said for the early 64-bit transition on Arm for Apple? They don't seem scared about biting off complexity today when they know they'll need it eventually. The other side of this is that we're now in a window where performance-sensitive developers are motivated to start looking at NEON and adding NEON support to existing SIMD code. Which is fine... but if SVE is the long-term answer and SVE will (as its goal) support better performance scaling of SIMD code with future processors, it seems like there's a real motivation for Apple to push developers into doing things "the right way" as early as possible. Assuming (!) that SVE is indeed the future, missing it for the first few generations is shades of 32-bit-Intel-for-Apple -- a stop-gap with some remarkably long-tailed support costs, compared to jumping straight to x64.
specialist 6y ago
I'm eager for someone to run the Dan Luu style keyboard latency tests on their new M1 MacBooks. It's the kind of fit & finish that I would expect Apple to obsess over. http://danluu.com/keyboard-latency/ http://danluu.com/keyboard-latency/

brigade 6y ago

M1’s big cores already match Intel and AMD consumer SKUs in flops/cycle... x86 pretty much had to go with wider ALUs because it was cheaper than widening instruction decode. Anyway there’s already AMX

dirtypersian 6y ago

Do you have a source that they don't already support some sort of SVE? I don't think they would make this public information if they don't have to. As far as I can tell they wouldn't need to recompile any previous ARM binaries so this could remain under the hood and they would never need to tell you whether or not they're doing it. So I'd be interested if you have any source on this.

saagarjha 6y ago
I don't think anyone has found any SVE being used by Apple's binaries. Sure, it could be there, hidden, but that would in practice mean that applications can't use it regardless.