3 ms·

This replaces AMX, it is its successor. The older Apple CPUs implemented a custom form of AMX that was not standardized by Arm. Presumably as a result of coop

This replaces AMX, it is its successor. The older Apple CPUs implemented a custom form of AMX that was not standardized by Arm. Presumably as a result of cooperation with Apple, the Arm ISA now includes a set of instructions with the same purpose like the original Apple AMX. The newer Apple CPUs have been updated to use the standard Arm ISA, instead of their older proprietary ISA. In the Apple CPUs, the former AMX and the current SME provide a much higher throughput than the CPU cores, even if lower than the GPU, and a much lower latency than the GPU, even if higher than the CPU cores. AMX/SME is implemented as a separate accelerator, distinct from the CPU cores, because this saves power and area in comparison with implementing such instructions in each CPU core. The Apple CPUs do not attempt to compete in high-performance computing applications, so the extra throughput provided by a separate shared matrix operation accelerator is good enough for them.

saagarjha 2y ago
This has both actually.
- adrian_b 2y ago
  That must be for preserving the compatibility with the older software versions. The same happens in x86, where there are hundreds of obsolete instructions, which have been replaced by better instructions, but which are still supported to allow the execution of old programs. Both the new Arm SME instructions and the old Apple AMX instructions are executed by the same hardware matrix operation accelerator. Previously Arm has extended the Aarch64 ISA with the SVE instructions, in order to support the Fujitsu supercomputer. Then they were not satisfied with the original SVE and they have extended it into SVE2. I suppose that something similar has happened with SME. Apple must have negotiated with Arm the inclusion of matrix and vector operations implemented by a separate shared accelerator. The result was SME, which differs from the original AMX either because Apple has thought some improvements based on the experience with the first instruction set or because Arm has desired some changes from the Apple proposal.

saagarjha 2y ago

This has both actually.

adrian_b 2y ago
That must be for preserving the compatibility with the older software versions. The same happens in x86, where there are hundreds of obsolete instructions, which have been replaced by better instructions, but which are still supported to allow the execution of old programs. Both the new Arm SME instructions and the old Apple AMX instructions are executed by the same hardware matrix operation accelerator. Previously Arm has extended the Aarch64 ISA with the SVE instructions, in order to support the Fujitsu supercomputer. Then they were not satisfied with the original SVE and they have extended it into SVE2. I suppose that something similar has happened with SME. Apple must have negotiated with Arm the inclusion of matrix and vector operations implemented by a separate shared accelerator. The result was SME, which differs from the original AMX either because Apple has thought some improvements based on the experience with the first instruction set or because Arm has desired some changes from the Apple proposal.