15 ms·

I don't understand the significance of this. Could someone who does add some context?

by lwhi 6y ago

ericlewis 6y ago
From what I can tell reading this: the Accelerate framework from Apple seems to use custom instructions possibly unique to the M1 chip but potentially similar to Intel AMX. Accelerate is, as I sort of understand it- a way to do advanced math in a very quick / power efficient way. It is touted as deeply integrated with Apple processors from the day it launched and (I think) even recommended to be used over SIMD and the sort (I could certainly be very wrong about this). It dove tails with some of the machine learning work Apple did as well, is portable among Apple systems, and can be assumed to be optimized. Ninja edit: I’ve not seen how it was achieved until now, but a custom ISA extension isn’t wholly surprising. Edit 2: as for why this is significant.. I’m not sure it is. It is interesting to know how Accelerate can pull off what it does though. IIRC there are other machine learning frameworks which take advantage of the neural engine- but Accelerate possibly doesn’t- despite having machine learning capabilities.
- Lammy 6y ago
  It also means that any binaries built to use those instructions won't run on any non-Apple ARM64 CPUs, at least not easily. Sort of like how early-era Intel Hackintosh users had to have CPUs with SSE3 support or wait/hope for SSE2 patches.
  - ericlewis 6y ago
    I’m relatively sure that Accelerate does work fine on intel processors and is still the recommended means of doing heavy math. But that could just be because they knew of this eventual future. I’m not sure what Accelerate does under the hood for intel machines. But I would assume perhaps falls back to whatever intel might support.
- giantrobot 6y ago
  Say hello to Function Multi-versioning [0]. You tell the compiler to build a version for each march defined in the attribute. When that function is called a quick CPU check (via CPUID) is performed and if there's a version specific to the current CPU that will run instead of the more general implementation. ICC has supported this for a long time and it's widely used in Intel's MKL. LLVM also supports FMV but uses a slightly different method to define it in code. IIRC there was an LLVM issue tracking work to support GCC's way of declaring FMV but I don't know if that landed yet. Since the function's signature is the same between the multiple implementations the caller doesn't need to change of their code to get a free boost from features like AVX2 or these M1 instructions. When an M2 comes out with better vector instructions or something and a new Accelerate framework is released, anyone using the framework get a free performance increase without recompiling anything. [0] https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/
  - floatboth 6y ago
    At least __attribute__((ifunc(..))) is common between clang and gcc. (E.g. no compiler conditionals are used in https://cgit.freebsd.org/src/tree/sys/arm64/include/ifunc.h https://cgit.freebsd.org/src/tree/sys/arm64/include/ifunc.h) The fancier high level stuff that automatically compiles multiple versions of a function, I'm not sure.
  - saagarjha 6y ago
    Note that on ARM it is more annoying to get CPU capabilities as there is no straight cpuid equivalent.
    - stephencanon 6y ago
      On Apple platforms, CPU capabilities are cached on the commpage, so system libraries don’t even need to use cpuid on x86.
jonathonf 6y ago
From my limited understanding: it looks like a matrix coprocessor, and implies that hardware-accelerated matrix processing has been built into the M1. Matrix processing comes in useful for AI/ML/CV-type applications. It's something GPUs are generally very good for.
mhh__ 6y ago
In the bigger picture Apple aren't really documenting anything for their new toys, which is not the end of the world but is a huge step back for openness. There is - as far as I can see - no long-form documentation for M1 other than patents. I already refuse to own apple products so I don't really have any skin in the game, but consider that if microsoft were like this the blind eye would not be turned - buy our new product, everything you run is up to us, we won't document it, maybe we'll upstream the compiler, tough shit. The fact that they can be like this is really proof that their market position is much stronger than most HNers seem to think it is.
- MichaelZuo 6y ago
  The implication I assume that there would be pushback on Microsoft for doing the same would be because of the possibility of nefarious undocumented changes. What possibly nefarious things could an undocumented ISA extension do?
  - mhh__ 6y ago
    Undocumented It's not about EEE it's about Apple taking this attitude in the first place.
  - hyperman1 6y ago
    Presumably the next iteration of the processor can remove them and use the opcodes for something else. Apple recompiles their system library, hence is unaffected. You can't do the same so peak M1 performance is unavailable for you.
    - MichaelZuo 6y ago
      Could those opcodes then be used for anything nefarious, or would it just restrict third parties from fully utilizing the processor?
      - Someone 6y ago
        If you think Apple wants to do something nefarious to users of their hardware, don’t buy or use it. If they can’t be trusted, they don’t need to add opcodes.
vegetablepotpie 6y ago
Apple added custom instructions on top of the ARM instruction set to do matrix operations. Matrix operations are used a lot in some algorithms, such as in computer graphics and machine learning and these instructions help those operations go faster. Although the ARM ISA has a multiply and accumulate instruction, it would take many more instructions on a standard ARM core to compute a matrix with only standard instructions. That enables the M1 chip to be faster in this case when developers use these instructions in their applications.
- giantrobot 6y ago
  > That enables the M1 chip to be faster in this case when developers use these instructions in their applications. Or more typically the compiler will emit those instructions and everyone gets them for free.
  - teknopaul 6y ago
    only if the compiler knows they exist
  - a1369209993 6y ago
    That's actually generally not true WRT SIMD/vector instructions (see eg http://yosefk.com/blog/humans-and-compilers-need-each-other-the-vliw-simd-case.html http://yosefk.com/blog/humans-and-compilers-need-each-other-... which contrasts this with the more compiler-friendly VLIW case); they impose nontrivial requirements on data structures and ABI, so the application code needs to at least be aware of the possibility of vectorization, even if it's agnostic to whether the implementation actually uses it.
- 2wrist 6y ago
  Similar to Altivec from the PowerPC days?
  - floatboth 6y ago
    Altivec is regular SIMD like NEON or SSE/AVX
joseph_grobbles 6y ago
Apple added special AMX instructions specifically for matrix operations. They added them back with the A13, and then improved them for the M1. These are primarily focused on machine learning training where you do backpropagation through huge matrix operations. They provide a very wide coverage library set that works on all supported platforms optimally. You interact with AMX through those libraries, and direct usage is considered unsupported. Apple may completely change it with the next iteration, where they can just change those libraries and every arm's-length application will get those benefits for free. Apple sells whole systems. They aren't selling CPUs for third parties, and there is no reason they need to encourage their own proprietary extensions. Indeed, it would be more evil if they were asking people to pepper their own code with this, instead of just using libraries that abstract you from the magic. As a tiny, tiny vendor in the computing space -- as HN often assures us -- I'm not seeing the great evil many are claiming. (indeed, of course this is downvoted but I chuckle that another comment implores that Apple is evil because this will lead to code and binaries that only work on Apple devices. Which is quite literally exactly the opposite of what Apple is doing, which is forcing use through optimized libraries, abstracting from their own proprietary extensions. People just love bitching)
- notretarded 6y ago
  >These are primarily focused on machine learning training where you do backpropagation through huge matrix operations How is this different from normal matrix operations?
  - joseph_grobbles 6y ago
    Scale, and only scale. As with most extensions, there's negative value if you're doing a small number of calcs, but it pays off in larger needs like ML training. We've been calculating matrixes for time eternal, but AMX has just recently become a thing. Intel just introduced their own AMX instructions.
    - touisteur 6y ago
      I don't know, if you want to perform DL inference on 4k/8k video real-time you're gonna need some heavy duty matrix multiplication resources. GPU is great for batched inferences but for quick no-pcie-transfer, small-to-no-batching inferences you want something close to the CPU...
ChuckMcM 6y ago
Like Intel, Apple has chosen to add some instructions to the processor which accelerate certain operations (in this case matrix operations). Unlike Intel, Apple has not yet chosen to actually document the M1 architecture. This is where a vertically integrated company like Apple (or IBM back in the day) can leverage "inside knowledge" about their chips to achieve performance that is not readily comparable (or reproducible) on a different instruction set architecture (ISA). Historically (see IBM vs Memorex) obscuring your interfaces has been a losing proposition in the long term, even while delivering favorable margins in the short term. Unlike Intel which needs to get third parties to write software for their chips, Apple writes their own software and so it means they can hold on to their advantage while third parties (like the author of the post) reverse engineer what is going on. It will be "historically significant" if Intel adds a feature to their chips in order to stay competitive with this development. The last time they did that was when AMD introduced Opteron. All in all, its basically just a puzzle that people who are interested in how things work get to work on in their spare time.
- watersb 6y ago
  "IBM vs Memorex": Memorex created a disk drive that was plug compatible with IBM mainframes. IBM of course refused to support any system that dared to use these drives. And sued Memorex for copying their design. Memorex counter-sued. This all forced the court to rule that reverse engineering was permissible. https://www.computerhistory.org/storageengine/memorex-introduces-an-ibm-compatible-hdd/ https://www.computerhistory.org/storageengine/memorex-introd...
  - spullara 6y ago
    Is it [IBM] or is it Memorex?
- kijiki 6y ago
  > It will be "historically significant" if Intel adds a feature to their chips in order to stay competitive with this development. TFA points out that this Apple AMX is similar to the existing Intel AMX, even (confusingly) using the same name.
- 6y ago
ytch 6y ago
Theoretically we can find out why M1 is fast by instructions. But there is no instruction manual, we can only reverse engineering it

I don't understand the significance of this. Could someone who does add some context?

by lwhi 6y ago

I don't understand the significance of this. Could someone who does add some context?

From what I can tell reading this: the Accelerate framework from Apple seems to use custom instructions possibly unique to the M1 chip but potentially similar to Intel AMX. Accelerate is, as I sort of understand it- a way to do advanced math in a very quick / power efficient way. It is touted as deeply integrated with Apple processors from the day it launched and (I think) even recommended to be used over SIMD and the sort (I could certainly be very wrong about this). It dove tails with some of the machine learning work Apple did as well, is portable among Apple systems, and can be assumed to be optimized. Ninja edit: I’ve not seen how it was achieved until now, but a custom ISA extension isn’t wholly surprising. Edit 2: as for why this is significant.. I’m not sure it is. It is interesting to know how Accelerate can pull off what it does though. IIRC there are other machine learning frameworks which take advantage of the neural engine- but Accelerate possibly doesn’t- despite having machine learning capabilities.

Lammy 6y ago
It also means that any binaries built to use those instructions won't run on any non-Apple ARM64 CPUs, at least not easily. Sort of like how early-era Intel Hackintosh users had to have CPUs with SSE3 support or wait/hope for SSE2 patches.
- ericlewis 6y ago
  I’m relatively sure that Accelerate does work fine on intel processors and is still the recommended means of doing heavy math. But that could just be because they knew of this eventual future. I’m not sure what Accelerate does under the hood for intel machines. But I would assume perhaps falls back to whatever intel might support.
giantrobot 6y ago
Say hello to Function Multi-versioning [0]. You tell the compiler to build a version for each march defined in the attribute. When that function is called a quick CPU check (via CPUID) is performed and if there's a version specific to the current CPU that will run instead of the more general implementation. ICC has supported this for a long time and it's widely used in Intel's MKL. LLVM also supports FMV but uses a slightly different method to define it in code. IIRC there was an LLVM issue tracking work to support GCC's way of declaring FMV but I don't know if that landed yet. Since the function's signature is the same between the multiple implementations the caller doesn't need to change of their code to get a free boost from features like AVX2 or these M1 instructions. When an M2 comes out with better vector instructions or something and a new Accelerate framework is released, anyone using the framework get a free performance increase without recompiling anything. [0] https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/
- floatboth 6y ago
  At least __attribute__((ifunc(..))) is common between clang and gcc. (E.g. no compiler conditionals are used in https://cgit.freebsd.org/src/tree/sys/arm64/include/ifunc.h https://cgit.freebsd.org/src/tree/sys/arm64/include/ifunc.h) The fancier high level stuff that automatically compiles multiple versions of a function, I'm not sure.
- saagarjha 6y ago
  Note that on ARM it is more annoying to get CPU capabilities as there is no straight cpuid equivalent.
  - stephencanon 6y ago
    On Apple platforms, CPU capabilities are cached on the commpage, so system libraries don’t even need to use cpuid on x86.

jonathonf 6y ago

From my limited understanding: it looks like a matrix coprocessor, and implies that hardware-accelerated matrix processing has been built into the M1. Matrix processing comes in useful for AI/ML/CV-type applications. It's something GPUs are generally very good for.

mhh__ 6y ago

In the bigger picture Apple aren't really documenting anything for their new toys, which is not the end of the world but is a huge step back for openness. There is - as far as I can see - no long-form documentation for M1 other than patents. I already refuse to own apple products so I don't really have any skin in the game, but consider that if microsoft were like this the blind eye would not be turned - buy our new product, everything you run is up to us, we won't document it, maybe we'll upstream the compiler, tough shit. The fact that they can be like this is really proof that their market position is much stronger than most HNers seem to think it is.

MichaelZuo 6y ago
The implication I assume that there would be pushback on Microsoft for doing the same would be because of the possibility of nefarious undocumented changes. What possibly nefarious things could an undocumented ISA extension do?
- mhh__ 6y ago
  Undocumented It's not about EEE it's about Apple taking this attitude in the first place.
- hyperman1 6y ago
  Presumably the next iteration of the processor can remove them and use the opcodes for something else. Apple recompiles their system library, hence is unaffected. You can't do the same so peak M1 performance is unavailable for you.
  - MichaelZuo 6y ago
    Could those opcodes then be used for anything nefarious, or would it just restrict third parties from fully utilizing the processor?
    - Someone 6y ago
      If you think Apple wants to do something nefarious to users of their hardware, don’t buy or use it. If they can’t be trusted, they don’t need to add opcodes.

vegetablepotpie 6y ago

Apple added custom instructions on top of the ARM instruction set to do matrix operations. Matrix operations are used a lot in some algorithms, such as in computer graphics and machine learning and these instructions help those operations go faster. Although the ARM ISA has a multiply and accumulate instruction, it would take many more instructions on a standard ARM core to compute a matrix with only standard instructions. That enables the M1 chip to be faster in this case when developers use these instructions in their applications.

giantrobot 6y ago
> That enables the M1 chip to be faster in this case when developers use these instructions in their applications. Or more typically the compiler will emit those instructions and everyone gets them for free.
- teknopaul 6y ago
  only if the compiler knows they exist
- a1369209993 6y ago
  That's actually generally not true WRT SIMD/vector instructions (see eg http://yosefk.com/blog/humans-and-compilers-need-each-other-the-vliw-simd-case.html http://yosefk.com/blog/humans-and-compilers-need-each-other-... which contrasts this with the more compiler-friendly VLIW case); they impose nontrivial requirements on data structures and ABI, so the application code needs to at least be aware of the possibility of vectorization, even if it's agnostic to whether the implementation actually uses it.
2wrist 6y ago
Similar to Altivec from the PowerPC days?
- floatboth 6y ago
  Altivec is regular SIMD like NEON or SSE/AVX

joseph_grobbles 6y ago

Apple added special AMX instructions specifically for matrix operations. They added them back with the A13, and then improved them for the M1. These are primarily focused on machine learning training where you do backpropagation through huge matrix operations. They provide a very wide coverage library set that works on all supported platforms optimally. You interact with AMX through those libraries, and direct usage is considered unsupported. Apple may completely change it with the next iteration, where they can just change those libraries and every arm's-length application will get those benefits for free. Apple sells whole systems. They aren't selling CPUs for third parties, and there is no reason they need to encourage their own proprietary extensions. Indeed, it would be more evil if they were asking people to pepper their own code with this, instead of just using libraries that abstract you from the magic. As a tiny, tiny vendor in the computing space -- as HN often assures us -- I'm not seeing the great evil many are claiming. (indeed, of course this is downvoted but I chuckle that another comment implores that Apple is evil because this will lead to code and binaries that only work on Apple devices. Which is quite literally exactly the opposite of what Apple is doing, which is forcing use through optimized libraries, abstracting from their own proprietary extensions. People just love bitching)

notretarded 6y ago
>These are primarily focused on machine learning training where you do backpropagation through huge matrix operations How is this different from normal matrix operations?
- joseph_grobbles 6y ago
  Scale, and only scale. As with most extensions, there's negative value if you're doing a small number of calcs, but it pays off in larger needs like ML training. We've been calculating matrixes for time eternal, but AMX has just recently become a thing. Intel just introduced their own AMX instructions.
  - touisteur 6y ago
    I don't know, if you want to perform DL inference on 4k/8k video real-time you're gonna need some heavy duty matrix multiplication resources. GPU is great for batched inferences but for quick no-pcie-transfer, small-to-no-batching inferences you want something close to the CPU...

ChuckMcM 6y ago

Like Intel, Apple has chosen to add some instructions to the processor which accelerate certain operations (in this case matrix operations). Unlike Intel, Apple has not yet chosen to actually document the M1 architecture. This is where a vertically integrated company like Apple (or IBM back in the day) can leverage "inside knowledge" about their chips to achieve performance that is not readily comparable (or reproducible) on a different instruction set architecture (ISA). Historically (see IBM vs Memorex) obscuring your interfaces has been a losing proposition in the long term, even while delivering favorable margins in the short term. Unlike Intel which needs to get third parties to write software for their chips, Apple writes their own software and so it means they can hold on to their advantage while third parties (like the author of the post) reverse engineer what is going on. It will be "historically significant" if Intel adds a feature to their chips in order to stay competitive with this development. The last time they did that was when AMD introduced Opteron. All in all, its basically just a puzzle that people who are interested in how things work get to work on in their spare time.

watersb 6y ago
"IBM vs Memorex": Memorex created a disk drive that was plug compatible with IBM mainframes. IBM of course refused to support any system that dared to use these drives. And sued Memorex for copying their design. Memorex counter-sued. This all forced the court to rule that reverse engineering was permissible. https://www.computerhistory.org/storageengine/memorex-introduces-an-ibm-compatible-hdd/ https://www.computerhistory.org/storageengine/memorex-introd...
- spullara 6y ago
  Is it [IBM] or is it Memorex?
kijiki 6y ago
> It will be "historically significant" if Intel adds a feature to their chips in order to stay competitive with this development. TFA points out that this Apple AMX is similar to the existing Intel AMX, even (confusingly) using the same name.
6y ago

ytch 6y ago

Theoretically we can find out why M1 is fast by instructions. But there is no instruction manual, we can only reverse engineering it