16 ms·
Undocumented arm64 ISA extension present on the Apple M1
- lwhi 6y agoI don't understand the significance of this. Could someone who does add some context?
- ericlewis 6y agoFrom what I can tell reading this: the Accelerate framework from Apple seems to use custom instructions possibly unique to the M1 chip but potentially similar to Intel AMX. Accelerate is, as I sort of understand it- a way to do advanced math in a very quick / power efficient way. It is touted as deeply integrated with Apple processors from the day it launched and (I think) even recommended to be used over SIMD and the sort (I could certainly be very wrong about this). It dove tails with some of the machine learning work Apple did as well, is portable among Apple systems, and can be assumed to be optimized. Ninja edit: I’ve not seen how it was achieved until now, but a custom ISA extension isn’t wholly surprising. Edit 2: as for why this is significant.. I’m not sure it is. It is interesting to know how Accelerate can pull off what it does though. IIRC there are other machine learning frameworks which take advantage of the neural engine- but Accelerate possibly doesn’t- despite having machine learning capabilities.
- Lammy 6y agoIt also means that any binaries built to use those instructions won't run on any non-Apple ARM64 CPUs, at least not easily. Sort of like how early-era Intel Hackintosh users had to have CPUs with SSE3 support or wait/hope for SSE2 patches.
- ericlewis 6y agoI’m relatively sure that Accelerate does work fine on intel processors and is still the recommended means of doing heavy math. But that could just be because they knew of this eventual future. I’m not sure what Accelerate does under the hood for intel machines. But I would assume perhaps falls back to whatever intel might support.
- giantrobot 6y agoSay hello to Function Multi-versioning [0]. You tell the compiler to build a version for each march defined in the attribute. When that function is called a quick CPU check (via CPUID) is performed and if there's a version specific to the current CPU that will run instead of the more general implementation. ICC has supported this for a long time and it's widely used in Intel's MKL. LLVM also supports FMV but uses a slightly different method to define it in code. IIRC there was an LLVM issue tracking work to support GCC's way of declaring FMV but I don't know if that landed yet. Since the function's signature is the same between the multiple implementations the caller doesn't need to change of their code to get a free boost from features like AVX2 or these M1 instructions. When an M2 comes out with better vector instructions or something and a new Accelerate framework is released, anyone using the framework get a free performance increase without recompiling anything. [0] https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/
- floatboth 6y agoAt least __attribute__((ifunc(..))) is common between clang and gcc. (E.g. no compiler conditionals are used in https://cgit.freebsd.org/src/tree/sys/arm64/include/ifunc.h https://cgit.freebsd.org/src/tree/sys/arm64/include/ifunc.h) The fancier high level stuff that automatically compiles multiple versions of a function, I'm not sure.
- saagarjha 6y agoNote that on ARM it is more annoying to get CPU capabilities as there is no straight cpuid equivalent.
- stephencanon 6y agoOn Apple platforms, CPU capabilities are cached on the commpage, so system libraries don’t even need to use cpuid on x86.
- jonathonf 6y agoFrom my limited understanding: it looks like a matrix coprocessor, and implies that hardware-accelerated matrix processing has been built into the M1. Matrix processing comes in useful for AI/ML/CV-type applications. It's something GPUs are generally very good for.
- mhh__ 6y agoIn the bigger picture Apple aren't really documenting anything for their new toys, which is not the end of the world but is a huge step back for openness. There is - as far as I can see - no long-form documentation for M1 other than patents. I already refuse to own apple products so I don't really have any skin in the game, but consider that if microsoft were like this the blind eye would not be turned - buy our new product, everything you run is up to us, we won't document it, maybe we'll upstream the compiler, tough shit. The fact that they can be like this is really proof that their market position is much stronger than most HNers seem to think it is.
- MichaelZuo 6y agoThe implication I assume that there would be pushback on Microsoft for doing the same would be because of the possibility of nefarious undocumented changes. What possibly nefarious things could an undocumented ISA extension do?
- mhh__ 6y agoUndocumented It's not about EEE it's about Apple taking this attitude in the first place.
- hyperman1 6y agoPresumably the next iteration of the processor can remove them and use the opcodes for something else. Apple recompiles their system library, hence is unaffected. You can't do the same so peak M1 performance is unavailable for you.
- MichaelZuo 6y agoCould those opcodes then be used for anything nefarious, or would it just restrict third parties from fully utilizing the processor?
- Someone 6y agoIf you think Apple wants to do something nefarious to users of their hardware, don’t buy or use it. If they can’t be trusted, they don’t need to add opcodes.
- vegetablepotpie 6y agoApple added custom instructions on top of the ARM instruction set to do matrix operations. Matrix operations are used a lot in some algorithms, such as in computer graphics and machine learning and these instructions help those operations go faster. Although the ARM ISA has a multiply and accumulate instruction, it would take many more instructions on a standard ARM core to compute a matrix with only standard instructions. That enables the M1 chip to be faster in this case when developers use these instructions in their applications.
- giantrobot 6y ago> That enables the M1 chip to be faster in this case when developers use these instructions in their applications. Or more typically the compiler will emit those instructions and everyone gets them for free.
- teknopaul 6y agoonly if the compiler knows they exist
- a1369209993 6y agoThat's actually generally not true WRT SIMD/vector instructions (see eg http://yosefk.com/blog/humans-and-compilers-need-each-other-the-vliw-simd-case.html http://yosefk.com/blog/humans-and-compilers-need-each-other-... which contrasts this with the more compiler-friendly VLIW case); they impose nontrivial requirements on data structures and ABI, so the application code needs to at least be aware of the possibility of vectorization, even if it's agnostic to whether the implementation actually uses it.
- 2wrist 6y agoSimilar to Altivec from the PowerPC days?
- floatboth 6y agoAltivec is regular SIMD like NEON or SSE/AVX
- joseph_grobbles 6y agoApple added special AMX instructions specifically for matrix operations. They added them back with the A13, and then improved them for the M1. These are primarily focused on machine learning training where you do backpropagation through huge matrix operations. They provide a very wide coverage library set that works on all supported platforms optimally. You interact with AMX through those libraries, and direct usage is considered unsupported. Apple may completely change it with the next iteration, where they can just change those libraries and every arm's-length application will get those benefits for free. Apple sells whole systems. They aren't selling CPUs for third parties, and there is no reason they need to encourage their own proprietary extensions. Indeed, it would be more evil if they were asking people to pepper their own code with this, instead of just using libraries that abstract you from the magic. As a tiny, tiny vendor in the computing space -- as HN often assures us -- I'm not seeing the great evil many are claiming. (indeed, of course this is downvoted but I chuckle that another comment implores that Apple is evil because this will lead to code and binaries that only work on Apple devices. Which is quite literally exactly the opposite of what Apple is doing, which is forcing use through optimized libraries, abstracting from their own proprietary extensions. People just love bitching)
- notretarded 6y ago>These are primarily focused on machine learning training where you do backpropagation through huge matrix operations How is this different from normal matrix operations?
- joseph_grobbles 6y agoScale, and only scale. As with most extensions, there's negative value if you're doing a small number of calcs, but it pays off in larger needs like ML training. We've been calculating matrixes for time eternal, but AMX has just recently become a thing. Intel just introduced their own AMX instructions.
- touisteur 6y agoI don't know, if you want to perform DL inference on 4k/8k video real-time you're gonna need some heavy duty matrix multiplication resources. GPU is great for batched inferences but for quick no-pcie-transfer, small-to-no-batching inferences you want something close to the CPU...
- ChuckMcM 6y agoLike Intel, Apple has chosen to add some instructions to the processor which accelerate certain operations (in this case matrix operations). Unlike Intel, Apple has not yet chosen to actually document the M1 architecture. This is where a vertically integrated company like Apple (or IBM back in the day) can leverage "inside knowledge" about their chips to achieve performance that is not readily comparable (or reproducible) on a different instruction set architecture (ISA). Historically (see IBM vs Memorex) obscuring your interfaces has been a losing proposition in the long term, even while delivering favorable margins in the short term. Unlike Intel which needs to get third parties to write software for their chips, Apple writes their own software and so it means they can hold on to their advantage while third parties (like the author of the post) reverse engineer what is going on. It will be "historically significant" if Intel adds a feature to their chips in order to stay competitive with this development. The last time they did that was when AMD introduced Opteron. All in all, its basically just a puzzle that people who are interested in how things work get to work on in their spare time.
- watersb 6y ago"IBM vs Memorex": Memorex created a disk drive that was plug compatible with IBM mainframes. IBM of course refused to support any system that dared to use these drives. And sued Memorex for copying their design. Memorex counter-sued. This all forced the court to rule that reverse engineering was permissible. https://www.computerhistory.org/storageengine/memorex-introduces-an-ibm-compatible-hdd/ https://www.computerhistory.org/storageengine/memorex-introd...
- spullara 6y agoIs it [IBM] or is it Memorex?
- kijiki 6y ago> It will be "historically significant" if Intel adds a feature to their chips in order to stay competitive with this development. TFA points out that this Apple AMX is similar to the existing Intel AMX, even (confusingly) using the same name.
- 6y ago
- ytch 6y agoTheoretically we can find out why M1 is fast by instructions. But there is no instruction manual, we can only reverse engineering it
- hyperpallium2 6y agoAfter four different CPU ISA's (68000, powerpc, x86, ARM) an Apple ISA is next.
- Teongot 6y agoThere's a tidy mathematical progression in Apple CPU architectures. 6502 (Apple 1,2,3): 1976 68k (Mac): 1984 (+8 years) PPC: 1994 (+10 years) x86: 2006 (+12 years) ARM: 2020 (+14 years) If the trend continues, we will see Apple's next architecture in 2036.
- ithkuil 6y ago2036, Apple Mill
- mhh__ 6y agoWell, the people behind the Mill might not being around in 16 years time. That being said, I would really like to see the Mill make it to Silicon - even if it doesn't work in the end, it's much more interesting to me than RISC-V.
- ithkuil 6y agoPatents will survive their retirement ;-) I assume RISC-V appeal is because it's open and thus allows people to explore (proprietary) implementations while maintaining compatibility at the ISA level. If I understand correctly you cannot do this with Mill either. Their work looks very interesting, but I'm not drawn in writing an fpga implementation or work on a toolchain for it, since I know any success may be impeded by patents
- jecel 6y agoWhat about the Apple IIgs (1986) and its 65C816 processor? I suppose it would be valid to consider it a version of the 6502 if you are not separating 32 and 64 bit PPCs, x86 and ARMs.
- 6y ago
- stephc_int13 6y agoI hate Apple for that. They very often use non-standard tech and we'll end up with Apple proprietary processors instead of the standard stuff x64/ARM64.
- princekolt 6y agoCompetitors can't compete with them even with standard stuff. Good for them to move forward. It's funny how innovation becomes a "bad" thing when a company you don't like innovates "too much".
- stephc_int13 6y agoThis is not about innovation, this is about standards. Like Objective-C instead of C++, Metal instead of Vulkan, Lightning instead of USB-C, and countless proprietary stuff they use to lock down their users and devs inside of their closed garden. This is wrong and detrimental to the whole industry, including users. Apple would have created a shitty internet if they were in charge at the time. To be honest, Microsoft did pretty nasty things during their era, but I think that Apple would have been so much worse.
- princekolt 6y ago* Objective-C first appeared one year before C++ * Metal was launched two years before Vulkan * Lightning was invented two years before USB-C
- mlazos 6y agoI’m thinking this is related to the machine-learning coprocessor. They likely expose this to developers through their ML API and don’t want to have to support this for external customers if it changes. Still this is a great find!
- londons_explore 6y agoOther vendors ML hardware is very much "A far away device over the PCI express bus which you give bunch of work to and later come back and see if its done". Looks like apple has decided to make it very tightly integrated into the CPU. It means while doing ML operations, the CPU core can't go off and do other useful work. It also forces all cores to require matching ML hardware (or have a lot of OS complexity as certain threads can only run on some cores). The benefit is latency to get ML stuff done is lowered from microseconds to nanoseconds. I suspect Apple probably messed up with that tradeoff... Very few applications can't wait a few microseconds for results of some ML computation...
- tgtweak 6y agoIt is unlikely that this is related to the much more purpose-built and conventionally-designed "neural engine" in the M1 (as they are dubbing it) which operates in the "set it up, feed it, and check the results" type of pipeline you allude to above. These AMX instructions are doing similar matrix operations (tile-based) but not at the same scale as dedicated hardware. Think of it like L1 cache for neural-nets - if you can fit your model into that on-die tile, it can happen there and be much quicker (and also leverage data already in cache or memory) than dispatching it to a separate device which is orders of magnitude higher latency. The tradeoff is not likely perceptible to developers since the compiler or runtime will decide if the net is small enough to run locally or whether it should be dispatched to the dedicated device (again, like L1 access for the majority of developers not writing inline assembly) - thus why it's not exposed. They (apple) could also do some very interesting things with it internally such as neural branch prediction and cache eviction based on contextual operations - again outside of the scope of what a developer would have access to. Intel is doing the same in their upcoming x86 silicon with Intel Advanced Matrix Extension - I would expect this ISA to be heavily inspired by that.
- lukeh 6y agoI see undocumented instruction extensions as a special case of private API: it buys Apple the freedom to change the underlying implementation in the future. As long as done in a manner that is not anticompetitive, I don't see the problem.
- deleted 6y ago[deleted]
- toxik 6y agoLike private APIs, they ultimately transfer power from consumers to Apple. They surely do have documentation internally, they just don’t want to commit to anything.
- lumost 6y agoit limits Assembler access and tools written against that particular instruction, but does it really create a closed API? Surely apple will release libraries that make use of it internally with appropriate compatibility guarantees. Is it inherently wrong for a hardware vendor to make a high-level C api the interface?
- my123 6y agoAccelerate.framework uses them today, and provides a public interface.
- deleted 6y ago[deleted]
- josefx 6y agoSupport for different OSes might be interesting given that the C libraries mentioned seem to be Apple OSes only. Then the issue of having hidden instructions for high performance computation when CPU vendors have a history of bad performance/security tradeoffs. Other than that some languages have a bad time interacting with C or a high overhead when calling C functions.
- jhayward 6y agoIt is really unfortunate that the very commonly used numerical/scientific packages Numpy and Scipy removed[1] Apple’s Accelerate framework compatibility this year due to ‘errors in the framework’. It looks like Python users won’t have access to this capability from the M1 based Macs for some time. [1] https://github.com/numpy/numpy/issues/15947 https://github.com/numpy/numpy/issues/15947
- lincpa 6y agoApple M1 chip adopts Warehouse/Workshop Model - Warehouse: unified memory - Workshop: CPU, GPU and other cores - Product(or raw material): information, data there’s also a new unified memory architecture that lets the CPU, GPU, and other cores exchange information between one another, and with unified memory, the CPU and GPU can access memory simultaneously rather than copying data between one area and another. Accessing the same pool of memory without the need for copying speeds up information exchange for faster overall performance. reference: 1. Developer Delves Into Reasons Why Apple’s M1 Chip is So Fast, https://www.macrumors.com/2020/11/30/m1-chip-speed-explanation-developer/ https://www.macrumors.com/2020/11/30/m1-chip-speed-explanati... 2. The Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Warehouse/Workshop Model, https://github.com/linpengcheng/PurefunctionPipelineDataflow https://github.com/linpengcheng/PurefunctionPipelineDataflow