It is unlikely that this is related to the much more purpose-built and conventionally-designed "neural engine" in the M1 (as they are dubbing it) which operates

It is unlikely that this is related to the much more purpose-built and conventionally-designed "neural engine" in the M1 (as they are dubbing it) which operates in the "set it up, feed it, and check the results" type of pipeline you allude to above. These AMX instructions are doing similar matrix operations (tile-based) but not at the same scale as dedicated hardware. Think of it like L1 cache for neural-nets - if you can fit your model into that on-die tile, it can happen there and be much quicker (and also leverage data already in cache or memory) than dispatching it to a separate device which is orders of magnitude higher latency. The tradeoff is not likely perceptible to developers since the compiler or runtime will decide if the net is small enough to run locally or whether it should be dispatched to the dedicated device (again, like L1 access for the majority of developers not writing inline assembly) - thus why it's not exposed. They (apple) could also do some very interesting things with it internally such as neural branch prediction and cache eviction based on contextual operations - again outside of the scope of what a developer would have access to. Intel is doing the same in their upcoming x86 silicon with Intel Advanced Matrix Extension - I would expect this ISA to be heavily inspired by that.