ML inference is not magic: by and large, it's just a combination of simple operations like matrix multiplications/dot products, element-wise nonlinearities, con

ML inference is not magic: by and large, it's just a combination of simple operations like matrix multiplications/dot products, element-wise nonlinearities, convolutions and other stuff that vector processors, GPUs and increasingly CPUs (thanks to SIMD) are very well optimized for. (In theory one could optimize a chip for some specific, well-defined ML architecture, even to the point of "wiring" the architecture in hardware, and people used to do such things back in the 1980s when this was needed in order to even experiment with e.g. neural network models. But given how fast ML is progressing these days, there's just no reason for doing anything like that nowadays!)