There are two important sets of ML workloads: training and inference. This Intel blog post does a reasonably good job summarizing the differences: https://www.i

There are two important sets of ML workloads: training and inference. This Intel blog post does a reasonably good job summarizing the differences: https://www.intel.com/content/www/us/en/artificial-intelligence/posts/deep-learning-training-and-inference.html https://www.intel.com/content/www/us/en/artificial-intellige.... The hardware requirements for training are much higher than for inference. Roughly: - Training: Requires massive throughput and compute, so is done on high-end GPUs or TPUs in datacenters (or on researchers' beefy workstations) - Inference: Compute requirement depends on the size and complexity of the model, but many models can be run on low-end smartphone hardware. Generally speaking, any time you see "ML hardware" in a consumer device, that hardware is for inference, not training. Apple's Neural Engine is an example, of this. Other current examples include the ARM Ethos-N series and Qualcomm AI Engine, among others. Often this hardware is designed to balance inference performance with power consumption, so that inference can be done efficiently on battery-powered devices. Training workloads on the M1 use the GPU, not the Neural Engine: https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html https://blog.tensorflow.org/2020/11/accelerating-tensorflow-...