I would guess that the ANE has some very specialised hardware (E.g. INT8 or FP16 only), and there’s isn’t a huge amount of it. So many nets either don’t fit com

I would guess that the ANE has some very specialised hardware (E.g. INT8 or FP16 only), and there’s isn’t a huge amount of it. So many nets either don’t fit completely, or aren’t using the right types of operation to match the ANE. Either can’t run on the ANE, or only have a subset of layers that can run on the ANE. So when running a neural net iOS / macOS needs to make a decision about where to run each net. Even if a net has layers in it that are a perfect match for the ANE, there’s still a trade off from having move the workload back and forth between a CPU core and the ANE (although the unified memory should eliminate a big chunk of this cost). It might be that in the general case it’s not worth the latency hit from using mixed processors when running net that isn’t 100% ANE compatible, or could just be that Apple haven’t got round to implementing the logic needed to gracefully spilt workloads across the ANE and a CPU core. Which would make sense, because they’ve got the time and expertise to ensure all their nets fit within the ANE. Something that’s difficult for 3rd party devs to do, because they don’t have access to detailed ANE docs.