4 ms·
Actually that's a really good question, I hadn't considered that the comparison here is just CPU vs using Metal (CPU+GPU). To answer the question though - I th
by kamranjon 1y ago
Actually that's a really good question, I hadn't considered that the comparison here is just CPU vs using Metal (CPU+GPU).
To answer the question though - I think this would be used for cases where you are building an app that wants to utilize a small AI model while at the same time having the GPU free to do graphics related things, which I'm guessing is why Apple stuck these into their hardware in the first place.
Here is an interesting comparison between the two from a whisper.cpp thread - ignoring startup times - the CPU+ANE seems about on par with CPU+GPU: https://github.com/ggml-org/whisper.cpp/pull/566#issuecomment-1725106230 https://github.com/ggml-org/whisper.cpp/pull/566#issuecommen...
- conradev 1y agoIt essentially never makes sense to run on the CPU and you will only ever see enthusiasts doing it. Yes, hammering the GPU too hard can affect the display server, but no, switching to the CPU is not a good alternative
- kamranjon 1y agoNot switching to the CPU - switching to the ANE (Neural Cores) - if you read the research papers Apple has released - the example I gave is pretty much how it's being used - small image classification models running on the ANE, alongside a graphics app that needs the GPU to be free.
- conradev 1y agoOh, yes, I misread! It’s great for that