5 ms·
> the Mac OS build of the GGML plugin uses the Metal API to run the inference workload on M1/M2/M3’s built-in neural processing engines I don't think that's ac
by anentropic 3y ago
> the Mac OS build of the GGML plugin uses the Metal API to run the inference workload on M1/M2/M3’s built-in neural processing engines
I don't think that's accurate (someone please correct me...)
GGML use of Metal API means it runs on the M1/2/3 GPU and not the neural engine
Which is all good, but for sake of being pedantic...
- btown 3y agoNot pedantic at all! https://github.com/ggerganov/llama.cpp/discussions/336 https://github.com/ggerganov/llama.cpp/discussions/336 is a (somewhat rambling) discussion on whether it would even be worthwhile to use the neural engine specifically, beyond the GPU.