3 ms·
Does llama.cpp use the npu or metal? I thought it uses NEON instructions? So currently those units aren't even used.
by kristianp 4y ago
Does llama.cpp use the npu or metal? I thought it uses NEON instructions? So currently those units aren't even used.
- brucethemoose2 4y agoI don't think so. M chips just happen to have a really good memory subsystem and good simd performance through accelerate, so the CPU performance is pretty good. Some stable diffusion implementations can use the NPU or GPU, or (experimentally and unsucessfully) both.