3 ms·
Any more details? I'm guessing they're leveraging the NPU in the pixel?
by Havoc 4y ago
Any more details? I'm guessing they're leveraging the NPU in the pixel?
- saidinesh5 4y agoI think they are using llama.cpp without any NPU/TPU patches. By default it only runs on CPU with support for various SIMD extensions. https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp
- zodester 4y agoIt uses the ARM NEON extensions to the instruction set for SIMD (as far as I understand).