3 ms·
Is this using only llama.cpp as inference engine? How is this days support there on NPU and GPU? Not sure if LLM can run on NPU but many models like STT and TTS
by pzo 1y ago
Is this using only llama.cpp as inference engine? How is this days support there on NPU and GPU? Not sure if LLM can run on NPU but many models like STT and TTS and vision often can run much faster on Apple NPU
- liuliu 1y agoYou don't need to guess: https://github.com/cactus-compute/cactus/tree/main/cpp https://github.com/cactus-compute/cactus/tree/main/cpp