4 ms·
might try it first. seems to be only CPU?
by quickthrower2 3y ago
might try it first. seems to be only CPU?
- themulticaster 3y agoI'd see that as a benefit of llama.cpp - it's specifically designed to be usable on consumer hardware such as laptops, without professional GPUs.
- azeirah 3y agoIt has partial gpu acceleration if you compile it with LLAMA_CUBLAS or LLAMA_CLBLAST They really have come a long way since... A few weeks ago. Using cublas with my 1080ti results in a 52% speedup compared to cpu-only. Vram usage is very minimal.