4 ms·
Does anybody know how to run this on Linux with an AMD GPU? Also do I have to bother with their crappy driver module that doesn't support most GPUs?
by jacooper 4y ago
Does anybody know how to run this on Linux with an AMD GPU?
Also do I have to bother with their crappy driver module that doesn't support most GPUs?
- Const-me 4y agollama.cpp doesn't need nor use GPU. You can run on Linux, but you gonna need a CPU with AVX2 support.
- heap_perms 4y agogrep -o avx2 /proc/cpuinfo | uniq If your CPU supports AVX2, the output will be "avx2". If your CPU does not support AVX2, the command will not produce any output.
- nextaccountic 4y agoBut if you have a GPU available, it can't use it? Why not?
- Const-me 4y agoIt can not, that particular library is CPU-only. If you want to run on GPU, use the official Python-based stack, which BTW takes about 10GB of runtime binaries on disk, and only supports nVidia GPUs because CUDA.
- kid64 4y agoDo you have a link for that?
- Const-me 4y agoSource code: https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp
- kid64 4y agoMy apologies @Const-me, are you referring to the module "convert-pth-to-ggml.py" in that repo? That appears to be the only Python in there.
- hnfong 4y agoThe link from GP is the CPU only one implemented in C++. The python + GPU one can be found on the official facebook repo: https://github.com/facebookresearch/llama https://github.com/facebookresearch/llama (Presumably GP thought this was already known to everyone so they pasted the other link)
- Const-me 4y agoIt seems that question can be interpreted in two ways: 1. Do you have a link which proves the implementation being discussed is CPU-only? 2. Do you have a link to the official Python-based implementation? When I wrote the above answer, I was only aware about the first interpretation.