5 ms·
Ollama is a model-management app that runs on top of llama.cpp so you should ask there about AMD support.
by visarga 3y ago
Ollama is a model-management app that runs on top of llama.cpp so you should ask there about AMD support.
- progman32 3y agoI've been running llama.cpp with full GPU acceleration on my AMD card, using the text-generation-webui install script on kubuntu. Same with stable diffusion using a1111. AMD's compute stack is indeed quite broken and is more fragile, but it does work using most modern cards. The kernel panics though... Yeah, I had those on my Radeon vii before I upgraded.
- 65a 3y agollama.cpp has had ROCm support for a long time