4 ms·
Olamma currently has only one "supported backend" which is llama.cpp. It enables downloading and running models on CPU. And might have more mature server. This
by jerrygenser 2y ago
Olamma currently has only one "supported backend" which is llama.cpp. It enables downloading and running models on CPU. And might have more mature server.
This allows running models on GPU as well.
- darkteflon 2y agoOllama runs on GPUs just fine - on Macs, at least.
- Zambyte 2y agoI have been running Ollama on AMD GPUs (which support for came after NVIDIA GPUs) since February. Llama.cpp has supported it even longer.
- tarruda 2y agoHow well does it run in AMD GPUs these days compared to Nvidia or Apple silicon? I've been considering buying one of those powerful Ryzen mini PCs to use as an LLM server in my LAN, but I've read before that the AMD backend (ROCm IIRC) is kinda buggy
- RealStickman_ 2y agoNot sure about Ollama, but llama.cpp supports vulkan for GPU computing.
- SushiHippie 2y agoI have an RTX 7900 XTX and never had AMD specific issues, except that I needed to set some environment variable. But it seems like integrated GPUs are not supported https://github.com/ollama/ollama/issues/2637 https://github.com/ollama/ollama/issues/2637
- deleted 2y ago[deleted]