3 ms·
Llama.cpp works well on AMD, even for really outdated GPUs. Ollama refuses to work with my RX 570 from 2019 but llama.cpp supports it via Vulkan.
by _lvbh 1y ago
Llama.cpp works well on AMD, even for really outdated GPUs. Ollama refuses to work with my RX 570 from 2019 but llama.cpp supports it via Vulkan.
- Havoc 1y ago>Ollama refuses to work with my RX 570 from 2019 but llama.cpp supports it via Vulkan. That's a bit odd given ollama utilizing llama to do the inference...
- LorenDB 1y agoSee recent discussion about this very topic: https://news.ycombinator.com/item?id=42886680 https://news.ycombinator.com/item?id=42886680
- DiabloD3 1y agoIt isn't odd at all. Ollama uses an ancient version of llama.cpp, and was originally meant to just be a GUI frontend. They forked, and then never resynchronized... and now lack the willpower and technical skill to achieve that. Ollama is essentially a dead, yet semi-popular, project with a really good PR team. If you really want to do it right, you use llama.cpp.
- washadjeffmad 1y agoDon't you dare say anything unpositive about Ollama this close to whatever it is they're planning to distinguish themselves from llama.cpp. They've been out hustling, handshaking, dealmaking, and big businessing their butts off, whether or not they clearly indicate the shoulders of the titans like Georgi Gerganov they're wrapping, and you are NO ONE to stand in their way. Do NOT blow this for them. Understand? They've scooted under the radar successfully this far, and they will absolutely lose their shit if one more peon shrugs at how little they contribute upstream for what they've taken that could have gone to supporting their originator. Ollama supports its own implementation of ggml, btw. gglm is a mysterious format that no one knows the origins of, which is all the more reason to support Ollama, imo.
- DiabloD3 1y agoMan, best /s text I've seen on here in awhile. I hope other people appreciate it.