3 ms·
What is the difference between Ollama, llama.cpp, ggml and gguf?
by brcmthrowaway 7mo ago
What is the difference between Ollama, llama.cpp, ggml and gguf?
- xiconfjs 7mo agoOllama on MacOS is a one-click solution with stable obe-click updates. Happy so far. But the mlx support was the only missing piece for me.
- yard2010 7mo agoCan you please write about your hardware?
- xiconfjs 7mo ago* macOS 26.x on MacBookPro M1 Max 32GB * Ollama on macOS, cursor to play around * Open WebUI [1] on my Homeserver via API to Ollama (also for remote „A.I.“ access) * running gpt-oss:20b, qwen3.5:9b with ease, qwen3.5:27b for more complex tasks [1] https://github.com/open-webui/open-webui https://github.com/open-webui/open-webui
- brcmthrowaway 7mo agoSeems complicated. Switch to LMStudio
- xiconfjs 7mo agoI tried man times but at least with its API active, LMStudio has some kind of memory leaks which will slow down the whole system (after ~1-2 days of uptime) even after unloading the model and stopping LMStudio up to a point where even playing a 1080p video results in frame drops. No such issues with Ollama.
- benob 7mo agoOllama is a user-friendly UI for LLM inference. It is powered by llama.cpp (or a fork of it) which is more power-user oriented and requires command-line wrangling. GGML is the math library behind llama.cpp and GGUF is the associated file format used for storing LLM weights.
- redmalang 7mo agoi've found llama.cpp (as i understand it, ollama now uses their own version of this) to work much better in practice, faster and much more flexible.