3 ms·

LMStudio is not open source though, ollama is but people should use llama.cpp instead

behnamoh 10mo ago
> LMStudio is not open source though, ollama is and why should that affect usage? it's not like ollama users fork the repo before installing it.
- thehamkercat 10mo ago
  It was worth mentioning.
smcleod 10mo ago
I suspect Ollama is at least partly moving away open source as they look to raise capitol, when they released their replacement desktop app they did so as closed source. You're absolutely right that people should be using llama.cpp - not only is it truly open source but it's significantly faster, has better model support, many more features, better maintained and the development community is far more active.
- parthsareen 10mo ago
  Desktop app is open-source now.
- calgoo 10mo ago
  Only issue I have found with llama.cpp is trying to get it working with my amd GPU. Ollama almost works out of the box, in docker and directly on my Linux box.
  - Lapel2742 10mo ago
    >Only issue I have found with llama.cpp is trying to get it working with my amd GPU. I had no problems with ROCm 6.x but couldn't get it to run with ROCm 7.x. I switched to Vulkan and the performance seems ok for my use cases
nateb2022 10mo ago
> but people should use llama.cpp instead MLX is a lot more performant than Ollama and llama.cpp on Apple Silicon, comparing both peak memory usage + tok/s output. edit: LM Studio benefits from MLX optimizations when running MLX compatible models.
Abishek_Muthian 10mo ago
Besides optimizations specific to running locally lands in lamma.cpp first.
ekianjo 10mo ago
Ollama did not open source their GUI.
- jmorgan 10mo ago
  The source is available here: https://github.com/ollama/ollama/tree/main/app https://github.com/ollama/ollama/tree/main/app
  - ekianjo 10mo ago
    Thanks, I stand corrected.
skhameneh 10mo ago
ik_llama is almost always faster when tuned. However, when untuned I've found them to be very similar in performance with varied results as to which will perform better. But vLLM and Sglang tend to be faster than both of those.
DavideNL 10mo ago
Note that there's also "LlamaBarn" (macOS app): https://github.com/ggml-org/LlamaBarn https://github.com/ggml-org/LlamaBarn

> LMStudio is not open source though, ollama is and why should that affect usage? it's not like ollama users fork the repo before installing it.

thehamkercat 10mo ago
It was worth mentioning.

smcleod 10mo ago

I suspect Ollama is at least partly moving away open source as they look to raise capitol, when they released their replacement desktop app they did so as closed source. You're absolutely right that people should be using llama.cpp - not only is it truly open source but it's significantly faster, has better model support, many more features, better maintained and the development community is far more active.

parthsareen 10mo ago
Desktop app is open-source now.
calgoo 10mo ago
Only issue I have found with llama.cpp is trying to get it working with my amd GPU. Ollama almost works out of the box, in docker and directly on my Linux box.
- Lapel2742 10mo ago
  >Only issue I have found with llama.cpp is trying to get it working with my amd GPU. I had no problems with ROCm 6.x but couldn't get it to run with ROCm 7.x. I switched to Vulkan and the performance seems ok for my use cases

nateb2022 10mo ago

> but people should use llama.cpp instead MLX is a lot more performant than Ollama and llama.cpp on Apple Silicon, comparing both peak memory usage + tok/s output. edit: LM Studio benefits from MLX optimizations when running MLX compatible models.

Abishek_Muthian 10mo ago

Besides optimizations specific to running locally lands in lamma.cpp first.

ekianjo 10mo ago

Ollama did not open source their GUI.

jmorgan 10mo ago
The source is available here: https://github.com/ollama/ollama/tree/main/app https://github.com/ollama/ollama/tree/main/app
- ekianjo 10mo ago
  Thanks, I stand corrected.

skhameneh 10mo ago

ik_llama is almost always faster when tuned. However, when untuned I've found them to be very similar in performance with varied results as to which will perform better. But vLLM and Sglang tend to be faster than both of those.

DavideNL 10mo ago

Note that there's also "LlamaBarn" (macOS app): https://github.com/ggml-org/LlamaBarn https://github.com/ggml-org/LlamaBarn