3 ms·

This story talks about MLX and Ollama but doesn't mention LM Studio - https://lmstudio.ai/ https://lmstudio.ai/ LM Studio can run both MLX and GGUF models but

• ZeroCool2u 11mo ago
  LMStudio is so much better than Ollama it's silly it's not more popular.
  • thehamkercat 11mo ago
    LMStudio is not open source though, ollama is but people should use llama.cpp instead
    • behnamoh 11mo ago
      > LMStudio is not open source though, ollama is and why should that affect usage? it's not like ollama users fork the repo before installing it.
      • thehamkercat 11mo ago
        It was worth mentioning.
    • smcleod 11mo ago
      I suspect Ollama is at least partly moving away open source as they look to raise capitol, when they released their replacement desktop app they did so as closed source. You're absolutely right that people should be using llama.cpp - not only is it truly open source but it's significantly faster, has better model support, many more features, better maintained and the development community is far more active.
      • parthsareen 11mo ago
        Desktop app is open-source now.
      • calgoo 11mo ago
        Only issue I have found with llama.cpp is trying to get it working with my amd GPU. Ollama almost works out of the box, in docker and directly on my Linux box.
        • Lapel2742 11mo ago
          >Only issue I have found with llama.cpp is trying to get it working with my amd GPU. I had no problems with ROCm 6.x but couldn't get it to run with ROCm 7.x. I switched to Vulkan and the performance seems ok for my use cases
    • nateb2022 11mo ago
      > but people should use llama.cpp instead MLX is a lot more performant than Ollama and llama.cpp on Apple Silicon, comparing both peak memory usage + tok/s output. edit: LM Studio benefits from MLX optimizations when running MLX compatible models.
    • Abishek_Muthian 11mo ago
      Besides optimizations specific to running locally lands in lamma.cpp first.
    • ekianjo 11mo ago
      Ollama did not open source their GUI.
      • jmorgan 11mo ago
        The source is available here: https://github.com/ollama/ollama/tree/main/app https://github.com/ollama/ollama/tree/main/app
        • ekianjo 11mo ago
          Thanks, I stand corrected.
    • skhameneh 11mo ago
      ik_llama is almost always faster when tuned. However, when untuned I've found them to be very similar in performance with varied results as to which will perform better. But vLLM and Sglang tend to be faster than both of those.
    • DavideNL 11mo ago
      Note that there's also "LlamaBarn" (macOS app): https://github.com/ggml-org/LlamaBarn https://github.com/ggml-org/LlamaBarn
• midius 11mo ago
  Makes me think it's a sponsored post.
  • Cadwhisker 11mo ago
    LMStudio? No, it's the easiest way to run am LLM locally that I've seen to the point where I've stopped looking at other alternatives. It's cross-platform (Win/Mac/Linux), detects the most appropriate GPU in your system and tells you whether the model you want to download will run within it's RAM footprint. It lets you set up a local server that you can access through API calls as if you were remotely connected to an online service.
    • vunderba 11mo ago
      FWIW, Ollama already does most of this: - Cross-platform - Sets up a local API server The tradeoff is a somewhat higher learning curve, since you need to manually browse the model library and choose the model/quantization that best fit your workflow and hardware. OTOH, it's also open-source unlike LMStudio which is proprietary.
      • randallsquared 11mo ago
        I assumed from the name that it only ran llama-derived models, rather than whatever is available at huggingface. Is that not the case?
        • fenykep 11mo ago
          No, they have quite a broad list of models: https://ollama.com/search https://ollama.com/search [edit] Oh and apparently you can also directly run some models directly from HuggingFace: https://huggingface.co/docs/hub/ollama https://huggingface.co/docs/hub/ollama
          • ashirviskas 11mo ago
            Just use llama.cpp. Ollama tried to force their custom API (not the openai standard), they obscure the downloaded models making them a pain to use with other implementations, blatantly used llama.cpp as a thin wrapper without communicating it properly and now has to differentiate somehow to start making money. If you've ever used a terminal, use llama.cpp. You can also directly run models from llama.cpp afaik.
• thehamkercat 11mo ago
  I think you should mention that LM Studio isn't open source. I mean, what's the point of using local models if you can't trust the app itself?
  • satvikpendem 11mo ago
    Depends what people use them for, not every user of local models is doing so for privacy, some just don't like paying for online models.
    • thehamkercat 11mo ago
      Most LLM sites are now offering free plans, and they are usually better than what you can run locally, So I think people are running local models for privacy 99% of the time
  • behnamoh 11mo ago
    > I mean, what's the point of using local models if you can't trust the app itself? and you think ollama doesn't do telemetry/etc. just because it's open source?
    • thehamkercat 11mo ago
      That's why i suggested using llama.cpp in my other comment.
    • parthsareen 11mo ago
      You're welcome to go through the source: https://github.com/ollama/ollama/ https://github.com/ollama/ollama/
  • rubymamis 11mo ago
    You can always use something like Little Snitch to not allow it to dial home.
• evacchi 11mo ago
  ramalama.ai is worth mentioning too
• ekianjo 11mo ago
  Lmstudio runs llama.cpp under the hood.
  • selcuka 11mo ago
    They also run the Apple MLX engine on macOS.