4 ms·

Does Ollama offer a GUI? I don't think they do. I use them because they run as a systemd service with a convenient HTTP API. That's been extremely helpful for

by gavmor 2y ago

Does Ollama offer a GUI? I don't think they do. I use them because they run as a systemd service with a convenient HTTP API. That's been extremely helpful for switching between GUIs. I also like their model orgazation scheme, and the modelfile paradigm. It's also really handy that it loads and unloads models when called, which is handy for experimentation and some complex workflows eg embedding followed by inference. Is llama.cpp doing 100% of the "heavy lifting"? Sure, but some light lifting is also needed to lower the activation threshold and bring the value to life. I would not use llama.cpp, it's simply too cumbersome. If Ollama did not exist, I would have to invent it. Is it not "innovative"? Who cares! I want it. Commodity GUI? Again, I don't think they have a GUI at all. Are you maybe thinking of OpenWebUI?

DrPhish 2y ago
I think we agree on almost all points, but I thought ollama-gui was an official gui, so I’m even more baffled as to what the draw is. Running as a llama.cpp as a service/API endpoint is trivial (I do just that). Maybe you can outline for me what the value proposition of ollama is so I can better understand what it does that plain llama.cpp doesn’t.
- disgruntledphd2 2y ago
  My experience of ollama is that it makes it super easy to pull various models and use them locally. Sure, I could do this myself but it's helpful not to have to.
- FireInsight 2y ago
  Ollama allows me to use a single podman command, which uses the latest version of ollama, downloads a model of my choosing, and starts a local http endpoint widely supported by different clients. I can just run this one command to chat with a local model through a web interface, get code completions in VSCode, ask about the content of my local Markdown notes. Now, I don't use AI that much, I could totally live without this. But if it weren't for the robust one-liner I probably wouldn't use local LLMs at all.

by gavmor 2y ago

DrPhish 2y ago

I think we agree on almost all points, but I thought ollama-gui was an official gui, so I’m even more baffled as to what the draw is. Running as a llama.cpp as a service/API endpoint is trivial (I do just that). Maybe you can outline for me what the value proposition of ollama is so I can better understand what it does that plain llama.cpp doesn’t.

disgruntledphd2 2y ago
My experience of ollama is that it makes it super easy to pull various models and use them locally. Sure, I could do this myself but it's helpful not to have to.
FireInsight 2y ago
Ollama allows me to use a single podman command, which uses the latest version of ollama, downloads a model of my choosing, and starts a local http endpoint widely supported by different clients. I can just run this one command to chat with a local model through a web interface, get code completions in VSCode, ask about the content of my local Markdown notes. Now, I don't use AI that much, I could totally live without this. But if it weren't for the robust one-liner I probably wouldn't use local LLMs at all.