4 ms·

It's very hard to put into words without coming off as being unfair to one side or the other, but the ollama project really does provide little-to-no _innovativ

It's very hard to put into words without coming off as being unfair to one side or the other, but the ollama project really does provide little-to-no _innovative_ value over simply running components of llama.cpp directly from the command line. 100% of the heavy lifting (from an LLM perspective) is in the llama.cpp codebase. The ollama parts are all simple, well understood, commodity components that most any developer could have produced. Now, applications like ollama obviously need to exist, as not everyone can run CLI utilities, let alone clone a git repo and compile themselves. Easy to use GUIs are essential for the adoption of new tech (much like how there are many apps that wrap ffmpeg and are mostly UI). However, if ollama are mostly doing commodity GUI things over a fully fleshed-out, _unique_ codebase to which their very existence is owed, they should do everything in their power to point that out. I'm sure they're legally within their rights because of the licensing, but just from an ethical perspective. I think there is a lot of ill-will towards ollama in some hard-core OG LLM communities because ollama appears to be attempting to capture the value that ggerganov has provided to the world in this tool without adequate attribution (although there is a small footnote, iirc). Basically, the debt that ollama owes to llama.cpp is so immense that they need to do a much better job recognizing it imo.

gavmor 2y ago
Does Ollama offer a GUI? I don't think they do. I use them because they run as a systemd service with a convenient HTTP API. That's been extremely helpful for switching between GUIs. I also like their model orgazation scheme, and the modelfile paradigm. It's also really handy that it loads and unloads models when called, which is handy for experimentation and some complex workflows eg embedding followed by inference. Is llama.cpp doing 100% of the "heavy lifting"? Sure, but some light lifting is also needed to lower the activation threshold and bring the value to life. I would not use llama.cpp, it's simply too cumbersome. If Ollama did not exist, I would have to invent it. Is it not "innovative"? Who cares! I want it. Commodity GUI? Again, I don't think they have a GUI at all. Are you maybe thinking of OpenWebUI?
- DrPhish 2y ago
  I think we agree on almost all points, but I thought ollama-gui was an official gui, so I’m even more baffled as to what the draw is. Running as a llama.cpp as a service/API endpoint is trivial (I do just that). Maybe you can outline for me what the value proposition of ollama is so I can better understand what it does that plain llama.cpp doesn’t.
  - disgruntledphd2 2y ago
    My experience of ollama is that it makes it super easy to pull various models and use them locally. Sure, I could do this myself but it's helpful not to have to.
  - FireInsight 2y ago
    Ollama allows me to use a single podman command, which uses the latest version of ollama, downloads a model of my choosing, and starts a local http endpoint widely supported by different clients. I can just run this one command to chat with a local model through a web interface, get code completions in VSCode, ask about the content of my local Markdown notes. Now, I don't use AI that much, I could totally live without this. But if it weren't for the robust one-liner I probably wouldn't use local LLMs at all.
CMay 2y ago
I'm not going to pretend to know everything about Ollama, llama.cpp or llamafile, but my experiences using llama.cpp and llamafile (llama.cpp based) were both negative. Web UI frontends aren't relevant here, this is just purely about whether I can load a model and get it to produce coherent results that are in the realm of what the people who created the model intended. With llama.cpp or llamafile, I was constantly having to look up a model's paper, documentation or other pages to see what the recommended parameters were, recommended templates were, and so on. My understanding is that GGUFs were supposed to solve that, yet still I was getting poor results. You know, I don't know all the details or if there's any difference between what Modelfiles are for versus what GGUF metadata is for, but my experiences with Ollama have been that it just worked. It took me a while to even try Ollama, because the expectation is that it would simply be another interface on top of the same issues. There are things I don't like about Ollama, but mostly they were easy to work around by writing a few scripts. Not using any web UI with it at all.

gavmor 2y ago

Does Ollama offer a GUI? I don't think they do. I use them because they run as a systemd service with a convenient HTTP API. That's been extremely helpful for switching between GUIs. I also like their model orgazation scheme, and the modelfile paradigm. It's also really handy that it loads and unloads models when called, which is handy for experimentation and some complex workflows eg embedding followed by inference. Is llama.cpp doing 100% of the "heavy lifting"? Sure, but some light lifting is also needed to lower the activation threshold and bring the value to life. I would not use llama.cpp, it's simply too cumbersome. If Ollama did not exist, I would have to invent it. Is it not "innovative"? Who cares! I want it. Commodity GUI? Again, I don't think they have a GUI at all. Are you maybe thinking of OpenWebUI?

DrPhish 2y ago
I think we agree on almost all points, but I thought ollama-gui was an official gui, so I’m even more baffled as to what the draw is. Running as a llama.cpp as a service/API endpoint is trivial (I do just that). Maybe you can outline for me what the value proposition of ollama is so I can better understand what it does that plain llama.cpp doesn’t.
- disgruntledphd2 2y ago
  My experience of ollama is that it makes it super easy to pull various models and use them locally. Sure, I could do this myself but it's helpful not to have to.
- FireInsight 2y ago
  Ollama allows me to use a single podman command, which uses the latest version of ollama, downloads a model of my choosing, and starts a local http endpoint widely supported by different clients. I can just run this one command to chat with a local model through a web interface, get code completions in VSCode, ask about the content of my local Markdown notes. Now, I don't use AI that much, I could totally live without this. But if it weren't for the robust one-liner I probably wouldn't use local LLMs at all.

CMay 2y ago

I'm not going to pretend to know everything about Ollama, llama.cpp or llamafile, but my experiences using llama.cpp and llamafile (llama.cpp based) were both negative. Web UI frontends aren't relevant here, this is just purely about whether I can load a model and get it to produce coherent results that are in the realm of what the people who created the model intended. With llama.cpp or llamafile, I was constantly having to look up a model's paper, documentation or other pages to see what the recommended parameters were, recommended templates were, and so on. My understanding is that GGUFs were supposed to solve that, yet still I was getting poor results. You know, I don't know all the details or if there's any difference between what Modelfiles are for versus what GGUF metadata is for, but my experiences with Ollama have been that it just worked. It took me a while to even try Ollama, because the expectation is that it would simply be another interface on top of the same issues. There are things I don't like about Ollama, but mostly they were easy to work around by writing a few scripts. Not using any web UI with it at all.