3 ms·
I don’t quite follow why people use ollama ? It sounds like lama.cpp with less features and training wheels Is it just ease of use or is there something I’m mi
by Havoc 3y ago
I don’t quite follow why people use ollama ? It sounds like lama.cpp with less features and training wheels
Is it just ease of use or is there something I’m missing?
- __loam 3y agoIt's always ease of use lol. Thinking the best technology wins is a fallacy.
- sp332 3y agoThe CLI for llama.cpp is very clunky IMO. I put some kind of UI on it when I want to get something done.
- spmurrayzzz 3y agoIt also ships with an openai-compatible server implementation as well now that you could point your UI at (if you wanted to run leaner w/out ollama). https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md https://github.com/ggerganov/llama.cpp/blob/master/examples/...
- titaniumtown 3y agoIt's a wrapper around llama.cpp that provides a stable api
- boarush 3y agoOllama is just easier to use and serve the model on a local http server. I personally use it for testing stuff with llama-index as well. Pretty useful to say the least with zero configuration issues.
- skp1995 3y agonot sure why you are getting downvoted, its a very valid question. Its kind of down to the ergonomics of running LLM. Downloading a user friendly CLI tool with good UX beats having to clone a repo and run make files. llama.cpp is the better option if you want to do anything non-trivial when it comes to LLMs
- mark_l_watson 3y agoI started by using lama.cpp directly, and a few other options. I now just use Ollama because it is simple to download models, keep software and models up to date, and really easy to run a local REST query service. I like spending more time playing with application ideas and less time running infrastructure. Of course, lama.cpp under the hood provides the magic.
- dizhn 3y agoDownloading and activating models is very convenient. This llm stuff is really complicated and every little bit helps at the beginning. I only started two weeks ago and was very frustrated. A tool that just works is good for that kind of thing. Of course at that point you think it's their models and there's something special they are doing to the models etc. Honestly no tool that allows easy downloads goes out of their way to say they are just downloading TheBloke's gguf files and that the same models will run anywhere. (minus ollama's blob format on disk) :)