3 ms·
Have you tried ollama? It’s another llama.cpp server implementation that is becoming popular.
by speedgoose 3y ago
Have you tried ollama? It’s another llama.cpp server implementation that is becoming popular.
- gbickford 3y agoThe authors don't seem to care about the principle of least privilege: https://github.com/ollama/ollama/issues/851#issuecomment-1779885366 https://github.com/ollama/ollama/issues/851#issuecomment-177... It makes me wonder what other security issues they might now care about.
- smoothjazz 3y agoThis is a mac problem, not an ollama problem. It also sounds like it's solved by using homebrew (or linux).
- brucethemoose2 3y ago(afaik) it does not support batching, and some other server focused features. I was talking more about a high throughput server, which is not appropriate for ollama or llama.cpp in general.