4 ms·

This is the point of it: https://github.com/ggerganov/llama.cpp/pull/11016#issuecomment-2599740463 https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme

This is the point of it: https://github.com/ggerganov/llama.cpp/pull/11016#issuecomment-2599740463 https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...

buyucu 2y ago
Thanks for this context, I will give RamaLlama a try!
ericyd 2y ago
I wish this were on the readme. Or if it already is, I wish it were significantly higher up.
mohsen1 2y ago
I see! Now I understand why I need to create those useless `Modelfile` files... I'm glad there is a more open source alternative to Ollama now.
- zozbot234 2y ago
  I don't get it. The 'Modelfile' files are used to save and restore chat history as well, set custom system prompts and lots of other stuff that would require custom coding with most other local AI frameworks. Llama.cpp certainly doesn't offer anything like that out of the box. Those sorts of complaints seem pointless to me.
  - homarp 2y ago
    you tried a recent build of llama-server (from llama.cpp) ? the web interface does remember my chat, and obviously let me change all settings.
- threecheese 2y ago
  Interested to know why one is “more open source” than the other.
mchiang 2y ago
I’m one of the maintainers of Ollama. It’s amazing to see others build on top of open-source projects. Forks like RamaLama are exactly what open source is all about. Developers with different design philosophies can still collaborate in the open for everyone’s benefit. Some folks on the Ollama team have contributed directly to the OCI spec, so naturally we started with tools we know best. But we made a conscious decision to deviate because AI models are massive in size - on the order of gigabytes - and we needed performance optimizations that the existing approaches didn’t offer. We have not forked llama.cpp, We are a project written in Go, so naturally we’ve made our own server side serving in server.go. Now, we are beginning to hit performance, reliability and model support problems. This is why we have begun transition to Ollama’s new engine that will utilize multiple engine designs. Ollama is now naturally responsible for the portability between different engines. I did see the complaint about Ollama not using Jinja templates. Ollama is written in Go. I’m listening but it seems to me that it makes perfect sense to support Go templates. We are only a couple of people, and building in the open. If this sounds like vendor lock-in, I'm not sure what vendor lock-in is? You can check the source code: https://github.com/ollama/ollama https://github.com/ollama/ollama
- zozbot234 2y ago
  These comments seem reasonable to me. Could you clarify the Ollama maintainers' POV wrt. the recent discussion of Ollama Vulkan support at https://news.ycombinator.com/item?id=42886680 https://news.ycombinator.com/item?id=42886680 ? Many people seem to be upset that this PR seems to have gotten zero acknowledgment from the Ollama folks, even with so many users being quite interested in it for obvious reasons. (To be clear, I'm not sure that the PR is in a mergeable state as-is, so I would disagree with many of those comments. But this is just my personal POV - and with no statement on the matter from the Ollama maintainers, users will be confused.) EDIT: I'm seeing a newly added comment in the Vulkan PR GitHub thread, at https://github.com/ollama/ollama/pull/5059#issuecomment-2628002106 https://github.com/ollama/ollama/pull/5059#issuecomment-2628... . Quite overdue, but welcome nonetheless!
- deleted 2y ago
  [deleted]
- 2y ago

This is the point of it: https://github.com/ggerganov/llama.cpp/pull/11016#issuecomment-2599740463 https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...

buyucu 2y ago

Thanks for this context, I will give RamaLlama a try!

ericyd 2y ago

I wish this were on the readme. Or if it already is, I wish it were significantly higher up.

mohsen1 2y ago

I see! Now I understand why I need to create those useless `Modelfile` files... I'm glad there is a more open source alternative to Ollama now.

zozbot234 2y ago
I don't get it. The 'Modelfile' files are used to save and restore chat history as well, set custom system prompts and lots of other stuff that would require custom coding with most other local AI frameworks. Llama.cpp certainly doesn't offer anything like that out of the box. Those sorts of complaints seem pointless to me.
- homarp 2y ago
  you tried a recent build of llama-server (from llama.cpp) ? the web interface does remember my chat, and obviously let me change all settings.
threecheese 2y ago
Interested to know why one is “more open source” than the other.

mchiang 2y ago

I’m one of the maintainers of Ollama. It’s amazing to see others build on top of open-source projects. Forks like RamaLama are exactly what open source is all about. Developers with different design philosophies can still collaborate in the open for everyone’s benefit. Some folks on the Ollama team have contributed directly to the OCI spec, so naturally we started with tools we know best. But we made a conscious decision to deviate because AI models are massive in size - on the order of gigabytes - and we needed performance optimizations that the existing approaches didn’t offer. We have not forked llama.cpp, We are a project written in Go, so naturally we’ve made our own server side serving in server.go. Now, we are beginning to hit performance, reliability and model support problems. This is why we have begun transition to Ollama’s new engine that will utilize multiple engine designs. Ollama is now naturally responsible for the portability between different engines. I did see the complaint about Ollama not using Jinja templates. Ollama is written in Go. I’m listening but it seems to me that it makes perfect sense to support Go templates. We are only a couple of people, and building in the open. If this sounds like vendor lock-in, I'm not sure what vendor lock-in is? You can check the source code: https://github.com/ollama/ollama https://github.com/ollama/ollama

zozbot234 2y ago
These comments seem reasonable to me. Could you clarify the Ollama maintainers' POV wrt. the recent discussion of Ollama Vulkan support at https://news.ycombinator.com/item?id=42886680 https://news.ycombinator.com/item?id=42886680 ? Many people seem to be upset that this PR seems to have gotten zero acknowledgment from the Ollama folks, even with so many users being quite interested in it for obvious reasons. (To be clear, I'm not sure that the PR is in a mergeable state as-is, so I would disagree with many of those comments. But this is just my personal POV - and with no statement on the matter from the Ollama maintainers, users will be confused.) EDIT: I'm seeing a newly added comment in the Vulkan PR GitHub thread, at https://github.com/ollama/ollama/pull/5059#issuecomment-2628002106 https://github.com/ollama/ollama/pull/5059#issuecomment-2628... . Quite overdue, but welcome nonetheless!
deleted 2y ago
[deleted]
2y ago