4 ms·
Amazing work! This looks great. Question: "no config setup" You mean the user doesn't have to pre-install Ollama or build llama.cpp or anything? If you don't
by _akhe 2y ago
Amazing work! This looks great.
Question: "no config setup" You mean the user doesn't have to pre-install Ollama or build llama.cpp or anything?
If you don't mind me asking - how are you accomplishing that in an offline desktop app?
- xyc 2y agoThank you! Yes the app has bundled built-in llama.cpp binary with it. The app just launches the executable - nothing fancy. We are a little llama.cpp wrapper :) It doesn't require Ollama, but if you have existing Ollama, it works with Ollama's API compatible layer https://recurse.chat/faq/#openai-chat-completion-models https://recurse.chat/faq/#openai-chat-completion-models.
- _akhe 2y agoVery cool you covered all bases. I'm going through this exact thing in an app, just moved it over from relying on ollama to using node-llama-cpp internally, but maybe I'll copy you and still support ollama too :D
- xyc 2y agohave fun :) your users will probably let you know which way they want, or both