3 ms·
Thank you! Yes the app has bundled built-in llama.cpp binary with it. The app just launches the executable - nothing fancy. We are a little llama.cpp wrapper :)
by xyc 2y ago
Thank you! Yes the app has bundled built-in llama.cpp binary with it. The app just launches the executable - nothing fancy. We are a little llama.cpp wrapper :)
It doesn't require Ollama, but if you have existing Ollama, it works with Ollama's API compatible layer https://recurse.chat/faq/#openai-chat-completion-models https://recurse.chat/faq/#openai-chat-completion-models.
- _akhe 2y agoVery cool you covered all bases. I'm going through this exact thing in an app, just moved it over from relying on ollama to using node-llama-cpp internally, but maybe I'll copy you and still support ollama too :D
- xyc 2y agohave fun :) your users will probably let you know which way they want, or both