2 ms·
OK, let me try to recover. :) I'm using JS to wrap around the llama.cpp project. Most of the work is done by that. JS is just used to pull the tokens out, norm
by xrd 3y ago
OK, let me try to recover. :)
I'm using JS to wrap around the llama.cpp project. Most of the work is done by that. JS is just used to pull the tokens out, normalize them, and then send them to a real-time database (pocketbase). The frontend is a Svelte app.
It's not a lot of code, llama.cpp does almost everything.