3 ms·
I find these imperative statements much less intimidating than llama.cpp grammars - the Python code I copied here looks a lot more approachable to me than the g
by simonw 3y ago
I find these imperative statements much less intimidating than llama.cpp grammars - the Python code I copied here looks a lot more approachable to me than the gbnf syntax from https://raw.githubusercontent.com/ggerganov/llama.cpp/master/grammars/json_arr.gbnf https://raw.githubusercontent.com/ggerganov/llama.cpp/master...
I don't think the key idea here is to run llama.cpp itself in WASM - it's to run LLMs in native code, but have fast custom-written code from end-users that can help pick the next token. WASM is a neat mechanism for that because many different languages can use it as a compile target, and it comes with a robust sandbox by default.