> Which brings me to the other approach: steering the LLM's output __as it is generating tokens__ A relevant PR: https://github.com/ggerganov/llama.cpp/pull/

> Which brings me to the other approach: steering the LLM's output __as it is generating tokens__ A relevant PR: https://github.com/ggerganov/llama.cpp/pull/1773 https://github.com/ggerganov/llama.cpp/pull/1773 The plan is to support arbitrary grammar files to constrain token generation, similar to the grammar files here: https://github.com/antlr/grammars-v4 https://github.com/antlr/grammars-v4