It's only the controller that runs in Wasm, not the inference. The pyctrl is a just a sample controller, you can write a controller that takes any kind of gram

It's only the controller that runs in Wasm, not the inference. The pyctrl is a just a sample controller, you can write a controller that takes any kind of grammar (eg., a yacc grammar [0] - the python code in that example is only used for glueing). Llama.cpp grammars were quite slow in my testing (20ms per token or so, compared to 2ms for the yacc grammar referenced above). [0] https://github.com/microsoft/aici/blob/main/controllers/pyctrl/samples/cfg.py#L7 https://github.com/microsoft/aici/blob/main/controllers/pyct...