The whole point of GBNF is to serve as part of the API that lets downstream applications control token sampling in a high-level way without having to drop to ra

The whole point of GBNF is to serve as part of the API that lets downstream applications control token sampling in a high-level way without having to drop to raw logit distributions or pull model-specific tricks. Ollama has a hardcoded GBNF grammar to force generic json output for example, the code is here: https://github.com/ollama/ollama/blob/da09488fbfc437c55a94bc5374b0850d935ea09f/llm/server.go#L634-L654 https://github.com/ollama/ollama/blob/da09488fbfc437c55a94bc... Ollama can also turn user-passed json schema into a more tightly specified GBNF grammar, the code is here and is a bit harder to understand: https://github.com/ollama/ollama/blob/da09488fbfc437c55a94bc5374b0850d935ea09f/llama/sampling_ext.cpp#L62 https://github.com/ollama/ollama/blob/da09488fbfc437c55a94bc... This thread was about doing structured generation in a model-agnostic way without wrapping try/except around json.parse(), and GBNF is _the_ way to do that.