All generated output depends on all the previous output. The model "looks" at (mostly) everything every time. Generation never actually stops, the model just e

All generated output depends on all the previous output. The model "looks" at (mostly) everything every time. Generation never actually stops, the model just emits a special stop token when stopping is most likely next token. Hence the grammar implementation can prevent this stop token from being emitted prematurely. There was some discussion of models getting "stuck" where there is no syntactically correct token to emit. Some proposals included a "backspace token" IIRC, but I dunno what they actually did. You can look through the discussion in the PR.