> this just add another layer of output verification IMO. It's not verifying the output after it's done, it's constraining the output as it's generated. > But

> this just add another layer of output verification IMO. It's not verifying the output after it's done, it's constraining the output as it's generated. > But does it really save much as comparing to get the output then parse and reject the output that doesn't follow the grammar? Might be debateable. I don't think it's debatable at all. Forcing the model to conform to a grammar during generation means there is never a need to discard and regenerate because it got the grammar wrong. Think of the compute involved in generating the whole output and then re-generating if it's non-conformant. There is no comparison.