3 ms·
Depends on the way you do constrained generation. If all you do is reject tokens using a grammar, then yeah it is bad. If your software inserts things like fiel
by imtringued 2y ago
Depends on the way you do constrained generation. If all you do is reject tokens using a grammar, then yeah it is bad. If your software inserts things like field names and braces instead of forcing the model to produce them token by token and then afterwards rejecting the wrong tokens, then you should be good to go.
- potatoman22 2y agoWhy is it bad to reject tokens using a grammar?