3 ms·
Oh, interesting - thanks for the link. I really haven't explored this but it should slot in fairly easily I think? Gotta dig into it more.
by zambelli 4mo ago
Oh, interesting - thanks for the link. I really haven't explored this but it should slot in fairly easily I think? Gotta dig into it more.
- tmzt 4mo agoIt's basically restricting what logits are allowed when sampling the model to conform with the JSON (or whatever) shape. It can also cause the model to get "confused" though and doesn't always result in the output you want.