3 ms·
Totally. When using GPT I exclusively use function calling, which is far more reliable for JSON output. When using other models I don't even bother with stric
by binarymax 3y ago
Totally. When using GPT I exclusively use function calling, which is far more reliable for JSON output. When using other models I don't even bother with strict JSON because of the error rate - and instead opt for HTML which is more forgiving when parsing.
- msp26 3y agonono, the same thing happens with OpenAI's JSON mode and function calling. It does output parsable JSON very reliably but it comes out mangled with a bunch of whitespace sometimes. GPT-4-turbo's output context window is limited to 4096 so fixing this is relevant. You can use logit_bias for it.