2 ms·
>But it struggles with json You should customise your sampler to mandate JSON grammar after ```json tokens.
by tucnak 2y ago
>But it struggles with json
You should customise your sampler to mandate JSON grammar after ```json tokens.
- ed 2y agoGrammar samplers are clever! But in the case of a missing escape character you’ll end up with a corrupted string. Take for example: "A dog says \"Woof!\"" With a grammar, you’ll end up with "A dog says " when the model forgets to escape. Which is valid JSON, but not what the model intended. So it’s usually better to catch the exception and ask the model to try again. Unless you’ve come across a sampler with backtracking? That would be cool