5 ms·
You would also need to keep generating until the whole string is valid. And what if it gets caught in a loop? Not sure how this can really guarantee 100%
by dontreact 3y ago
You would also need to keep generating until the whole string is valid. And what if it gets caught in a loop?
Not sure how this can really guarantee 100%
- orlp 3y ago> And what if it gets caught in a loop? Not sure how this can really guarantee 100% It's not great but after some timeout you can just set the mask to only include closing brackets.
- aassddffasdf 3y agoYou would still have to ensure balancing somehow. Both "]" and "}" are valid "closing brackets" and the correct one to choose is context-dependent.
- gyy52380 3y agoYou can determine which brackets you need in which order by parsing the incomplete json which was generated so far.
- dontreact 3y agoThat won't do it, also need to close other stuf {"this": "is valid json so farrrrrrrrrrrrrr But yeah the general idea makes sense. Once you hit a timeout, change the mask to things that will close existing open things in a valid manner (}, ), ], ")
- kristjansson 3y agoSame problem with normal sampling - if it doesn't pick the <end> token, you're stuck generating until you hit some stopping heuristic (max tokens, timeout, etc.)