3 ms·
That works for regular computer programs, but the problem is that the user can invent a different delimiter and the AI will "play along" and start using that on
by skybrian 4y ago
That works for regular computer programs, but the problem is that the user can invent a different delimiter and the AI will "play along" and start using that one too.
The AI has no memory of what happened other than the transcript, and when it reads a transcript with multiple delimiters in use, it's not necessarily going to follow any particular escaping rules to figure out which delimiters to ignore.
- duvenaud 4y agoI agree, and this makes my proposed patch a weak solution. I was imagining that the specialness of the token would be reinforced during fine-tuning, but even that wouldn't provide any sort of guarantee.