3 ms·
> Fix curly brace tokens to the beginning Regular LLMs can already do this, by prefilling the start of the assistant's response. But there is actually somethi
by zakkor1 2y ago
> Fix curly brace tokens to the beginning
Regular LLMs can already do this, by prefilling the start of the assistant's response.
But there is actually something even better: you can constrain the LLM's output to a specific grammar (like JSON), so it'll only be able to answer with syntactically valid JSON.
- vessenes 2y agoYes. And you can have a grammar parser only select from valid tokens in a randomized distribution. But, this feels much more sophisticated to me, especially if you can mix specific token-based grammar requirements with other instructions during the token selection phase.