4 ms·
Couldn't you program the sampler to not output certain token sequences?
by potatoman22 3y ago
Couldn't you program the sampler to not output certain token sequences?
- edgyquant 3y agoThat’s seems extremely difficult if not impossible. There’s a million ways an idea can be conveyed in language.
- namibj 3y agoYeah. E.g. GPT-4-turbo's JSON-mode seems to forcibly block non-JSON-compliant outputs, at least in some way. They document that forgetting to instruct it to emit JSON may lead to producing whitespace until the output length limit is reached. In related info, there is "Guiding Language Models of Code with Global Context using Monitors" ( https://arxiv.org/abs/2306.10763 https://arxiv.org/abs/2306.10763 ), which essentially gives IDE-typical type-aware autocomplete to an LLM to primarily study the scenario of enforcing type-consistent method completion in a Java repository.