3 ms·
If you're choosing the next token based on a list of valid next tokens, a uniform random distribution can always generate valid JSON too!
by dwattttt 3y ago
If you're choosing the next token based on a list of valid next tokens, a uniform random distribution can always generate valid JSON too!
- quickthrower2 3y agoYep. So can this: fun generate_valid_json(seed): return "{}"
- neoncontrails 3y agoBut that's not what an LLM does.
- antonvs 3y agoThe point is that if you're "choosing the next token based on a list of valid next tokens," it's not surprising that you'll only generate valid output, since absolutely any choice mechanism will suffice.