3 ms·
The structured data JSON output thing is a special case: it works by interacting directly with the "select next token" mechanism, restricting the LLM to only pi
by simonw 1y ago
The structured data JSON output thing is a special case: it works by interacting directly with the "select next token" mechanism, restricting the LLM to only picking from a token that would be valid given the specified schema.
This makes invalid output (as far as the JSON schema goes) impossible, with one exception: if the model runs out of output tokens the output could be an incomplete JSON object.
Most of the other things that people call "guardrails" offer far weaker protection - they tend to use additional models which can often be tricked in other ways.
- manquer 1y agoYou are right of course. I didn't mean to imply that all methods give 100% reliability as the structured data does. My point was just that there are non system prompt approaches which give on par or better reliability and/or injection security, it is not just system prompt or bust as other posters suggest.
- gerardatkonvo 1y agoDo you have any sources? Is it the same thing for tool calling parameters?