2 ms·
It can but it'd make up its own schema. The driver is deterministic logic and needs a completely predictable token sequence.
by mike_hearn 4y ago
It can but it'd make up its own schema. The driver is deterministic logic and needs a completely predictable token sequence.
- thomashop 4y agoNot if you give it the correct schema in the prompt
- rfoo 4y agoIn my experience it still sometimes makes up schema or outright outputs mansplaining-like plaintext instead of JSON even if I give it correct schema. Happened once about 15~20 attempts, 5% is still too high to be considered reliable :( I've tuned temperature, added a logit_bias to heavily prefer `{` token, this helped with the plain English v.s. JSON issue, but didn't help with hallucination. I guess I really need API access to {role: "tool"}.