5 ms·
As I understand you don't even need to train the model, you can just tell it in plain English how to use the plugin, ie how to format the data, and it will do t
by iforgotpassword 4y ago
As I understand you don't even need to train the model, you can just tell it in plain English how to use the plugin, ie how to format the data, and it will do that if it sees fit.
- mike_hearn 4y agoThat uses up context window and from what I understand it isn't as reliable as fine tuning. My guess is it's not just stuff in the prompt, it's been fine-tuned (re-trained) on examples.
- rfoo 4y agoThe ability to reliably follow such instructions given a JSON manifest likely comes from training on a (small) dataset of these "plugin manifests".
- iforgotpassword 4y agoI guess some additional training doesn't hurt and could make it more deterministic and reliable. But it's impressive how you can already tell it to create simple JSON structures from naturally worded descriptions, so I'm convinced it would already work reasonably well without additional training.
- mike_hearn 4y agoIt can but it'd make up its own schema. The driver is deterministic logic and needs a completely predictable token sequence.
- thomashop 4y agoNot if you give it the correct schema in the prompt
- rfoo 4y agoIn my experience it still sometimes makes up schema or outright outputs mansplaining-like plaintext instead of JSON even if I give it correct schema. Happened once about 15~20 attempts, 5% is still too high to be considered reliable :( I've tuned temperature, added a logit_bias to heavily prefer `{` token, this helped with the plain English v.s. JSON issue, but didn't help with hallucination. I guess I really need API access to {role: "tool"}.