2 ms·
Slightly off topic but is there recommended advice on how to tune / train not for document retrieval but for consistent JSON output with specific enums? i.e gi
by kmkarim 3y ago
Slightly off topic but is there recommended advice on how to tune / train not for document retrieval but for consistent JSON output with specific enums?
i.e given a text, always return back a certain set of fields. For some keys here is the possible set of enums etc. One shot prompting does work but curious how others approach this if you have training data on hand.
- techn00 3y agoYou want grammars to restrict the output, search for "gbnf grammar". That and combined with a good prompt with an example, also check out outlines.dev
- Hugsun 3y agoThere are many interesting tools that achieve this, like Outlines[0] and jsonformer[1]. I haven't tried them myself but they look very promising. [0]: https://github.com/outlines-dev/outlines https://github.com/outlines-dev/outlines [1]: https://github.com/1rgs/jsonformer https://github.com/1rgs/jsonformer
- simonw 3y agoFor OpenAI, use their functions schema mechanism. Aside from that, take a look at llama.cpp grammars.
- ukuina 3y agoMicrosoft Guidance will do this.
- Der_Einzige 3y agoSeems Microsoft spun them out and gave them independence. Not sure why, given it's the kind of IP that helps keep microsoft dominant.
- michaelt 3y agoAsk the model nicely, check any json in the output against your schema, regenerate if it doesn’t match. Crude, I know, but it’s compatible with every model. Which is useful if you want to compare the many different models out there.
- mark_l_watson 3y agoThat is a fairly good strategy. Six years ago at Capital One, I experimented with generating synthetic JSON AWS CloudWatch files for testing (that would not contain any sensitive information). Way back then I used LSTM models and I simply had code to check output and only keep valid samples. LLMs are so much better for this than LSTMs now.