3 ms·
Are any OSS models producing JSON with specific keys as reliably as OpenAI models do with function calling? That would be needed to implement function calling.
by danenania 3y ago
Are any OSS models producing JSON with specific keys as reliably as OpenAI models do with function calling? That would be needed to implement function calling.
- dragonwriter 3y ago> Are any OSS models producing JSON with specific keys as reliably as OpenAI models do with function calling? Yes, OSS models -- from my understanding, this is true in theory of any model over which you have fine-grained control of inference, which you do if the inference code isn't a black box, but this has been generally implemented in the frameworks for running OSS models -- can be constrained to generate according to a grammar of arbitrary specificity.
- danenania 3y agoCool, that makes sense. Are there any now doing this off-the-shelf where a schema can be passed in and the response will be valid JSON that matches the schema?
- dragonwriter 3y agoTooling for this was announced on HN 3 months ago, it now does a lot more: https://news.ycombinator.com/item?id=37125118 https://news.ycombinator.com/item?id=37125118 Here is an plugin for one of the popular tools for running multiple local models (it is a tool that presents an OpenAI-style API for downstream consumption, as well as presenting its own Web UI for direct interaction with the models.): https://github.com/hallucinate-games/oobabooga-jsonformer-plugin https://github.com/hallucinate-games/oobabooga-jsonformer-pl...