3 ms·
> Are any OSS models producing JSON with specific keys as reliably as OpenAI models do with function calling? Yes, OSS models -- from my understanding, this is
by dragonwriter 3y ago
> Are any OSS models producing JSON with specific keys as reliably as OpenAI models do with function calling?
Yes, OSS models -- from my understanding, this is true in theory of any model over which you have fine-grained control of inference, which you do if the inference code isn't a black box, but this has been generally implemented in the frameworks for running OSS models -- can be constrained to generate according to a grammar of arbitrary specificity.
- danenania 3y agoCool, that makes sense. Are there any now doing this off-the-shelf where a schema can be passed in and the response will be valid JSON that matches the schema?
- dragonwriter 3y agoTooling for this was announced on HN 3 months ago, it now does a lot more: https://news.ycombinator.com/item?id=37125118 https://news.ycombinator.com/item?id=37125118 Here is an plugin for one of the popular tools for running multiple local models (it is a tool that presents an OpenAI-style API for downstream consumption, as well as presenting its own Web UI for direct interaction with the models.): https://github.com/hallucinate-games/oobabooga-jsonformer-plugin https://github.com/hallucinate-games/oobabooga-jsonformer-pl...