3 ms·
I made a serious attempt to do precisely that, and yes, it output a valid JSON structure highly reliably. The problem was stopping it from just inventing values
by chipotle_coyote 2y ago
I made a serious attempt to do precisely that, and yes, it output a valid JSON structure highly reliably. The problem was stopping it from just inventing values for parameters that weren't actually specified by the user.
Consider the possibility that at least some of the criticisms of LLMs are a result of serious attempts to use them.
- _flux 2y agollama.cpp has a way to constrain responses to a grammar, which is 100% reliable as it is implemented in the inference itself. You still need to tell the model to produce a certain format to get good results, though.