3 ms·
sadly, I think we are stuck with JSON as the most reliable way to get data / code in and out of an LLM (could be worse, could be YAML) … I’m interested in custo
by librasteve 3mo ago
sadly, I think we are stuck with JSON as the most reliable way to get data / code in and out of an LLM (could be worse, could be YAML) … I’m interested in custom DSLs that improve LLM predictability and it is quite nice to see that even the Microsoft dinosaur “gets it” … see the Contacts example at https://slangify.org/examples https://slangify.org/examples which does VCARD to JCARD round tripping as a way to easily roll your own DSL
- chenglong-hn 3mo agoI felt conflicted as well, json is portable and easy to parse / validate and edit. But many models do still struggle. There are some stuff from functional programming might be worth bringing back here.
- catlifeonmars 3mo agoWhat is it about json that models struggle with, in your experience? Is it syntax or structure?
- chenglong-hn 3mo agoIt's often comes with missing keys, use wrong value type (e.g., list over dictionaries). Mostly a small model issue and open source models, they don't follow instructions on the structure guidance that well, and there is no easy way to do generation-time validation.
- catlifeonmars 3mo agoHmm this is a problem that has long been solved with code completion in a JSON language server and a valid JSON schema. By this I mean that you can basically tab-complete your way into a valid JSON document matching a particular schema. Maybe there is a way to restrict the set of output tokens based on the current position in a JSON schema of a JSON document the model is predicting?