3 ms·
What is it about json that models struggle with, in your experience? Is it syntax or structure?
by catlifeonmars 3mo ago
What is it about json that models struggle with, in your experience? Is it syntax or structure?
- chenglong-hn 3mo agoIt's often comes with missing keys, use wrong value type (e.g., list over dictionaries). Mostly a small model issue and open source models, they don't follow instructions on the structure guidance that well, and there is no easy way to do generation-time validation.
- catlifeonmars 3mo agoHmm this is a problem that has long been solved with code completion in a JSON language server and a valid JSON schema. By this I mean that you can basically tab-complete your way into a valid JSON document matching a particular schema. Maybe there is a way to restrict the set of output tokens based on the current position in a JSON schema of a JSON document the model is predicting?