3 ms·
> Is there any available comparison of using XML, JSON, or even Markdown for prompts and structured output? I've done a lot of testing with this and found that
by NeatPasta 2y ago
> Is there any available comparison of using XML, JSON, or even Markdown for prompts and structured output?
I've done a lot of testing with this and found that XML is the best input AND output if you want to produce machine readable data. Markdown is okay as input, but testing shows better accuracy if input components are wrapped in XML tags. While it would be ideal for extracting data, JSON syntax is too sensitive as output. Often an offending character will be unescaped no matter how good your prompt is.
If you stay within the very basic syntax of XML (just tags) it is also the best output. I suspect it is to do with the descriptive nature of the tag structure. Something wrapped in the tag <summary> will hint to the model that it should produce a summary, reinforcing the input prompt as the output is being constructed.
- ai_what 2y agoThis is really good to know, thanks! Was this for every LLM or only Claude?
- jiggawatts 2y agoI really don’t understand why not all LLMs use schema validation on their output 100% guarantee that it is actually JSON and that it matches the expected format.
- nl 2y agoFunction calling in OpenAI models is essentially this. You define the functions using JSON Schema and it is guaranteed to match. See also https://python.useinstructor.com/ https://python.useinstructor.com/