2 ms·

Hardly matters, this isn't a problem that you'd have these days with modern LLMs. Also, a model can always use a proxy to turn your tool calls into XML And fe

• andrew_zhong 7mo ago
  We do see fewer invalid JSONs on latest bigger LLMs but still can happen on smaller and cheaper models. There is also case when input is truncated or a required field not found, which are inherently difficult. On XML vs JSON, I think the goal here is to generate typed output where JSON with zod shines - for example the result can type check and be inserted to database typed columns later
  • faangguyindia 7mo ago
    Thing is even with XML LLM will fail every now and then. I've built an agent in both tool calling and by parsing XML You always need a self correcting loop built in, if you are editing a file with LLM you need provide hints so LLM gets it right the second time or 3rd or n time. Just by switching to XML you'll not get that. I used to use XML now i only use it for examples in in system prompt for model to learn. That's all
    • andrew_zhong 7mo ago
      Agreed - in this project I did a one path sanitation to recover invalid optional / nullable fields or discard invalid objects in nested array. I know multi path LLM approaches exist: e.g. generating JSON patches https://github.com/hinthornw/trustcall https://github.com/hinthornw/trustcall