3 ms·

Any reason for this for my own learning? Was XML more prevalent during training? Something better about XML that makes it easier for the LLM to work with? XML

• CSMastermind 1y ago
  It's in the official OpenAI prompting guidelines: https://cookbook.openai.com/examples/gpt4-1_prompting_guide#prompting-induced-planning--chain-of-thought https://cookbook.openai.com/examples/gpt4-1_prompting_guide#... But it's also evident for anyone who has used these models. It's also not unique to OpenAI, this bias is prevalent in every model I've ever tested from GPT 3 to the latest offerings from every single frontier model provider. As to why I would guess it's because XML bakes semantic meaning into the tags it uses so it's easier for the model to understand the structure of the data. <employee>...</employee> is a lot easier to understand than { "employee": { ... }}. I would guess that the models are largely ignoring the angular brackets and just focusing on the words which have unique tokens and thus are easier to pair up than the curly braces that are the same throughout JSON. Just speculation on my part though. And this only applies to the input. Earlier models struggled to reliably output JSON so they've been both fine-tuned and wrapped in specific formatters that reliably force clean JSON outputs.
• nitwit005 1y ago
  I've seen the suggestion it's because it's been trained on a lot of HTML, but the GPT docs suggest using markdown as a default choice, which is relatively less common.