4 ms·
In my tests it's worst with adding extra formatting or output: https://aibenchy.com/compare/anthropic-claude-opus-4-6-medium/anthropic-claude-sonnet-4-6-medium/
by XCSme 7mo ago
In my tests it's worst with adding extra formatting or output: https://aibenchy.com/compare/anthropic-claude-opus-4-6-medium/anthropic-claude-sonnet-4-6-medium/openai-gpt-5-3-codex-medium/google-gemini-3-1-pro-preview-medium/ https://aibenchy.com/compare/anthropic-claude-opus-4-6-mediu...
For example, sometimes it outputs in markdown, without being asked to (e.g. "**13**" instead of "13"), even when asked to respond with a number only.
This might be fine in a chat-environment, but not in a workflow, agentic use-case or tool usage.
Yes, it can be enforced via structured output, but in a string field from a structured output you might still want to enforce a specific natural-language response format, which can't be defined by a schema.