3 ms·
Opus 5 generates really good code and terminal commands though. It's just bad at the accompanying text it tells you. These benchmarks don't grade the text gener
by didibus 2mo ago
Opus 5 generates really good code and terminal commands though. It's just bad at the accompanying text it tells you. These benchmarks don't grade the text generation of the response I don't think, only the task outcome.
- simondotau 2mo agoOpus 5 keeps making howlingly stupid errors, like one recently where a regexp would catch an invalid date because -\d{2} won’t match -00 Seriously.
- didibus 2mo agoThat's just all models though. They're not 100% accurate.
- simondotau 2mo agoI regularly have issues like that with Opus. I rarely do with the supposedly sub-frontier Grok 4.6, or Kimi K3, or even with Sol.