4 ms·

A relevant recent tweet from antirez: https://x.com/antirez/status/2054854124848415211 https://x.com/antirez/status/2054854124848415211 > Gentle reminder on ho

A relevant recent tweet from antirez: https://x.com/antirez/status/2054854124848415211 https://x.com/antirez/status/2054854124848415211 > Gentle reminder on how, in the recent DS4 fiesta, not just me but every other contributor found GPT 5.5 able to help immensely and Opus completely useless. I've noticed the same for lower level squeezing-as-much-performance-as-possible code work.

throwaway041207 6mo ago
Assuming we are talking about Code/Codex are you on API billing or subscription? I have essentially unlimited API billing at my disposal and I haven't noticed any degradation of quality across Opus versions.
- chatmasta 6mo ago
  Same here, the enterprise version of Claude has been great. Luckily I’m not the one paying for it. We also have CoPilot and when GPT-5.4 came out, and was 1x request cost, I was very impressed but haven’t had much time to compare the two. I also don’t have time to do much personal coding outside of work, so I haven’t subscribed to a personal one yet. But I intend to go for Codex just to balance the Claude at work and also because of the hostile moves from Anthropic toward their consumer business.
sanxiyn 6mo ago
There is a benchmark for performance work, and I think it is not being optimized by model vendors. The latest result from GSO is that both Opus 4.6 and 4.7 slightly outperforms GPT 5.5. This also matches my experience. https://gso-bench.github.io/ https://gso-bench.github.io/
- vitorsr 6mo ago
  Tasks are taken from commit histories in public Git repositories which defeats the purpose.
deleted 6mo ago
[deleted]
rjh29 6mo ago
There's so much subjectivity with models. As soon as a new model comes out people act like the last model they used for 6 months was completely useless.
- sowbug 5mo ago
  That's a corollary of the Jevons paradox, isn't it? Six months ago I was watching every line of my LLM's code generation like a hawk. Now I give it a few assignments before bed and wake up to see what's live from the CI/CD pipeline. Expectations and usage have ratcheted up.

throwaway041207 6mo ago

Assuming we are talking about Code/Codex are you on API billing or subscription? I have essentially unlimited API billing at my disposal and I haven't noticed any degradation of quality across Opus versions.

chatmasta 6mo ago
Same here, the enterprise version of Claude has been great. Luckily I’m not the one paying for it. We also have CoPilot and when GPT-5.4 came out, and was 1x request cost, I was very impressed but haven’t had much time to compare the two. I also don’t have time to do much personal coding outside of work, so I haven’t subscribed to a personal one yet. But I intend to go for Codex just to balance the Claude at work and also because of the hostile moves from Anthropic toward their consumer business.

sanxiyn 6mo ago

There is a benchmark for performance work, and I think it is not being optimized by model vendors. The latest result from GSO is that both Opus 4.6 and 4.7 slightly outperforms GPT 5.5. This also matches my experience. https://gso-bench.github.io/ https://gso-bench.github.io/

vitorsr 6mo ago
Tasks are taken from commit histories in public Git repositories which defeats the purpose.

deleted 6mo ago

[deleted]

rjh29 6mo ago

There's so much subjectivity with models. As soon as a new model comes out people act like the last model they used for 6 months was completely useless.

sowbug 5mo ago
That's a corollary of the Jevons paradox, isn't it? Six months ago I was watching every line of my LLM's code generation like a hawk. Now I give it a few assignments before bed and wake up to see what's live from the CI/CD pipeline. Expectations and usage have ratcheted up.