3 ms·
The only one it doesn't win is SWE bench which it is significantly behind Claude Sonnet. You just can't take down Sonnet.
by ilaksh 11mo ago
The only one it doesn't win is SWE bench which it is significantly behind Claude Sonnet. You just can't take down Sonnet.
- stavros 11mo agoCodex has been much better than Sonnet for me.
- dotancohen 11mo agoOn what types of tasks?
- svantana 11mo agoOne percentage point is not significant, neither in the colloquial nor the scientific sense[1]. [1] Binomial formula gives a confidence interval of 3.7%, using p=0.77, N=500, confidence=95%