3 ms·
You've overstated the conclusion. The SWE-bench series has had issues since its inception. OpenAI no longer recommends SWE-Bench-Pro as a benchmark: https://op
by sk4rekr0w 3mo ago
You've overstated the conclusion. The SWE-bench series has had issues since its inception.
OpenAI no longer recommends SWE-Bench-Pro as a benchmark: https://openai.com/index/separating-signal-from-noise-coding-evaluations/ https://openai.com/index/separating-signal-from-noise-coding...
- bigfishrunning 3mo agoThat's smart, they only recommend benchmarks that make them look better then their competitors.