3 ms·
This paper doesn’t really tell us much. The cutoff was September of 2025. The models have improved so much that I just don’t know what you can take away from th
by woeirua 7mo ago
This paper doesn’t really tell us much. The cutoff was September of 2025. The models have improved so much that I just don’t know what you can take away from this experiment.
- croemer 7mo agoSWE-bench scores are inflated compared against actual maintainer merge decisions as opposed to an LLM grader.