3 ms·
[Multi-] SWE-bench Leaderboards: SWE-bench: https://www.swebench.com/ https://www.swebench.com/ : > SWE-bench is a dataset that tests systems' ability to solv
by westurner 2y ago
[Multi-] SWE-bench Leaderboards:
SWE-bench: https://www.swebench.com/ https://www.swebench.com/ :
> SWE-bench is a dataset that tests systems' ability to solve GitHub issues automatically. The dataset collects 2,294 Issue-Pull Request pairs from 12 popular Python repositories. Evaluation is performed by unit test verification using post-PR behavior as the reference solution.
Multi-SWE-bench: A Multi-Lingual and Multi-Modal GitHub Issue Resolving Benchmark: https://multi-swe-bench.github.io/ https://multi-swe-bench.github.io/