44 ms·
Am I correct in understanding that SWE-bench is limited to python?
by nateburke 1y ago
Am I correct in understanding that SWE-bench is limited to python?
- babushkaboi 1y agoyeah, they're all python at the moment.
- simonw 1y agoThe core benchmark is only Python, but there is also SWE-bench Multimodal which uses JavaScript: https://arxiv.org/abs/2410.03859 https://arxiv.org/abs/2410.03859 And the new SWE-bench Multilingual (released a couple of weeks ago) which covers 9 programming languages - C, C++, Go, Java, JavaScript, TypeScript, PHP, Ruby and Rust: https://www.swebench.com/multilingual.html https://www.swebench.com/multilingual.html