2 ms·
This bench seems to be entirely python based. Are there similar benchmarks that test different languages for these tools?
by attentive 2y ago
This bench seems to be entirely python based.
Are there similar benchmarks that test different languages for these tools?
- ofirpress 2y agoI'm one of the co-authors of SWE-bench. We just created a Javascript (+visual) SWE-bench: https://www.swebench.com/multimodal.html https://www.swebench.com/multimodal.html We're going to release the eval suite for this soon so that people can start making submissions.