3 ms·
What a lot of people don’t know is that SWE-bench is over 50% Django code, so all of the top labs hyper optimize to perform well on it.
by m_ke 10mo ago
What a lot of people don’t know is that SWE-bench is over 50% Django code, so all of the top labs hyper optimize to perform well on it.
- kristianp 10mo agoI know python is more prevalent in SWE-Bench than any other language, but more than 50% django sounds like a big stretch. Citation? Edit, it's about 37%, and python-only. https://arxiv.org/pdf/2310.06770v3 https://arxiv.org/pdf/2310.06770v3