4 ms·
I'm skeptical. Partially because if you go to https://www.swebench.com/ https://www.swebench.com/, you can see this company underreported results from their com
by potatoman22 2y ago
I'm skeptical. Partially because if you go to https://www.swebench.com/ https://www.swebench.com/, you can see this company underreported results from their competitors like Amazon Q Developer. I've also seen plenty of other projects claim they've reached 30%+ on SWE-bench without verifying or posting their results on this site.