3 ms·
Have you tried your model on this new benchmark from Princeton NLP? https://www.swebench.com/ https://www.swebench.com/ It's more "real-world" than the toy-lik
by svantana 3y ago
Have you tried your model on this new benchmark from Princeton NLP? https://www.swebench.com/ https://www.swebench.com/
It's more "real-world" than the toy-like problems in HumanEval, it would be interesting to see if you can crack some of those.