3 ms·

I have been trying to figure out how to publish evals or benchmarks for this. But where can I get high quality data of codebases, prompts, and expected results

I have been trying to figure out how to publish evals or benchmarks for this. But where can I get high quality data of codebases, prompts, and expected results? How do I benchmark one codebase output vs another? Would love any tips from the HN community

JTyQZSnP3cQGa8B 2y ago
That's the problem with people who use AI. You think too much and fail to deliver. I'm not asking for benchmarks or complicated stuff, I want source code, actual proof that I can diff myself. Also that's why the SWE is doomed because of AI, but that's another story.