3 ms·

So this works for repositories with decent unit tests. Which excludes 80+% of real world bug and feature issues, in my experience…

• abhikrc 3y ago
  No it does not need an unit test to work. We responded to another similar question from another user.
  • prmph 3y ago
    Then how can you have confidence that it actually fixes the bug? It means you still need a human to review the fix, no?
    • yuntong 3y ago
      The developer written testcases are provided in SWE-bench-lite so those could be used to check the generated patches. The auto-generated patches are to reduce the effort of resolving issues. In practice, they should be reviewed and verified by human developers before they are integrated.
  • rrr_oh_man 3y ago
    Thank you for the clarification. And shame on me for talking out of my ass!