3 ms·

This is super fascinating stuff, excellent work. As most of us don't have the time to read the entirety of the paper, are you able to directly link to some issu

This is super fascinating stuff, excellent work. As most of us don't have the time to read the entirety of the paper, are you able to directly link to some issues which have been landed and closed? Some personal favorites would be awesome :) I think I speak for others when I say the best way to judge the efficacy of this project is some real-world, on-site examples of it being used in prod. I'm especially curious for its performance in feature-request or flakey bug report type issues as opposed to reliable test failures. I expect the former is much tougher!

dboreham 3y ago
fwiw the example issue highlighted in the post was already fixed by a human 3 years ago so I wouldn't expect to see much in the way of real life fixed issues yet.
yuntong 3y ago
Thank you for your interest. There are some interesting examples in the SWE-bench-lite benchmark which are resolved by AutoCodeRover: - From sympy: https://github.com/sympy/sympy/issues/13643 https://github.com/sympy/sympy/issues/13643. AutoCodeRover's patch for it: https://github.com/nus-apr/auto-code-rover/blob/main/results/acr-run-2/applicable_patch/sympy__sympy-13647_2024-04-04_13-43-51/extracted_patch_1.diff https://github.com/nus-apr/auto-code-rover/blob/main/results... - Another one from scikit-learn: https://github.com/scikit-learn/scikit-learn/issues/13070 https://github.com/scikit-learn/scikit-learn/issues/13070. AutoCodeRover's patch (https://github.com/nus-apr/auto-code-rover/blob/main/results/acr-run-2/applicable_patch/scikit-learn__scikit-learn-13142_2024-04-04_13-28-08/extracted_patch_1.diff https://github.com/nus-apr/auto-code-rover/blob/main/results...) modified a few lines below (compared to the developer patch) and wrote a different comment. There are more examples in the results directory (https://github.com/nus-apr/auto-code-rover/tree/main/results https://github.com/nus-apr/auto-code-rover/tree/main/results).

dboreham 3y ago

fwiw the example issue highlighted in the post was already fixed by a human 3 years ago so I wouldn't expect to see much in the way of real life fixed issues yet.

yuntong 3y ago

Thank you for your interest. There are some interesting examples in the SWE-bench-lite benchmark which are resolved by AutoCodeRover: - From sympy: https://github.com/sympy/sympy/issues/13643 https://github.com/sympy/sympy/issues/13643. AutoCodeRover's patch for it: https://github.com/nus-apr/auto-code-rover/blob/main/results/acr-run-2/applicable_patch/sympy__sympy-13647_2024-04-04_13-43-51/extracted_patch_1.diff https://github.com/nus-apr/auto-code-rover/blob/main/results... - Another one from scikit-learn: https://github.com/scikit-learn/scikit-learn/issues/13070 https://github.com/scikit-learn/scikit-learn/issues/13070. AutoCodeRover's patch (https://github.com/nus-apr/auto-code-rover/blob/main/results/acr-run-2/applicable_patch/scikit-learn__scikit-learn-13142_2024-04-04_13-28-08/extracted_patch_1.diff https://github.com/nus-apr/auto-code-rover/blob/main/results...) modified a few lines below (compared to the developer patch) and wrote a different comment. There are more examples in the results directory (https://github.com/nus-apr/auto-code-rover/tree/main/results https://github.com/nus-apr/auto-code-rover/tree/main/results).