4 ms·

Why is it absurd? If it writes readable code, you can check it. Writing the first version often takes a long time, so this is clearly a breakthrough.

joshvm 4y ago
Then there's a vicious circle. You need to have technical expertise to evaluate whether the code from these models is fit for purpose. Until these models get sufficiently reliable that you can use them without worrying if the results are correct then you still need developers. This may be much better than Stack Overflow, but I imagine it will still suffer from the same problems with regard to copying "answer" code. I would give an answer from ChatGPT where it confidently told me that I should evaluate an object detection model by ranking matches using negative IoU (and it indeed generated code to do it and gave a confident explanation of how this was normal in computer vision, but it was completely backwards).
bhj 4y ago
I would much rather use it as a code review tool than become the code review tool. I suspect the latter will happen at a lot of companies, though.
- sh4rks 4y ago
  Why? When the latter is just as effective but also much quicker?
  - jeremyjh 4y ago
    It isn't just as effective for me. No matter how much I'd like to, I can't review code with the same thoughtfulness and thoroughness that I apply when I write it. I know the same is true for the people who have reviewed my PRs as well, but maybe its different for others. I do use Copilot but mostly it only generates one liners for me that save a little time.
    - agentultra 4y ago
      Empirical studies on large-scale projects employing informal code review (the study I'm talking about monitored the Qt project repositories) suggest that humans have a very low impact on error rates. Reviewing more than ~200LOC every couple of hours makes the effect disappear. So you're not alone. You can even point to the plethora of "find the undefined behaviour," tests: humans are really bad at finding errors in code.

joshvm 4y ago

Then there's a vicious circle. You need to have technical expertise to evaluate whether the code from these models is fit for purpose. Until these models get sufficiently reliable that you can use them without worrying if the results are correct then you still need developers. This may be much better than Stack Overflow, but I imagine it will still suffer from the same problems with regard to copying "answer" code. I would give an answer from ChatGPT where it confidently told me that I should evaluate an object detection model by ranking matches using negative IoU (and it indeed generated code to do it and gave a confident explanation of how this was normal in computer vision, but it was completely backwards).

bhj 4y ago

I would much rather use it as a code review tool than become the code review tool. I suspect the latter will happen at a lot of companies, though.

sh4rks 4y ago
Why? When the latter is just as effective but also much quicker?
- jeremyjh 4y ago
  It isn't just as effective for me. No matter how much I'd like to, I can't review code with the same thoughtfulness and thoroughness that I apply when I write it. I know the same is true for the people who have reviewed my PRs as well, but maybe its different for others. I do use Copilot but mostly it only generates one liners for me that save a little time.
  - agentultra 4y ago
    Empirical studies on large-scale projects employing informal code review (the study I'm talking about monitored the Qt project repositories) suggest that humans have a very low impact on error rates. Reviewing more than ~200LOC every couple of hours makes the effect disappear. So you're not alone. You can even point to the plethora of "find the undefined behaviour," tests: humans are really bad at finding errors in code.