3 ms·

How is github copilot worse than a very laborious but a very junior developer who can search stack overflow for you and bring you code snippets that sometimes a

How is github copilot worse than a very laborious but a very junior developer who can search stack overflow for you and bring you code snippets that sometimes are spot on and sometimes are plain wrong? Presumably that's why code review exists...

yarcob 5y ago
We have mandatory code review for every commit that goes into our app, and still stuff like that routinely slips through. You need a lot of experience to spot errors like this, and stuff will still slip through.
TeMPOraL 5y ago
- The exact StackOverflow snippet said junior is copying was seen by others and likely already debugged to some extent. There's typically also a lot of relevant information in the text surrounding the snippet. - The snippet you get from Copilot is generated by an DNN model. I'd think that alone should scare people. DNN models are correct on a continuous scale, not discrete. A bad pixel on a generated image, an extra comma in generated text, are considered OK. That kind of correctness framework doesn't work with code. From what I read, Codex (the model powering Copilot) is a GPT-3 derivative. Surely you've played with GPT-3 before, you've seen the types of mistakes it makes. In my eyes, this approach just won't fly for code.
- IncRnd 5y ago
  > - The exact StackOverflow snippet said junior is copying was seen by others and likely already debugged to some extent. There's typically also a lot of relevant information in the text surrounding the snippet. Empirically, this is false. There is an incredibly high amount of bad, incomplete, or vulnerable code on SO.
azornathogron 5y ago
A very junior developer that brings me code snippets that are sometimes plain wrong is not useful to me directly - such a developer costs more time in review than they save by providing the snippets in the first place. The value of such a developer is that over time and with some mentorship they will become a slightly less junior developer who can bring me whole features that I can be reasonably confident are mostly correct. Having said that, I don't know how the much more integrated write/review/edit cycle will work in practice when using copilot. I don't think it will be the same as a junior developer/pair programming partner in any real sense. My initial reaction to copilot is negative, but I'm open to being proven wrong about it.
IncRnd 5y ago
The difference is that copilot constantly emits code visibly on the screen.
naniwaduni 5y ago
This point might be a bit lost in the current state of the industry as HN is exposed to it, but usually the idea behind keeping a very junior developer around is that gradually it'll stop being a very junior developer and become a slightly less junior developer who will spit out code snippets that aren't usually just plain wrong.
tengbretson 5y ago
I often find that doing good code review is much more exhausting than writing good code. Anything that puts more garbage on my plate for review is a net drain on resources.

yarcob 5y ago

We have mandatory code review for every commit that goes into our app, and still stuff like that routinely slips through. You need a lot of experience to spot errors like this, and stuff will still slip through.

TeMPOraL 5y ago

- The exact StackOverflow snippet said junior is copying was seen by others and likely already debugged to some extent. There's typically also a lot of relevant information in the text surrounding the snippet. - The snippet you get from Copilot is generated by an DNN model. I'd think that alone should scare people. DNN models are correct on a continuous scale, not discrete. A bad pixel on a generated image, an extra comma in generated text, are considered OK. That kind of correctness framework doesn't work with code. From what I read, Codex (the model powering Copilot) is a GPT-3 derivative. Surely you've played with GPT-3 before, you've seen the types of mistakes it makes. In my eyes, this approach just won't fly for code.

IncRnd 5y ago
> - The exact StackOverflow snippet said junior is copying was seen by others and likely already debugged to some extent. There's typically also a lot of relevant information in the text surrounding the snippet. Empirically, this is false. There is an incredibly high amount of bad, incomplete, or vulnerable code on SO.

azornathogron 5y ago

A very junior developer that brings me code snippets that are sometimes plain wrong is not useful to me directly - such a developer costs more time in review than they save by providing the snippets in the first place. The value of such a developer is that over time and with some mentorship they will become a slightly less junior developer who can bring me whole features that I can be reasonably confident are mostly correct. Having said that, I don't know how the much more integrated write/review/edit cycle will work in practice when using copilot. I don't think it will be the same as a junior developer/pair programming partner in any real sense. My initial reaction to copilot is negative, but I'm open to being proven wrong about it.

IncRnd 5y ago

The difference is that copilot constantly emits code visibly on the screen.

naniwaduni 5y ago

This point might be a bit lost in the current state of the industry as HN is exposed to it, but usually the idea behind keeping a very junior developer around is that gradually it'll stop being a very junior developer and become a slightly less junior developer who will spit out code snippets that aren't usually just plain wrong.

tengbretson 5y ago

I often find that doing good code review is much more exhausting than writing good code. Anything that puts more garbage on my plate for review is a net drain on resources.