3 ms·

As of now, these chatbots still make some subtle, egregious errors in code that could easily create more bugs than handwriting if you don't thoroughly audit the

As of now, these chatbots still make some subtle, egregious errors in code that could easily create more bugs than handwriting if you don't thoroughly audit the output. It's wise to ban them until they improve or naive users get more instruction how to use them properly. And from experience with some Samsung products, they could do with tightening up their code QA standards a bit.

jstarfish 3y ago
Always append "...and include a passing unit test" to code prompts. Sometimes it's a dick and will test the assertion that 1==1 though.
- zer0tonin 3y ago
  ChatGPT can perfectly return faulty code, and a unit test that runs this faulty code and outputs green
davely 3y ago
I treat it the same was as a code review. Makes it easy to generate a bunch of boilerplate code, but still want to verify it. (Just as I would if it was something I found on Stack Overflow.)
- snotrockets 3y ago
  Code reviews aren't meant to prove correctness. When I review code, I assume the author thought about it, realized this is the best solution, and tested it, all not happening with the models.
  - davely 3y ago
    Honest question -- what are you doing otherwise then? Just checking for typos in variable names?
    - snotrockets 3y ago
      That's what linters are for. Each of us has a limited capacity to take critique from other humans, before we become irritated by it and just ignore it. Therefore, I've learned to leave things that can be caught by automated tools to automated tools. When I look at a PR, there are two things I try to validate: 1. Do I understand what the code is supposed to do, and why the author chose to do those things that way. If not, then either the code need rewriting (because it does things in a way that shouldn't be done,) or more commenting (because it ain't obvious), or both. 2. Catch anything that is obvious to me, but not the author, or to the linter. Maybe there's a different function that does a thing the author didn't know about? And while doing that, it's important to keep context in mind. You could write the kind of code that would be very smart, but can't be understood by most, and that's code is just as bad as a very dumb code (see (1)). If I have to catch obvious errors, than the author didn't do their job. If I have to catch spelling mistakes or formatting issues, than the linter didn't do their job. My job is to catch anything that neither the author nor the tools can.
      - davely 3y ago
        Ah, these are good things for me to keep in mind. Thanks for responding!
AtNightWeCode 3y ago
As the joke goes, senior devs copy code from answers on Stack overflow. Junior devs from questions. It is always better to educate people. ChatGPT needs to be treated as Wikipedia.
- 1123581321 3y ago
  Staff engineers copy code from that one surly guy’s comment on the question.

jstarfish 3y ago

Always append "...and include a passing unit test" to code prompts. Sometimes it's a dick and will test the assertion that 1==1 though.

zer0tonin 3y ago
ChatGPT can perfectly return faulty code, and a unit test that runs this faulty code and outputs green

davely 3y ago

I treat it the same was as a code review. Makes it easy to generate a bunch of boilerplate code, but still want to verify it. (Just as I would if it was something I found on Stack Overflow.)

snotrockets 3y ago
Code reviews aren't meant to prove correctness. When I review code, I assume the author thought about it, realized this is the best solution, and tested it, all not happening with the models.
- davely 3y ago
  Honest question -- what are you doing otherwise then? Just checking for typos in variable names?
  - snotrockets 3y ago
    That's what linters are for. Each of us has a limited capacity to take critique from other humans, before we become irritated by it and just ignore it. Therefore, I've learned to leave things that can be caught by automated tools to automated tools. When I look at a PR, there are two things I try to validate: 1. Do I understand what the code is supposed to do, and why the author chose to do those things that way. If not, then either the code need rewriting (because it does things in a way that shouldn't be done,) or more commenting (because it ain't obvious), or both. 2. Catch anything that is obvious to me, but not the author, or to the linter. Maybe there's a different function that does a thing the author didn't know about? And while doing that, it's important to keep context in mind. You could write the kind of code that would be very smart, but can't be understood by most, and that's code is just as bad as a very dumb code (see (1)). If I have to catch obvious errors, than the author didn't do their job. If I have to catch spelling mistakes or formatting issues, than the linter didn't do their job. My job is to catch anything that neither the author nor the tools can.
    - davely 3y ago
      Ah, these are good things for me to keep in mind. Thanks for responding!

AtNightWeCode 3y ago

As the joke goes, senior devs copy code from answers on Stack overflow. Junior devs from questions. It is always better to educate people. ChatGPT needs to be treated as Wikipedia.

1123581321 3y ago
Staff engineers copy code from that one surly guy’s comment on the question.