3 ms·

People have different workflows, but mine is frequently, skim the documentation, make a prototype, refine code a bit, add tests, move stuff around, break stuff,

People have different workflows, but mine is frequently, skim the documentation, make a prototype, refine code a bit, add tests, move stuff around, break stuff, rework code, study documentation, refactor a bit more, and then at that point I have enough understanding of the problem to go in at yank out 80% of my code and do it right. If Copilot gives me working code in the prototype stage, good enough that I can just move on to the next thing, my understanding is never going to be good enough that I can go in and structure everything correctly. It will effectively allow me to skip 90% of my workflow, but pay the price. That's not to say that Copilot can't be extremely helpful during the final steps of development. If those findings are correct, I can't say that I'm surprised. Bad code is written by poor understanding and Copilot can't have any understanding beyond what you provide it. It may write better code than the average programmer, but the result is no better than the input given. People are extremely focused on "prompt engineering", so why act surprised when a poor "prompt" in VScode yields a poor result?

andybak 3y ago
I'm not sure why you decided that "use copilot" also implies missing out most of your later steps. Who decides to skip all those steps? Presumably you? My experience is that Copilot is great at getting me started. Sometimes the code is good, sometimes it's mediocre or completely broken. But it's invaluable at getting me thinking. I wasted a lot more time before I started using it. That might just be my weird brain wiring... (Edited to sound less narky. I shouldn't post from a mobile device)
- fwsgonzo 3y ago
  I recently tried Copilot out of curiosity and this is my experience too: It helps me getting started, which for me is 99% of the challenge. I know how to solve problems, even complex ones, but for some reason getting started is just so extremely hard, sometimes. Copilot lets me get started, even if it's wrong sometimes. There have been times where I have been surprised by how it took something I wrote for a server, and presented the correct client-side implementation. I've used it a few times to describe a problem and let it handle the solution. It's not very good, but I wonder if one should place more blame on PEBCAK and put more time into problem-description. I gave it a few more paragraphs to describe the problem, and eventually I could take it from there. It was still wrong, but enough to get me started. Immensely helpful that way. Another aspect that I'm wondering about is if it will be able to do more with better documented code. Anyone have experience with that? I've started to write more doxygen comments, and hoping to see if there's a slow shift to more accurate predictions.
  - theshrike79 3y ago
    > It helps me getting started This is like writing in general. It's easy to edit crappy text you've written into something better. It's completely impossible to do it to a text you didn't write at all. LLM models are pretty good in doing the crappy first version. It might use abandoned packages or old APIs but the skeleton is there. It's not that hard to add some meat on the bones when the structure exists. Recently I had to parse a pretty crappy XML format (planned by committee) with Go. I just fed the XML to GPT4 and asked it to parse specific values from it. It got like 95% there. I just had to do a few fixes and polish it a bit. Saved me a lot of headache and poking around in documentation.
- bamboozled 3y ago
  I’ve circumvented all of this “getting started” trouble with the pomodoro method. It’s simple and I don’t have to real with maybe broken code and it works for everything in my life. Worth a try.

andybak 3y ago

I'm not sure why you decided that "use copilot" also implies missing out most of your later steps. Who decides to skip all those steps? Presumably you? My experience is that Copilot is great at getting me started. Sometimes the code is good, sometimes it's mediocre or completely broken. But it's invaluable at getting me thinking. I wasted a lot more time before I started using it. That might just be my weird brain wiring... (Edited to sound less narky. I shouldn't post from a mobile device)

fwsgonzo 3y ago
I recently tried Copilot out of curiosity and this is my experience too: It helps me getting started, which for me is 99% of the challenge. I know how to solve problems, even complex ones, but for some reason getting started is just so extremely hard, sometimes. Copilot lets me get started, even if it's wrong sometimes. There have been times where I have been surprised by how it took something I wrote for a server, and presented the correct client-side implementation. I've used it a few times to describe a problem and let it handle the solution. It's not very good, but I wonder if one should place more blame on PEBCAK and put more time into problem-description. I gave it a few more paragraphs to describe the problem, and eventually I could take it from there. It was still wrong, but enough to get me started. Immensely helpful that way. Another aspect that I'm wondering about is if it will be able to do more with better documented code. Anyone have experience with that? I've started to write more doxygen comments, and hoping to see if there's a slow shift to more accurate predictions.
- theshrike79 3y ago
  > It helps me getting started This is like writing in general. It's easy to edit crappy text you've written into something better. It's completely impossible to do it to a text you didn't write at all. LLM models are pretty good in doing the crappy first version. It might use abandoned packages or old APIs but the skeleton is there. It's not that hard to add some meat on the bones when the structure exists. Recently I had to parse a pretty crappy XML format (planned by committee) with Go. I just fed the XML to GPT4 and asked it to parse specific values from it. It got like 95% there. I just had to do a few fixes and polish it a bit. Saved me a lot of headache and poking around in documentation.
bamboozled 3y ago
I’ve circumvented all of this “getting started” trouble with the pomodoro method. It’s simple and I don’t have to real with maybe broken code and it works for everything in my life. Worth a try.