4 ms·

GitHub’s CoPilot is the most disappointing of AI products out there.

bhouston 3y ago
For some developers for sure, but not all. I am a senior developer with +20 years of experience and I find Github Copilot amazing. It significantly boosts my productivity. I do not expect the world from it, and I do run into limitations, but it helps me tremendously to be productive in languages and tools which I have low/no familiarity. I think Github Copilot paired with ChatGPT fails the most in idiosyncratic code bases where you are doing maintenance or complex refactors. I think Github Copilot with ChatGPT shines in writing green field code in areas where you are will to use the best practices and the open sources libraries that Chat GPT recommends. Basically, if you align with where Chat GPT recommends to go, then it will help you a lot more than if you try to fight it all the way.
- uxp8u61q 3y ago
  It's also amazing to write documentation or unit tests. Or repetitive editing tasks that are hard to automate (e.g. "invert this if condition").
  - insanitybit 3y ago
    > or unit tests Seriously! I write a documentation comment like: /// Given: blah /// When: blah /// Then: blah and Copilot writes the test for me. I can generate a dozen tests like this and it's awesome. I write a lot more tests because of Copilot.
  - baq 3y ago
    It even does an ok job with ‘rewrite this callback heavy code as async/await’ if you tell it which functions to use instead of ones accepting callbacks.
- pjc50 3y ago
  Which language? New or maintenance code?
  - bhouston 3y ago
    JavaScript/TypeScript. New code generally. I find that ChatGPT/Github Copilot knows certain types of patterns, libraries better than others. If you get it to suggest the path forward, and then do that path, it will be better at maintaining that code. Basically ChatGPT sort of has happy paths which you want to follow. Or you could say it in another way: AI designed code / architecture is easier for an AI to maintain.
- __mharrison__ 3y ago
  If you are an expert (or very good) with your tools, these AI products can save tons of time. Due to hallucinations, beginners can get knee deep very quickly, but experts can see through this and use these tools will skill. I don't write as much code as I used to, but I still write code daily and appreciate how they can turn what was a 10-60 minute task into 1 minute of prompting plus 30 seconds of generation/reading to make sure it is valid.
- infecto 3y ago
  Agree with you there. I find it pretty useful especially for the price and where it might evolve in the future. I am not sure why so many people have hard time getting value out of it.
- LeicaLatte 3y ago
  To clarify, I have been paying the 10$ since it was released last year. I used it in neovim, vscode and found it helpful. Cut to few months later with OpenAI API release, so many code completers released in many useful form factors willing to go wherever I code even if its Android Studio or Xcode. The product has stagnated completely. I do expect it to change significantly in the future though, might go back then.
tkellogg 3y ago
imo it's the form factor that works. ChatGPT is good for doing standalone things like generating a script. CoPilot is good for making incremental changes to an existing code base. Honestly, its REALLY good at it. One of my favorite parts is that it mimics the style of the surrounding code.
nmfisher 3y ago
Yeah, I was very disappointed. I cancelled my free trial three days in, it was clearly inferior to the alternative (copy+paste into Claude).

For some developers for sure, but not all. I am a senior developer with +20 years of experience and I find Github Copilot amazing. It significantly boosts my productivity. I do not expect the world from it, and I do run into limitations, but it helps me tremendously to be productive in languages and tools which I have low/no familiarity. I think Github Copilot paired with ChatGPT fails the most in idiosyncratic code bases where you are doing maintenance or complex refactors. I think Github Copilot with ChatGPT shines in writing green field code in areas where you are will to use the best practices and the open sources libraries that Chat GPT recommends. Basically, if you align with where Chat GPT recommends to go, then it will help you a lot more than if you try to fight it all the way.

uxp8u61q 3y ago
It's also amazing to write documentation or unit tests. Or repetitive editing tasks that are hard to automate (e.g. "invert this if condition").
- insanitybit 3y ago
  > or unit tests Seriously! I write a documentation comment like: /// Given: blah /// When: blah /// Then: blah and Copilot writes the test for me. I can generate a dozen tests like this and it's awesome. I write a lot more tests because of Copilot.
- baq 3y ago
  It even does an ok job with ‘rewrite this callback heavy code as async/await’ if you tell it which functions to use instead of ones accepting callbacks.
pjc50 3y ago
Which language? New or maintenance code?
- bhouston 3y ago
  JavaScript/TypeScript. New code generally. I find that ChatGPT/Github Copilot knows certain types of patterns, libraries better than others. If you get it to suggest the path forward, and then do that path, it will be better at maintaining that code. Basically ChatGPT sort of has happy paths which you want to follow. Or you could say it in another way: AI designed code / architecture is easier for an AI to maintain.
__mharrison__ 3y ago
If you are an expert (or very good) with your tools, these AI products can save tons of time. Due to hallucinations, beginners can get knee deep very quickly, but experts can see through this and use these tools will skill. I don't write as much code as I used to, but I still write code daily and appreciate how they can turn what was a 10-60 minute task into 1 minute of prompting plus 30 seconds of generation/reading to make sure it is valid.
infecto 3y ago
Agree with you there. I find it pretty useful especially for the price and where it might evolve in the future. I am not sure why so many people have hard time getting value out of it.
LeicaLatte 3y ago
To clarify, I have been paying the 10$ since it was released last year. I used it in neovim, vscode and found it helpful. Cut to few months later with OpenAI API release, so many code completers released in many useful form factors willing to go wherever I code even if its Android Studio or Xcode. The product has stagnated completely. I do expect it to change significantly in the future though, might go back then.

tkellogg 3y ago

imo it's the form factor that works. ChatGPT is good for doing standalone things like generating a script. CoPilot is good for making incremental changes to an existing code base. Honestly, its REALLY good at it. One of my favorite parts is that it mimics the style of the surrounding code.

nmfisher 3y ago

Yeah, I was very disappointed. I cancelled my free trial three days in, it was clearly inferior to the alternative (copy+paste into Claude).