4 ms·

Ask HN: How is it going to those who are using Copilot when coding? Or any LLMs?

13 points by sawirricardo 3y ago

SheinhardtWigCo 3y ago
About 30% of the time, Copilot generates excellent suggestions. Often the output is more elegant than whatever I would have done. The other 70%, it generates distracting garbage. I still like it because that trade-off works for me, but it's clearly not for everyone.
wryanzimmerman 3y ago
It’s an interesting novelty sometimes, but difficult to get used to, because it often suggests more than a LSP backed autocomplete but it is never guaranteed to be correct (unlike an LSP backed autocomplete which guarantees a certain level of correctness based on your project setup and language). I have traditional autocomplete and copilot mapped to separate keys in my editor and have learned to treat them separately. Copilot seems to be best for things like refactoring boilerplate within the same file, where it grabs patterns and can expand them, or if you are working “backwards” like you use a variable and then move up in the file to define it (where the LSP would see the error but won’t help you). I never let copilot complete more than the end of my current line because I’ve been burned by very sneaky bugs way too often. It’s easier and less error prone to write code than read it. LSP understands your project but has no idea what you want to do, copilot is like a helpful coworker looking over your shoulder at what you’re doing and suggesting things based on patterns they’ve seen in the past in different projects.
aurareturn 3y ago
It's less useful than my initial impression. The reason is because Copilot does not know about the entire codebase - only the current file. This means it makes a ton of mistakes. In addition, it's often annoying because it replaces the default IDE autocomplete when you don't want it to. For example, when importing code from another file, it disables the autocomplete and then tries to make up what you're trying to import which is wrong 80% of the time as it does not know the whole code base. There is a long running thread asking Github to disable Copilot for import statements. Thus far, Github doesn't care. Link to thread: https://github.com/orgs/community/discussions/8375 https://github.com/orgs/community/discussions/8375
- dtagames 3y ago
  It is not correct to say that Copilot knows only about the current file. If I remember correctly, it tracks the last 50 opened files. In practice, especially with daily use over time, you can see it suggest code, functions, and variables from files quite "far away" in your repo. It's actually tremendously helpful when it suggests exactly what I was about to write anyway.
  - aurareturn 3y ago
    I think it's 20/80 for me. 20% of the time, it produces working code. 80% of the time, it produces errors. Sometimes it's just more annoying to have it enabled. I'm not going to crown Copilot yet until it can read the whole code base.
rcme 3y ago
The most success I’ve had is with libraries I have no experience in. Like I was doing a Stripe integration and asked ChatGPT to generate an example integration. The integration was wrong, but it gave me enough of an idea to get started. I also asked it for a Vulkan compute shader and it generated one mostly correctly, which was helpful.
jejeyyy77 3y ago
It's a decent autocomplete. That's about the extent of it.

13 points by sawirricardo 3y ago

About 30% of the time, Copilot generates excellent suggestions. Often the output is more elegant than whatever I would have done. The other 70%, it generates distracting garbage. I still like it because that trade-off works for me, but it's clearly not for everyone.

wryanzimmerman 3y ago

It’s an interesting novelty sometimes, but difficult to get used to, because it often suggests more than a LSP backed autocomplete but it is never guaranteed to be correct (unlike an LSP backed autocomplete which guarantees a certain level of correctness based on your project setup and language). I have traditional autocomplete and copilot mapped to separate keys in my editor and have learned to treat them separately. Copilot seems to be best for things like refactoring boilerplate within the same file, where it grabs patterns and can expand them, or if you are working “backwards” like you use a variable and then move up in the file to define it (where the LSP would see the error but won’t help you). I never let copilot complete more than the end of my current line because I’ve been burned by very sneaky bugs way too often. It’s easier and less error prone to write code than read it. LSP understands your project but has no idea what you want to do, copilot is like a helpful coworker looking over your shoulder at what you’re doing and suggesting things based on patterns they’ve seen in the past in different projects.

aurareturn 3y ago

It's less useful than my initial impression. The reason is because Copilot does not know about the entire codebase - only the current file. This means it makes a ton of mistakes. In addition, it's often annoying because it replaces the default IDE autocomplete when you don't want it to. For example, when importing code from another file, it disables the autocomplete and then tries to make up what you're trying to import which is wrong 80% of the time as it does not know the whole code base. There is a long running thread asking Github to disable Copilot for import statements. Thus far, Github doesn't care. Link to thread: https://github.com/orgs/community/discussions/8375 https://github.com/orgs/community/discussions/8375

dtagames 3y ago
It is not correct to say that Copilot knows only about the current file. If I remember correctly, it tracks the last 50 opened files. In practice, especially with daily use over time, you can see it suggest code, functions, and variables from files quite "far away" in your repo. It's actually tremendously helpful when it suggests exactly what I was about to write anyway.
- aurareturn 3y ago
  I think it's 20/80 for me. 20% of the time, it produces working code. 80% of the time, it produces errors. Sometimes it's just more annoying to have it enabled. I'm not going to crown Copilot yet until it can read the whole code base.

rcme 3y ago

The most success I’ve had is with libraries I have no experience in. Like I was doing a Stripe integration and asked ChatGPT to generate an example integration. The integration was wrong, but it gave me enough of an idea to get started. I also asked it for a Vulkan compute shader and it generated one mostly correctly, which was helpful.

jejeyyy77 3y ago

It's a decent autocomplete. That's about the extent of it.