4 ms·

Does anyone here get genuinely useful suggestions from Copilot? I found it amazing for 1 thing: filling up verbose configuration like Terraform or Kubernetes m

Does anyone here get genuinely useful suggestions from Copilot? I found it amazing for 1 thing: filling up verbose configuration like Terraform or Kubernetes manifests. For code, it’s horrible, to the degree that I spend more time rejecting Copilot suggestions or having to extensively modify the ones I accept. I stopped using it couple months ago

zarzavat 2y ago
This is not my experience at all. How are you using it? And in what programming language? The technique to using Copilot (as with any auto-completion) is to keep typing until you get a completion that is equal to what you were going to type. So yes you have to keep rejecting things but the rejection doesn’t take any more effort than normal auto completion.
- budududuroiu 2y ago
  Had this experience across statically typed languages and not. I do admit if your type system is set up correctly, Copilot improves greatly. For me I didn’t find the value I got out of typing 90% of the code I wanted for 10% completion to be worth the $20/month subscription.
  - zarzavat 2y ago
    The reason I ask is that it’s going to perform better if the language you use is well-represented in the training set. Much of the value of Copilot is that it helps you navigate new things that you are not familiar with. For example, I use scipy/numpy a few times a month, so I am competent but not an expert by any means. Copilot is incredibly useful to suggest ways of doing things that are more idiomatic than what I was planning to write. Another way that it is useful is that it massively reduces logical bugs such as off-by-one errors that aren’t caught by typechecking, since it’s unlikely that both me and copilot will make the same off-by-one error at the same time. If only copilot makes an off-by-one error then I will catch it because the suggestion will be different to what I expected. If I am about to make an off-by-one error then copilot will suggest something contrary to my expectation and that forces me to reconsider what I was about to write.
Starlevel004 2y ago
When you remember most people here are Go or JS developers, it makes a lot more sense that they get use out of copilot and co.
langsoul-com 2y ago
To me, it's a nice and fancy intellisense. Great for finishing the line or making comments. Saves tons of time
- everforward 2y ago
  Yes, the comments. My ability to write comments has gotten significantly better because of Copilot. It’s awful but in a helpful way. Like it will suggest “if $argument is empty, this function will return an error”, which is not true because $argument is an int, but it does do something wonky if you give it negative numbers so I remember to add that.
  - the_lucifer 2y ago
    > It’s awful but in a helpful way. Like it will suggest “if $argument is empty, this function will return an error”, which is not true because $argument is an int, but it does do something wonky if you give it negative numbers so I remember to add that. So like an intern/new hire asking what they think are basic questions but lead you to realize an edge case you missed?
drcongo 2y ago
Works well for me in Python, often writing code exactly how I would write it, even to the point of using my own custom packages.
margorczynski 2y ago
I hadn't tried Copilot but Continue + Claude Sonnet worked pretty well for me, really sped up development of the last hobby project I've done in Rust.
haiku2077 2y ago
Yes, it successfully wrote significant portions of an open source Go project I maintain.
- probably_a_gpt 2y ago
  yeah its fantastic for go and python knowing how to trigger it properly is key though, just like all LLMs the context it has determines the quality of its output
- Wheaties466 2y ago
  good to know. I knew it they are good with python in General but I had no idea they were good with go syntax.
anonzzzies 2y ago
We are well over 60% of our team their daily code being written by LLMs, one of them being copilot. We have a lot of custom setup, our on vs code plugins and basically our own ide built over vscode, but it works very well. I believe we will reach 80%+ this year; our current bottleneck is speed (which translates somewhat to money). But definitely our team would be hit pretty hard if we had to drop copilot now.
- lijok 2y ago
  Woah, really? Would love to hear more. Do you have any blog posts or anything about this?
  - anonzzzies 2y ago
    For some of the reasons discussed earlier on hn today (augment) we (not augment but similar) are in stealth mode. Currently we get deals that get us very far; that’s over with when we launch…
- citizenpaul 2y ago
  60% of the team uses LLM? OR 60% of their code is LLM generated? Is the source the stats that it collects like you have accepted 60% of the reccomendatios ive noticed those are rather inflated. Ive yet to see one that deincrements its count even if yoy delete it rigjt after accepting, much less if you have to go back later and delete bunch of stuff it did subtly wrong.
dotnet00 2y ago
I often have to write little scripts in Python or C++ to test/verify things. Copilot is great for handling the setup/boilerplate in such cases. It has made me spend a lot less time trying to modify and come up with ways to debug complex issues when I could just tear out the part I'm working on and have Copilot help me quickly fill in the holes enough to get the torn out portion running independently. It's also great for picking up new packages to do 'simple' tasks like reading in a specific file format.
iLoveOncall 2y ago
Yeah my experience is that, like you said, it's good for configuration, and beside this it only generates utils it has seen before. The thing is I don't need to rewrite a function I've seen before, I'll use a library or framework that anyways offers all of that.
cmpalmer52 2y ago
Yeah, I struggle some days to get my AI assistant to produce useful code for medium to hard problems. On the other hand, if I say “Write a Python program to recursively search through our code files and find or change something,” it will work 95% of the time. My favorite AI success story was a module of c# code that did text overlays on photos. It was written by hand. When we started encountering some issues with the graphics package it was using, I asked the AI (JetBrains in this case) to rewrite the module using a different package. Tedious but relatively straightforward and the results were correct first try. That was what convinced me that (eventually) these type of tools will be great. Meanwhile, I’m so sick of seeing “my apologies, I misunderstood your request…” after it generates suggestions with non-existent methods, bad logic, or missing parameters.
mrjin 2y ago
I guess we are on the same page. It was able to get something right maybe 50% of the time, but it's getting worse as worse and I disabled it months ago.

zarzavat 2y ago

This is not my experience at all. How are you using it? And in what programming language? The technique to using Copilot (as with any auto-completion) is to keep typing until you get a completion that is equal to what you were going to type. So yes you have to keep rejecting things but the rejection doesn’t take any more effort than normal auto completion.

budududuroiu 2y ago
Had this experience across statically typed languages and not. I do admit if your type system is set up correctly, Copilot improves greatly. For me I didn’t find the value I got out of typing 90% of the code I wanted for 10% completion to be worth the $20/month subscription.
- zarzavat 2y ago
  The reason I ask is that it’s going to perform better if the language you use is well-represented in the training set. Much of the value of Copilot is that it helps you navigate new things that you are not familiar with. For example, I use scipy/numpy a few times a month, so I am competent but not an expert by any means. Copilot is incredibly useful to suggest ways of doing things that are more idiomatic than what I was planning to write. Another way that it is useful is that it massively reduces logical bugs such as off-by-one errors that aren’t caught by typechecking, since it’s unlikely that both me and copilot will make the same off-by-one error at the same time. If only copilot makes an off-by-one error then I will catch it because the suggestion will be different to what I expected. If I am about to make an off-by-one error then copilot will suggest something contrary to my expectation and that forces me to reconsider what I was about to write.

Starlevel004 2y ago

When you remember most people here are Go or JS developers, it makes a lot more sense that they get use out of copilot and co.

langsoul-com 2y ago

To me, it's a nice and fancy intellisense. Great for finishing the line or making comments. Saves tons of time

everforward 2y ago
Yes, the comments. My ability to write comments has gotten significantly better because of Copilot. It’s awful but in a helpful way. Like it will suggest “if $argument is empty, this function will return an error”, which is not true because $argument is an int, but it does do something wonky if you give it negative numbers so I remember to add that.
- the_lucifer 2y ago
  > It’s awful but in a helpful way. Like it will suggest “if $argument is empty, this function will return an error”, which is not true because $argument is an int, but it does do something wonky if you give it negative numbers so I remember to add that. So like an intern/new hire asking what they think are basic questions but lead you to realize an edge case you missed?

drcongo 2y ago

Works well for me in Python, often writing code exactly how I would write it, even to the point of using my own custom packages.

margorczynski 2y ago

I hadn't tried Copilot but Continue + Claude Sonnet worked pretty well for me, really sped up development of the last hobby project I've done in Rust.

haiku2077 2y ago

Yes, it successfully wrote significant portions of an open source Go project I maintain.

probably_a_gpt 2y ago
yeah its fantastic for go and python knowing how to trigger it properly is key though, just like all LLMs the context it has determines the quality of its output
Wheaties466 2y ago
good to know. I knew it they are good with python in General but I had no idea they were good with go syntax.

anonzzzies 2y ago

We are well over 60% of our team their daily code being written by LLMs, one of them being copilot. We have a lot of custom setup, our on vs code plugins and basically our own ide built over vscode, but it works very well. I believe we will reach 80%+ this year; our current bottleneck is speed (which translates somewhat to money). But definitely our team would be hit pretty hard if we had to drop copilot now.

lijok 2y ago
Woah, really? Would love to hear more. Do you have any blog posts or anything about this?
- anonzzzies 2y ago
  For some of the reasons discussed earlier on hn today (augment) we (not augment but similar) are in stealth mode. Currently we get deals that get us very far; that’s over with when we launch…
citizenpaul 2y ago
60% of the team uses LLM? OR 60% of their code is LLM generated? Is the source the stats that it collects like you have accepted 60% of the reccomendatios ive noticed those are rather inflated. Ive yet to see one that deincrements its count even if yoy delete it rigjt after accepting, much less if you have to go back later and delete bunch of stuff it did subtly wrong.

dotnet00 2y ago

I often have to write little scripts in Python or C++ to test/verify things. Copilot is great for handling the setup/boilerplate in such cases. It has made me spend a lot less time trying to modify and come up with ways to debug complex issues when I could just tear out the part I'm working on and have Copilot help me quickly fill in the holes enough to get the torn out portion running independently. It's also great for picking up new packages to do 'simple' tasks like reading in a specific file format.

iLoveOncall 2y ago

Yeah my experience is that, like you said, it's good for configuration, and beside this it only generates utils it has seen before. The thing is I don't need to rewrite a function I've seen before, I'll use a library or framework that anyways offers all of that.

cmpalmer52 2y ago

Yeah, I struggle some days to get my AI assistant to produce useful code for medium to hard problems. On the other hand, if I say “Write a Python program to recursively search through our code files and find or change something,” it will work 95% of the time. My favorite AI success story was a module of c# code that did text overlays on photos. It was written by hand. When we started encountering some issues with the graphics package it was using, I asked the AI (JetBrains in this case) to rewrite the module using a different package. Tedious but relatively straightforward and the results were correct first try. That was what convinced me that (eventually) these type of tools will be great. Meanwhile, I’m so sick of seeing “my apologies, I misunderstood your request…” after it generates suggestions with non-existent methods, bad logic, or missing parameters.

mrjin 2y ago

I guess we are on the same page. It was able to get something right maybe 50% of the time, but it's getting worse as worse and I disabled it months ago.