4 ms·

I finally decided to install Copilot recently now that they have the business subscription with a more sensible privacy policy, and it's so much better than exp

I finally decided to install Copilot recently now that they have the business subscription with a more sensible privacy policy, and it's so much better than expected. Even on internal code it hasn't been trained against it often just works anyway based on the context and other nearby code. If something requires even a slight bit of boilerplate that has a pattern to it, it writes it much faster than I could. I find it almost uncanny sometimes how well it knows what I wanted to do next even without writing any comment telling it what to do. Not to mention the hundreds of random small suggestions like writing similar method definitions, completing all of the parameters I was going to pass to something, writing out the rest of some math formula after I typed the name of the variable for the result I want to compute, etc. People weren't kidding about the productivity boost. On the other hand, I found trying to get ChatGPT to write large chunks of code almost completely useless - it would always be incorrect in some subtle way or failing to use the style I gave it as an example, unless the task is to write some simple python scripts.

moltar 3y ago
I kind of agree with you and I followed almost the same path. Excerpt one thing I noticed. That the times it is subtly wrong I waste a ton of time finding the bug. It’s like looking for bugs in someone else’s code. I lack the context and muscle memory of the code written by the assistant. I’m still not sure if it’s net positive. I wish there was an objective way to measure the effect on a personal, project and org level.
- cageface 3y ago
  This was my experience as well. The time it saved me didn't add up to the time I spent in the debugger fixing subtle bugs in code that looked fine but didn't quite work. I think it would be very easy to ship hard to find bugs in production code with copilot.
  - macNchz 3y ago
    I think there’s a “eyes glaze over” effect with generating lots of boilerplate, where things look generally correct, but have issues you need to be paying attention to notice and correct. One example recently: I watched a developer scaffolding a database schema with an ORM. Plenty of boilerplate defining the fields and stuff, nice to have Copilot running, however the suggestions were baking in bad design that would work now, but almost certainly cause pain later: nullable foreign keys, text fields with inappropriate max lengths, redundant FK relationships where a relationship could be ambiguous depending on whether you joined directly on the FK or through another related table, etc. Obviously all of that is on the developer to notice and be responsible for, but it’s really easy to just tab through the completions, see that it works (right now) and move on.

moltar 3y ago

I kind of agree with you and I followed almost the same path. Excerpt one thing I noticed. That the times it is subtly wrong I waste a ton of time finding the bug. It’s like looking for bugs in someone else’s code. I lack the context and muscle memory of the code written by the assistant. I’m still not sure if it’s net positive. I wish there was an objective way to measure the effect on a personal, project and org level.

cageface 3y ago
This was my experience as well. The time it saved me didn't add up to the time I spent in the debugger fixing subtle bugs in code that looked fine but didn't quite work. I think it would be very easy to ship hard to find bugs in production code with copilot.
- macNchz 3y ago
  I think there’s a “eyes glaze over” effect with generating lots of boilerplate, where things look generally correct, but have issues you need to be paying attention to notice and correct. One example recently: I watched a developer scaffolding a database schema with an ORM. Plenty of boilerplate defining the fields and stuff, nice to have Copilot running, however the suggestions were baking in bad design that would work now, but almost certainly cause pain later: nullable foreign keys, text fields with inappropriate max lengths, redundant FK relationships where a relationship could be ambiguous depending on whether you joined directly on the FK or through another related table, etc. Obviously all of that is on the developer to notice and be responsible for, but it’s really easy to just tab through the completions, see that it works (right now) and move on.