6 ms·

Using it in practice, the sheer quantity of suggestions (often one for every line) is fatiguing especially when 99% of the time they seem fine. I posit it beco

Using it in practice, the sheer quantity of suggestions (often one for every line) is fatiguing especially when 99% of the time they seem fine. I posit it becomes increasingly likely over large periods of time over many engineers that severe bug or security issue will be introduced via an AI provided suggestion. This risk to me is inherently different than the risk accepted that engineers will use bad code from Stack Overflow. Even Stack Overflow has social signals (upvotes, comments) that allow even an inexperienced engineer to quickly estimate quality. The amount of code used by engineers from Stack Overflow or blogs etc, is much smaller. Github Copilot is constantly recommending things and does not gives you any social signals lower experienced engineers can use to discern quality or correctness. Even worse, these are suggestions that are written by an AI that does not have any self-preserving motivations.

lolinder 4y ago
Copilot's default behavior is stupid. You can turn off auto-suggest so that it only recommends something when you prompt it to, and that should really be the default behavior. This would encourage more thoughtful use of the tool, and solve the fatigue problem completely. In IntelliJ, disabling auto complete just requires clicking on the Copilot icon in the bottom and disabling it. Alt+\ will then trigger a prompt. I know there's a way to do this in VSCode as well, but I don't know how.
- joenot443 4y ago
  > I know there's a way to do this in VSCode as well, but I don't know how. I dug into this a bit, since I want the same functionality, I found I needed an extension called settings-cycler (https://marketplace.visualstudio.com/items?itemName=hoovercj.vscode-settings-cycler https://marketplace.visualstudio.com/items?itemName=hoovercj...) which lets one flip the 'github.copilot.inlineSuggest.enable' setting on and off with a keybind. Not sure who's in charge of the Copilot extension for VS Code, but if you're out there reading this, the people definitely want this :) Otherwise of course, your tool rocks!
- nprateem 4y ago
  I switched it off and never remember to bother using it. It's obvious why it's enabled by default.
tevon 4y ago
This is a very solid argument. How do we fix that? THIS is the article I want to read!
throwaway675309 4y ago
I would argue that this kind of problem is going to become less of an issue overtime, since they're going to have to also solve the issue of suggesting code samples from deprecated API versions - it's likely that eventually they'll figure out a similar way to promote more secure types of code in the suggestions based on Stack overflow or other types of ranking systems.
- visarga 4y ago
  Yes, the will surely improve a lot and also train users to write better prompts and comments. With millions of users accepting suggestions, then fixing them, they get tons of free labeling. If they monitor the execution errors they got another useful signal. If they use an execution environment they could use reinforcement learning, like AlphaGo, to generate more training data.
redleggedfrog 4y ago
"I posit it becomes increasingly likely over large periods of time over many engineers that severe bug or security issue will be introduced via an AI provided suggestion." I'll go one further with the "Co-pilot is stupid." It's supposed to be artificial intelligence. Why in the eff is it suggesting code with a bug or security issue? Isn't the whole point that it can use that fancy AI to analyze the code and check for those kind of things on top of suggesting code? Half-baked.
- visarga 4y ago
  Ah yes, humans are perfect, never make any mistakes. That's why only AI write bugs.
visarga 4y ago
> I posit it becomes increasingly likely over large periods of time over many engineers that severe bug or security issue will be introduced via an AI provided suggestion. AI can also do code review and documentation helping us reduce the number of bugs. Overall it might actually help.
bGl2YW5j 4y ago
"...does not gives you any social signals lower experienced engineers can use to discern quality or correctness" is very astute. I experienced this in practice. I was pairing with an inexperienced engineer who was using Copilot. He was blindly accepting every Copilot suggestion that came up. When I expressed doubt in the generated code (incorrect logic + unnecessarily complex syntax), he didn't believe me and instead trusted that the AI was right.
visarga 4y ago
As programmers we take pride in being DRY. Copilot is helping us not reinvent the same concept 1000 times. It also makes developers happier, reduces the need to context switch, increases speed and reduces frustration. > Github Copilot is constantly recommending things It's only a momentary problem, will be fixed or worked around. And is this a bad thing to get as many suggestions as you could? I think it's ok as long as you can control its verbiage. > does not gives you any social signals I don't see any reason it could not report on the number of stars and votes the code has received. It's a problem of similarity search between the generated code and the training set, thus finding attribution and having the ability to check votes and even the license. All doable. > an AI that does not have any self-preserving motivations Why touch on that, people have bodies and AIs like Copilot have only training sets. We can explore and do new things, AIs have to watch and learn but never make a move of their own.
- thesuperbigfrog 4y ago
  >> As programmers we take pride in being DRY. Copilot is helping us not reinvent the same concept 1000 times. That's what libraries are for. Copilot is just copy / paste of the code it was trained on. When the code it was trained on is later discovered to have CVEs, will it automatically patch the pasted code? With a library, you can update to the patched version. Copilot has no such feature.
  - lolinder 4y ago
    > Copilot is just copy / paste of the code it was trained on. Every time I hear someone say this, I hear "I've never really tried Copilot, but I have an opinion because I saw something on Twitter." Given the function name for a test and 1-2 examples of tests you've written, Copilot will write the complete test for you, including building complex data structures for the expected value. It correctly uses complex internal APIs that aren't even hosted on GitHub, much less publicly. Given nothing but an `@Test` annotation, it will actually generate complete tests that cover cases you haven't yet covered. There are all kinds of possible attacks on Copilot. If you had said it can copy/paste its training data I wouldn't have argued, but "it just copy/pastes the code it was trained on" is demonstrably false, and anyone who's really tried it will tell you the same thing. EDIT: There's also this fun Copilot use I stumbled across, which I dare you to find in the training data: /** Given this text: Call me Ishmael. Some years ago - never mind how long precisely - having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world. Fill in a JSON structure with my name, how much money I had, and where I'm going: */ { "name": "Ishmael", "money": 0, "destination": "the watery part of the world" }

lolinder 4y ago

Copilot's default behavior is stupid. You can turn off auto-suggest so that it only recommends something when you prompt it to, and that should really be the default behavior. This would encourage more thoughtful use of the tool, and solve the fatigue problem completely. In IntelliJ, disabling auto complete just requires clicking on the Copilot icon in the bottom and disabling it. Alt+\ will then trigger a prompt. I know there's a way to do this in VSCode as well, but I don't know how.

joenot443 4y ago
> I know there's a way to do this in VSCode as well, but I don't know how. I dug into this a bit, since I want the same functionality, I found I needed an extension called settings-cycler (https://marketplace.visualstudio.com/items?itemName=hoovercj.vscode-settings-cycler https://marketplace.visualstudio.com/items?itemName=hoovercj...) which lets one flip the 'github.copilot.inlineSuggest.enable' setting on and off with a keybind. Not sure who's in charge of the Copilot extension for VS Code, but if you're out there reading this, the people definitely want this :) Otherwise of course, your tool rocks!
nprateem 4y ago
I switched it off and never remember to bother using it. It's obvious why it's enabled by default.

tevon 4y ago

This is a very solid argument. How do we fix that? THIS is the article I want to read!

throwaway675309 4y ago

I would argue that this kind of problem is going to become less of an issue overtime, since they're going to have to also solve the issue of suggesting code samples from deprecated API versions - it's likely that eventually they'll figure out a similar way to promote more secure types of code in the suggestions based on Stack overflow or other types of ranking systems.

visarga 4y ago
Yes, the will surely improve a lot and also train users to write better prompts and comments. With millions of users accepting suggestions, then fixing them, they get tons of free labeling. If they monitor the execution errors they got another useful signal. If they use an execution environment they could use reinforcement learning, like AlphaGo, to generate more training data.

redleggedfrog 4y ago

"I posit it becomes increasingly likely over large periods of time over many engineers that severe bug or security issue will be introduced via an AI provided suggestion." I'll go one further with the "Co-pilot is stupid." It's supposed to be artificial intelligence. Why in the eff is it suggesting code with a bug or security issue? Isn't the whole point that it can use that fancy AI to analyze the code and check for those kind of things on top of suggesting code? Half-baked.

visarga 4y ago
Ah yes, humans are perfect, never make any mistakes. That's why only AI write bugs.

visarga 4y ago

> I posit it becomes increasingly likely over large periods of time over many engineers that severe bug or security issue will be introduced via an AI provided suggestion. AI can also do code review and documentation helping us reduce the number of bugs. Overall it might actually help.

bGl2YW5j 4y ago

"...does not gives you any social signals lower experienced engineers can use to discern quality or correctness" is very astute. I experienced this in practice. I was pairing with an inexperienced engineer who was using Copilot. He was blindly accepting every Copilot suggestion that came up. When I expressed doubt in the generated code (incorrect logic + unnecessarily complex syntax), he didn't believe me and instead trusted that the AI was right.

visarga 4y ago

As programmers we take pride in being DRY. Copilot is helping us not reinvent the same concept 1000 times. It also makes developers happier, reduces the need to context switch, increases speed and reduces frustration. > Github Copilot is constantly recommending things It's only a momentary problem, will be fixed or worked around. And is this a bad thing to get as many suggestions as you could? I think it's ok as long as you can control its verbiage. > does not gives you any social signals I don't see any reason it could not report on the number of stars and votes the code has received. It's a problem of similarity search between the generated code and the training set, thus finding attribution and having the ability to check votes and even the license. All doable. > an AI that does not have any self-preserving motivations Why touch on that, people have bodies and AIs like Copilot have only training sets. We can explore and do new things, AIs have to watch and learn but never make a move of their own.

thesuperbigfrog 4y ago
>> As programmers we take pride in being DRY. Copilot is helping us not reinvent the same concept 1000 times. That's what libraries are for. Copilot is just copy / paste of the code it was trained on. When the code it was trained on is later discovered to have CVEs, will it automatically patch the pasted code? With a library, you can update to the patched version. Copilot has no such feature.
- lolinder 4y ago
  > Copilot is just copy / paste of the code it was trained on. Every time I hear someone say this, I hear "I've never really tried Copilot, but I have an opinion because I saw something on Twitter." Given the function name for a test and 1-2 examples of tests you've written, Copilot will write the complete test for you, including building complex data structures for the expected value. It correctly uses complex internal APIs that aren't even hosted on GitHub, much less publicly. Given nothing but an `@Test` annotation, it will actually generate complete tests that cover cases you haven't yet covered. There are all kinds of possible attacks on Copilot. If you had said it can copy/paste its training data I wouldn't have argued, but "it just copy/pastes the code it was trained on" is demonstrably false, and anyone who's really tried it will tell you the same thing. EDIT: There's also this fun Copilot use I stumbled across, which I dare you to find in the training data: /** Given this text: Call me Ishmael. Some years ago - never mind how long precisely - having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world. Fill in a JSON structure with my name, how much money I had, and where I'm going: */ { "name": "Ishmael", "money": 0, "destination": "the watery part of the world" }