8 ms·

I think the biggest problem copilot will have in practice gaining traction is that verifying correctness isn’t any faster than writing the code yourself in many

by nojs 5y ago

I think the biggest problem copilot will have in practice gaining traction is that verifying correctness isn’t any faster than writing the code yourself in many cases. The Easter(y) function is a classic example - it would be way faster to write that than to try and verify that there’s no subtle bugs. Copilot is by design trying to give you something that _looks_ correct without caring whether it actually is - so it optimises for real-looking but subtly buggy code, which is the worst kind of broken code.

mattwad 5y ago
I've been using it and that's 100% correct. If it suggests more than a few lines, i might as well do it myself. However, it has been an awesome Intellisense tool for one-liners.. it can write out the rest of a comment or a simple map/filter method just fine. I don't think it will ever go further than that, but nor does it need to.
- visarga 5y ago
  Be sure it doesn't generate any GPL licensed code down to the size of a single letter, to be safe. The reaction to GPL inspired snippet generation has been more fierce than I could have imagined, even though usable snippets are so short.
  - z3t4 5y ago
    You can't license a code snippet. You can only copyright/license a complete work, as in a complete app or library. Software patterns might work differently though!?
    - dec0dedab0de 5y ago
      Any portion of a work has a copyright the moment there is a tangible copy. There is a uniqueness requirement, but it has nothing to do with length. A unique one liner would have a copyright.
      - z3t4 5y ago
        You can not take the source code of an app, change a few lines, and then call it your own. You can however make your own app by stealing a few lines of code here and there... funny enough many companies write code so that all functions depend on all other functions making it impossible to reuse any parts of the code in another app (but i think that is not deliberate)
nimbius 5y ago
see when I saw the words 'risk assessment' i figured the presuppositional framework of the authors argument wasnt that copilot was legally sound. In other words, i didnt expect to jump straight to the technical validity of the product. do not ignore the elephant in the room. copilot is stealing code from projects with open licenses.
- ShroudedNight 5y ago
  I would expect an information security expert to comment on the risks they have a professional background in assessing. More broadly, your comment almost seems to suggest that one should preclude all avenues of criticism beyond whichever singular issue is most "obvious" / "problematic". That strikes me as less than optimal.
mumblemumble 5y ago
Years ago, I ran into a similar problem working on a program that was doing named entity recognition to assist humans with data entry. We found that, for our purposes, there seemed to be no (realistic) accuracy threshold beyond which the tool would save clients money, because double-checking the machine-generated output was inherently more work than doing it by hand. So we pivoted the product to being something you would run on full auto, for situations where you didn't need a high level of quality. I'm not sure if that option is available to programmers, though.
- jpeloquin 5y ago
  Maybe Copilot could be turned into a context-aware search engine? That is, invoking it would return a list of examples that it thinks do the same thing as what you're trying to do, based on your work-in-progress code.
  - visarga 5y ago
    Seems like humans are more creative when they can get inspiration from language models. https://arxiv.org/abs/2107.04007 https://arxiv.org/abs/2107.04007
    - bshanks 5y ago
      Ironic if AIs end up being more useful for creativity than for detail-oriented tasks.
      - jessmartin 5y ago
        I honestly think this is where things are going. Man-machine partnership on creative tasks. Not only is it more amenable to current models, it’s higher leverage and less likely to be completely automated away. I wrote a little about this shift to what I think of as “conversational programming”: https://jessmart.in/articles/copilot https://jessmart.in/articles/copilot
      - atatatat 5y ago
intuitionist 5y ago
I think my best guess is that this is actually meant to produce broken code, so that Microsoft can sell you additional services (cloud fuzzing?) to find and fix the bugs.
intothev01d 5y ago
Yea, that's the major problem. I'd prefer just some sort of inline helper that could point directly to documentation, topics, stack overflow answers that might be helpful for whatever I'm developing. An enhanced "Intellisense" or something. That to me, is better, because ultimately it's up to the developer to place scrutiny on the solution. You basically can't blindly accept the implementation which makes this just a constant code review, of yourself? I dunno. This just seems half-baked
- visarga 5y ago
  Copilot does approximately that, but instead of showing you code that's written independently it adapts to your variable names and context.
tyingq 5y ago
It also isn't giving you any information on the source(s) of the generated code. Which might help determine how much to trust it, whether it could have licensing issues, etc.
- visarga 5y ago
  It's probably the only way it would work - to show top matching snippets from training data on request, with links to the source and ideally licensing information, if it can be gleaned automatically. This would also clearly show how much it is copying verbatim and what exactly is its contribution. The funny part will be when all the human programmers who steal code will get doxed as a side effect. It shine a light on lots of skeletons in the closet.
pessimizer 5y ago
> I think the biggest problem copilot will have in practice gaining traction is that verifying correctness isn’t any faster than writing the code yourself in many cases. Humorously, this is a similar problem to the one autonomous driving has. Being alert when something goes wrong randomly is more difficult than being alert all of the time.
- slg 5y ago
  However in the real world people don't always write bugless code and aren't always alert when driving. Therefore these AI assistants can still have a net positive result as long as they are better than the average performance of a human. Of course three quarters of us probably believe that "I'm not an average programmer so Copilot would only make me worse." Personally I think the more interesting angle is the trolley problem this creates. People will die in self-driving car accidents and bugs will exist in AI generated code. Those people and bugs are different than the people who will die in human caused accidents and the bugs in human written code. If the number and severity of the results are lessened by the computer, are we willing to forgive the damage directly caused by the AI that falls short of perfection?
  - tkiolp4 5y ago
    I’m a very mediocre developer and if Copilot is any better than me at writing code, then I will have a hard time understanding whatever Copilot throws at me. I cannot just save, commit and push whatever Copilot suggests... so it’s faster if I write the code myself than to review Copilot’s code.
    - slg 5y ago
      > I cannot just save, commit and push whatever Copilot suggests I don't think that is the goal just like the goal of the current generation of self-driving cars isn't for you to be able to take a nap in the driver's seat. Imagine you need some code that would have traditionally taken you and hour to write. I believe the goal of Copilot is to generate the code for you as a starting point. Maybe you don't understand that code immediately and it takes you 20 minutes to figure out what is going on. Then you spend another 20 minutes tweaking it for your exact purpose. If that results in code of similar quality to what you would have written alone, then Copilot makes you more efficient by saving you 20 minutes.
elcomet 5y ago
It works well for completing lines one by one. I just write a few characters and it can often complete very well the lines. Of course it will not write complete functions correctly.
ikiris 5y ago
I can think of multiple cases where the best result is probably volume of output that looks good without close scrutiny.
corpdrone2021 5y ago
I feel like a mix of hand written test cases and copilot generated code might go somewhere, but I think you've got the basic problem sorted out. I'd much rather type an algorithm in from scratch than wrap my head around whatever copilot spits out.
- dkersten 5y ago
  I had an idea long ago that you basically write unit tests (nowadays I would add property-based tests to the mix too) and a genetic algorithm (best I could come up with at the time, nowadays we obviously have much fancier techniques, as evidenced by Copilot) would come up with code to try and make the tests pass. I could see Copilot used in such a way. I think the interaction would have to change though: force the user to give it the tests as input, not give it some basic instruction, have it generate code, and then I try to write tests after. The tests should be the spec that Copilot uses to generate its output. Right now, I'm not excited about Copilot. Like you say, understanding what Copilot spits out is difficult and I suspect more error prone than just writing it yourself (since we often see what we want to see and can overlook even glaring mistakes). I'm also not excited about them ignoring the licenses of the code they trained on. But I can imagine a future iteration that generated code to pass some tests that I could get excited about.
  - corpdrone2021 5y ago
    It seems to me that "generate the code that makes these unit tests pass" is actually a much saner engineering task than "go from a comments to an implmentation"
fighterpilot 5y ago
I can foresee one niche where this doesn't matter: exploratory ad-hoc data science. In this exploration stage, total correctness doesn't matter since you're just getting a feel for the data. Copilot might help a lot with the associated boilerplate.
- jessmartin 5y ago
  I think Copilot-like tools could be excellent for the exploration phase. Marvin Minsky mused on this usage back in 1967: > The programmer does not even have to be exact in his own ideas‑he may have a range of acceptable computer answers in mind and may be content if the computer's answers do not step out of this range. The programmer does not have to fixate the computer with particular processes. In a range of uncertainty he may ask the computer to generate new procedures, or he may recommend rules of selection and give the computer advice about which choices to make. Thus, computers do not have to be programmed with extremely clear and precise formulations of what is to be executed, or how to do it. From: https://web.media.mit.edu/~minsky/papers/Why%20programming%20is--.html https://web.media.mit.edu/~minsky/papers/Why%20programming%2...
ForHackernews 5y ago
Producing code that kinda-mostly works, very quickly, is the behaviour the software industry optimises for. This tool will help do more of that, so it will be very widely adopted. Developers who not use this (or similar tools) will not be hired, or only in particular niche domains where correctness matters.
tkiolp4 5y ago
Agree. It’s way easier to write a function from scratch than to read/evaluate/fix whatever snippet Copilot throws at me. Replace Copilot with “junior engineer” or “senior engineer that knows more than me” and the result is the same (the junior engineer will probably introduce couple of subtle bugs that are hard to find; the senior engineer would write code in such a way that my mediocre brain won’t understand).
- jeltz 5y ago
  The best senior developers I have worked with knew how to keep code simple to understand for less skilled developers.
TameAntelope 5y ago
"Subtly buggy" is another way of saying "mostly working", and I'll take "mostly working" any day of the week!
- gspr 5y ago
  You must not be coding anything important.
  - TameAntelope 5y ago
    Ouch, but I'm not interesting in writing software as much as I'm interested in making enough money to spend the rest of my days sipping piña coladas on a beach in a foreign country. All I really need is for the product to work well enough that I can fundraise and hire someone who's better at programming than I am, someone who hopefully doesn't write comments about how unimportant other people's work is on HN.
    - gspr 5y ago
      OK, that's fair enough, you're not interested in the engineering side of things – but then expect blowback if you decide to comment on exactly that.
      - TameAntelope 5y ago
        I am interested in the engineering side of things, for sure, but only insofar as their actual outcomes and justification as part of a value-add to a business. GitHub Copilot, if it can create "mostly working" code, is a huge value-add to an early business trying to find product/market fit because it cuts down on the time it takes to prototype and generate early versions. Not every piece of software has to work perfectly every time, and I don't think that's the standard to which we should hold any automated coding tool.
- ohazi 5y ago
  More likely: "inscrutably buggy" It looks to you like it should work, but it doesn't, and you can't figure out why. That's not "mostly working," that's a frustrating waste of time. It's hard enough to notice when you accidentally swap `i` and `j` -- why would you want to make your life even more miserable by spending your time finding all of the instances where a pattern matching robot has done something similar in an unfamiliar block? And if you do happen to get "mostly working" code, but only want it to stay together long enough for you to fundraise, you're basically stating that you plan on foisting this technical debt onto the poor sod you happen to hire. Attitudes like yours are the reason this dogpile scares me.
shinjitsu 5y ago
> give you something that _looks_ correct without caring whether it actually is Oh this is going to make teaching intro to computer science sooooo 'interesting'. It wouldn't be so bad if the students looked at the generated code and understood it, but experience tells me most of them will not.
- staticassertion 5y ago
  Students will always cheat tbh. I doubt this will radically change anything with regards to that.
908B64B197 5y ago
I can't wait for coders who used copilot for all coding projects they did. Copy and pasting snippets until it works. No proofs or real exams at bootcamps! Having offline coding interviews to find Software Engineers will become even more important.
cyanydeez 5y ago
just use it to talso write the tests, problem solved!

by nojs 5y ago

mattwad 5y ago

I've been using it and that's 100% correct. If it suggests more than a few lines, i might as well do it myself. However, it has been an awesome Intellisense tool for one-liners.. it can write out the rest of a comment or a simple map/filter method just fine. I don't think it will ever go further than that, but nor does it need to.

visarga 5y ago
Be sure it doesn't generate any GPL licensed code down to the size of a single letter, to be safe. The reaction to GPL inspired snippet generation has been more fierce than I could have imagined, even though usable snippets are so short.
- z3t4 5y ago
  You can't license a code snippet. You can only copyright/license a complete work, as in a complete app or library. Software patterns might work differently though!?
  - dec0dedab0de 5y ago
    Any portion of a work has a copyright the moment there is a tangible copy. There is a uniqueness requirement, but it has nothing to do with length. A unique one liner would have a copyright.
    - z3t4 5y ago
      You can not take the source code of an app, change a few lines, and then call it your own. You can however make your own app by stealing a few lines of code here and there... funny enough many companies write code so that all functions depend on all other functions making it impossible to reuse any parts of the code in another app (but i think that is not deliberate)

nimbius 5y ago

see when I saw the words 'risk assessment' i figured the presuppositional framework of the authors argument wasnt that copilot was legally sound. In other words, i didnt expect to jump straight to the technical validity of the product. do not ignore the elephant in the room. copilot is stealing code from projects with open licenses.

ShroudedNight 5y ago
I would expect an information security expert to comment on the risks they have a professional background in assessing. More broadly, your comment almost seems to suggest that one should preclude all avenues of criticism beyond whichever singular issue is most "obvious" / "problematic". That strikes me as less than optimal.

mumblemumble 5y ago

Years ago, I ran into a similar problem working on a program that was doing named entity recognition to assist humans with data entry. We found that, for our purposes, there seemed to be no (realistic) accuracy threshold beyond which the tool would save clients money, because double-checking the machine-generated output was inherently more work than doing it by hand. So we pivoted the product to being something you would run on full auto, for situations where you didn't need a high level of quality. I'm not sure if that option is available to programmers, though.

jpeloquin 5y ago
Maybe Copilot could be turned into a context-aware search engine? That is, invoking it would return a list of examples that it thinks do the same thing as what you're trying to do, based on your work-in-progress code.
- visarga 5y ago
  Seems like humans are more creative when they can get inspiration from language models. https://arxiv.org/abs/2107.04007 https://arxiv.org/abs/2107.04007
  - bshanks 5y ago
    Ironic if AIs end up being more useful for creativity than for detail-oriented tasks.
    - jessmartin 5y ago
      I honestly think this is where things are going. Man-machine partnership on creative tasks. Not only is it more amenable to current models, it’s higher leverage and less likely to be completely automated away. I wrote a little about this shift to what I think of as “conversational programming”: https://jessmart.in/articles/copilot https://jessmart.in/articles/copilot
    - atatatat 5y ago

intuitionist 5y ago

I think my best guess is that this is actually meant to produce broken code, so that Microsoft can sell you additional services (cloud fuzzing?) to find and fix the bugs.

intothev01d 5y ago

Yea, that's the major problem. I'd prefer just some sort of inline helper that could point directly to documentation, topics, stack overflow answers that might be helpful for whatever I'm developing. An enhanced "Intellisense" or something. That to me, is better, because ultimately it's up to the developer to place scrutiny on the solution. You basically can't blindly accept the implementation which makes this just a constant code review, of yourself? I dunno. This just seems half-baked

visarga 5y ago
Copilot does approximately that, but instead of showing you code that's written independently it adapts to your variable names and context.

tyingq 5y ago

It also isn't giving you any information on the source(s) of the generated code. Which might help determine how much to trust it, whether it could have licensing issues, etc.

visarga 5y ago
It's probably the only way it would work - to show top matching snippets from training data on request, with links to the source and ideally licensing information, if it can be gleaned automatically. This would also clearly show how much it is copying verbatim and what exactly is its contribution. The funny part will be when all the human programmers who steal code will get doxed as a side effect. It shine a light on lots of skeletons in the closet.

pessimizer 5y ago

> I think the biggest problem copilot will have in practice gaining traction is that verifying correctness isn’t any faster than writing the code yourself in many cases. Humorously, this is a similar problem to the one autonomous driving has. Being alert when something goes wrong randomly is more difficult than being alert all of the time.

slg 5y ago
However in the real world people don't always write bugless code and aren't always alert when driving. Therefore these AI assistants can still have a net positive result as long as they are better than the average performance of a human. Of course three quarters of us probably believe that "I'm not an average programmer so Copilot would only make me worse." Personally I think the more interesting angle is the trolley problem this creates. People will die in self-driving car accidents and bugs will exist in AI generated code. Those people and bugs are different than the people who will die in human caused accidents and the bugs in human written code. If the number and severity of the results are lessened by the computer, are we willing to forgive the damage directly caused by the AI that falls short of perfection?
- tkiolp4 5y ago
  I’m a very mediocre developer and if Copilot is any better than me at writing code, then I will have a hard time understanding whatever Copilot throws at me. I cannot just save, commit and push whatever Copilot suggests... so it’s faster if I write the code myself than to review Copilot’s code.
  - slg 5y ago
    > I cannot just save, commit and push whatever Copilot suggests I don't think that is the goal just like the goal of the current generation of self-driving cars isn't for you to be able to take a nap in the driver's seat. Imagine you need some code that would have traditionally taken you and hour to write. I believe the goal of Copilot is to generate the code for you as a starting point. Maybe you don't understand that code immediately and it takes you 20 minutes to figure out what is going on. Then you spend another 20 minutes tweaking it for your exact purpose. If that results in code of similar quality to what you would have written alone, then Copilot makes you more efficient by saving you 20 minutes.

elcomet 5y ago

It works well for completing lines one by one. I just write a few characters and it can often complete very well the lines. Of course it will not write complete functions correctly.

ikiris 5y ago

I can think of multiple cases where the best result is probably volume of output that looks good without close scrutiny.

corpdrone2021 5y ago

I feel like a mix of hand written test cases and copilot generated code might go somewhere, but I think you've got the basic problem sorted out. I'd much rather type an algorithm in from scratch than wrap my head around whatever copilot spits out.

dkersten 5y ago
I had an idea long ago that you basically write unit tests (nowadays I would add property-based tests to the mix too) and a genetic algorithm (best I could come up with at the time, nowadays we obviously have much fancier techniques, as evidenced by Copilot) would come up with code to try and make the tests pass. I could see Copilot used in such a way. I think the interaction would have to change though: force the user to give it the tests as input, not give it some basic instruction, have it generate code, and then I try to write tests after. The tests should be the spec that Copilot uses to generate its output. Right now, I'm not excited about Copilot. Like you say, understanding what Copilot spits out is difficult and I suspect more error prone than just writing it yourself (since we often see what we want to see and can overlook even glaring mistakes). I'm also not excited about them ignoring the licenses of the code they trained on. But I can imagine a future iteration that generated code to pass some tests that I could get excited about.
- corpdrone2021 5y ago
  It seems to me that "generate the code that makes these unit tests pass" is actually a much saner engineering task than "go from a comments to an implmentation"

fighterpilot 5y ago

I can foresee one niche where this doesn't matter: exploratory ad-hoc data science. In this exploration stage, total correctness doesn't matter since you're just getting a feel for the data. Copilot might help a lot with the associated boilerplate.

jessmartin 5y ago
I think Copilot-like tools could be excellent for the exploration phase. Marvin Minsky mused on this usage back in 1967: > The programmer does not even have to be exact in his own ideas‑he may have a range of acceptable computer answers in mind and may be content if the computer's answers do not step out of this range. The programmer does not have to fixate the computer with particular processes. In a range of uncertainty he may ask the computer to generate new procedures, or he may recommend rules of selection and give the computer advice about which choices to make. Thus, computers do not have to be programmed with extremely clear and precise formulations of what is to be executed, or how to do it. From: https://web.media.mit.edu/~minsky/papers/Why%20programming%20is--.html https://web.media.mit.edu/~minsky/papers/Why%20programming%2...

ForHackernews 5y ago

Producing code that kinda-mostly works, very quickly, is the behaviour the software industry optimises for. This tool will help do more of that, so it will be very widely adopted. Developers who not use this (or similar tools) will not be hired, or only in particular niche domains where correctness matters.

tkiolp4 5y ago

Agree. It’s way easier to write a function from scratch than to read/evaluate/fix whatever snippet Copilot throws at me. Replace Copilot with “junior engineer” or “senior engineer that knows more than me” and the result is the same (the junior engineer will probably introduce couple of subtle bugs that are hard to find; the senior engineer would write code in such a way that my mediocre brain won’t understand).

jeltz 5y ago
The best senior developers I have worked with knew how to keep code simple to understand for less skilled developers.

TameAntelope 5y ago

"Subtly buggy" is another way of saying "mostly working", and I'll take "mostly working" any day of the week!

gspr 5y ago
You must not be coding anything important.
- TameAntelope 5y ago
  Ouch, but I'm not interesting in writing software as much as I'm interested in making enough money to spend the rest of my days sipping piña coladas on a beach in a foreign country. All I really need is for the product to work well enough that I can fundraise and hire someone who's better at programming than I am, someone who hopefully doesn't write comments about how unimportant other people's work is on HN.
  - gspr 5y ago
    OK, that's fair enough, you're not interested in the engineering side of things – but then expect blowback if you decide to comment on exactly that.
    - TameAntelope 5y ago
      I am interested in the engineering side of things, for sure, but only insofar as their actual outcomes and justification as part of a value-add to a business. GitHub Copilot, if it can create "mostly working" code, is a huge value-add to an early business trying to find product/market fit because it cuts down on the time it takes to prototype and generate early versions. Not every piece of software has to work perfectly every time, and I don't think that's the standard to which we should hold any automated coding tool.
ohazi 5y ago
More likely: "inscrutably buggy" It looks to you like it should work, but it doesn't, and you can't figure out why. That's not "mostly working," that's a frustrating waste of time. It's hard enough to notice when you accidentally swap `i` and `j` -- why would you want to make your life even more miserable by spending your time finding all of the instances where a pattern matching robot has done something similar in an unfamiliar block? And if you do happen to get "mostly working" code, but only want it to stay together long enough for you to fundraise, you're basically stating that you plan on foisting this technical debt onto the poor sod you happen to hire. Attitudes like yours are the reason this dogpile scares me.

shinjitsu 5y ago

> give you something that _looks_ correct without caring whether it actually is Oh this is going to make teaching intro to computer science sooooo 'interesting'. It wouldn't be so bad if the students looked at the generated code and understood it, but experience tells me most of them will not.

staticassertion 5y ago
Students will always cheat tbh. I doubt this will radically change anything with regards to that.

908B64B197 5y ago

I can't wait for coders who used copilot for all coding projects they did. Copy and pasting snippets until it works. No proofs or real exams at bootcamps! Having offline coding interviews to find Software Engineers will become even more important.

cyanydeez 5y ago

just use it to talso write the tests, problem solved!