5 ms·

I've seen a lot of people ragging on Copilot for "copy+pasting" code - does anyone have links to cases where it has done this without the user intentionally try

I've seen a lot of people ragging on Copilot for "copy+pasting" code - does anyone have links to cases where it has done this without the user intentionally trying to generate a specific (extremely famous) code snippet? I've seen tons of comments here and on Reddit that talk about multiple instances of entire functions being copied verbatim, but the only thing even remotely close to that I've seen is the fast inverse square root, so I must have missed a few tweets or something.

meibo 5y ago
It only seems to if you give it no or very little "source" input, like an empty file with a comment that says "// X algorithm". There's been a lot of bikeshedding on this, but GitHub decidedly hasn't given enough information on how it works and what the training dataset is, and the fair use question definitely needs to be answered, maybe even in court - it's just a matter of time.
- Hamuko 5y ago
  >what the training dataset is All non-private repositories on GitHub.
  - brutal_chaos_ 5y ago
    Is it though? I'd assume that too, but we don't really know, do we? I mean to ask, what have Microsoft stated that leads you to believe this? (Maybe there's a press release I missed?)
    - teraflop 5y ago
      https://twitter.com/NoraDotCodes/status/1412741339771461635 https://twitter.com/NoraDotCodes/status/1412741339771461635
      - ben0x539 5y ago
        (this doesn't say they didn't use other sources too, though)
        tyingq 5y ago
        "source code from publicly available sources, including code in public repositories on GitHub." (from the copilot home page) Pretty much says they did also use other sources.
- MadVikingGod 5y ago
  What I think is interesting and not talked about in the copilot cases is there is actually two different copyright actions that come from using it. The first is if GitHub had a license to distribute the code they have learned on. This is assuming that the code it produces is a derivative, which I don't think would be much of a stretch. For this I think most of the licenses that are used, GPL MIT etc, allow for someone to distribute in such a way. The second is the user of copilot. They would be getting and distributing code where they wouldn't know the original license, but that wouldn't be a viable defense of infringement. To actually comply with most licenses they would have to follow the requirements. In both cases I don't know if a fair use would really apply. Maybe GitHub could stretch the research aspect, but if you just use the code in your product there is no fair use.
  - wlesieutre 5y ago
    > This is assuming that the code it produces is a derivative, which I don't think would be much of a stretch I don't think that's easily settled. Is the code I write a derivative work of all prior code that I've learned from? The difference (IMO) is that Copilot has a much more precise memory than I do; I couldn't recall the exact text of a code example that I've encountered repeatedly even if I wanted to.
    - ric2b 5y ago
      > I couldn't recall the exact text of a code example that I've encountered repeatedly even if I wanted to. If you could it would be copyright infringement anyway.
  - orwin 5y ago
    easy solution: license all the code you write with copilot under AGPL or GPLv3, and you're good.
    - cassonmars 5y ago
      Not quite, there are far more aggressive licenses on repositories on GitHub, such as SSPL, which in theory by providing the SSPL licensed code, Co-Pilot should be open sourced under the same license to comply.
IshKebab 5y ago
Github did an analysis and found that it does do it, though very rarely, and usually when it has little context (e.g. at the start of a file). They're working on detecting those cases though so it doesn't happen accidentally, so it is unlikely to be a realistic problem.
dogecoinbase 5y ago
It's happily spitting out licenses and copyright notices with other people's names on them, it's pretty clearly half-baked.
- sellyme 5y ago
  While that's obviously a UX flaw that definitely shouldn't have made it to release, I find it hard to envision it ever actually being a problem. If someone's accepting auto-generated copyright notices and licenses that don't actually apply, they can't really point the finger at Github - it's called "Copilot", not "Pilot". The problem with full snippets of arbitrary obscure repositories being copy+pasted is that there's no realistic way for a user to know if that's happening without putting in more effort than just writing the code themselves, somewhat defeating the point. That's not really case when the first line of the file contains "(C) Someone Else 2003".
  - abrokenpipe 5y ago
    Thats much more than just a UX flaw, it's clearly showing that copilot just regurgitates code snippets from other peoples opensource. Which draws back to the issue of "did copilot AI actually learn from other peoples code or is it just copy+pasting other peoples code?". I don't think people are concerned about copilot user's liability/productivity.
    - sellyme 5y ago
      > it's clearly showing that copilot just regurgitates code snippets from other peoples opensource You consider a license file to be code?
      - abrokenpipe 5y ago
        That's irrelevant, if copilot is copy+pasting random licenses then it's definitely copy+pasting code as well, I don't see how this could be hard to believe.
tyingq 5y ago
Is it clear how many people have access? If access is fairly limited, I'm not surprised at the low number of examples. Here's an example from their own docs: Compare: https://docs.github.com/assets/images/help/copilot/example_robot.png https://docs.github.com/assets/images/help/copilot/example_r... To: https://github.com/nilavghosh/OMSCS/blob/master/AI4Robotics-8803-001/ProblemSets/ProblemSet1/RunawayRobot/Two/robot.py https://github.com/nilavghosh/OMSCS/blob/master/AI4Robotics-...
- sellyme 5y ago
  > Is it clear how many people have access? If access is fairly limited, I'm not surprised at the low number of examples. I wouldn't be too surprised either, what surprised me was the large number of people who explicitly said words like "many examples" and then couldn't link more than one. > Here's an example from their own docs: Thank you! This is definitely a much stronger example than the Quake one. Looking at the git repository it seems like this is generic startup code for a university assignment, so it probably shows up a significant number of times in the training data. While it definitely makes sense to interpret a piece of code showing up several hundred times in the exact same format as being okay to straight-up copy+paste (e.g., imports, some boilerplate in more verbose languages, keyboard input switch statements), this does seem to highlight that Copilot can't distinguish between code snippets that are always identical because that's just the correct way to do it, and code snippets that are always identical because only one person did that thing, and it just happens to be in hundreds of repositories for one reason or another.
  - tyingq 5y ago
    There's also the example of it spitting out valid sendgrid api keys, which wouldn't be a case of the code appearing in many places. https://news.ycombinator.com/item?id=27736460 https://news.ycombinator.com/item?id=27736460
- geekraver 5y ago
  I wonder how many of these verbatim examples are because the training data code itself occurs multiple times in GitHub because it in turn was copied from an upvoted answer in StackOverflow, LOL.
uberswe 5y ago
I have access and hints for entire functions have only appeared from code already in the same file. It uses the file you are in for context. It also says on their website that the AI may generate api keys that look real but it’s actually just a “fake” key that the AI generated as placeholder. I have had access for one day, spent my entire Saturday playing with while working on an addon for a game. I find it useful and most of the hints come from other code I have in the same files which saves me time or let’s me know when I’m too repetitive :)
- tyingq 5y ago
  "It also says on their website that the AI may generate api keys that look real but it’s actually just a “fake” key that the AI generated as placeholder." Maybe this is what you meant, but that's already been shown to be untrue. https://fossbytes.com/github-copilot-generating-functional-api-keys/ https://fossbytes.com/github-copilot-generating-functional-a...
  - uberswe 5y ago
    So the article mentions an issue reported by dtjm annd the maintainer answers the issue explainig that the keys are fictional. I don't see any statement by the github CEO about this being an issue as the article says. If you have access, link to the maintainers answer: https://github.com/github/copilot-preview/discussions/45#discussioncomment-956912 https://github.com/github/copilot-preview/discussions/45#dis... The answer references the FAQ which is what I referred to: http://copilot.github.com/#faq-does-github-copilot-ever-output-personal-data http://copilot.github.com/#faq-does-github-copilot-ever-outp...
dwild 5y ago
> I've seen a lot of people ragging on Copilot for "copy+pasting" code - does anyone have links to cases where it has done this without the user intentionally trying to generate a specific (extremely famous) code snippet? It's not important that the examples are really specific, the issue is that it does happens. This tool has the potential to infringe copyright (sure it does seems legal at some place, doesn't make it more right though). Look at how EA reversed engineered the Genesis in the past [1]. They had 2 teams, one that did the reversing, and another that did the implementation. That made it safe to say that no infringing code was going through. Plenty of emulators developers try to avoid source code leaks for similar reason. Co-Pilot can't do that. The fact that the copyright status of the code is hard to determine, doesn't means it's not copyrighted code nonetheless. Personally I got nothing against this kind of technology, but I do have a pretty big issue with how it learn. Once people published their code on Github, they didn't know it could have been used to do machine learning and it's bad that it is. If Github asked for the copyright over the code to do Co-Pilot, I wouldn't mind. [1] https://www.youtube.com/watch?v=x0qe1FNqtCo&t=280s https://www.youtube.com/watch?v=x0qe1FNqtCo&t=280s

meibo 5y ago

It only seems to if you give it no or very little "source" input, like an empty file with a comment that says "// X algorithm". There's been a lot of bikeshedding on this, but GitHub decidedly hasn't given enough information on how it works and what the training dataset is, and the fair use question definitely needs to be answered, maybe even in court - it's just a matter of time.

Hamuko 5y ago
>what the training dataset is All non-private repositories on GitHub.
- brutal_chaos_ 5y ago
  Is it though? I'd assume that too, but we don't really know, do we? I mean to ask, what have Microsoft stated that leads you to believe this? (Maybe there's a press release I missed?)
  - teraflop 5y ago
    https://twitter.com/NoraDotCodes/status/1412741339771461635 https://twitter.com/NoraDotCodes/status/1412741339771461635
    - ben0x539 5y ago
      (this doesn't say they didn't use other sources too, though)
      - tyingq 5y ago
        "source code from publicly available sources, including code in public repositories on GitHub." (from the copilot home page) Pretty much says they did also use other sources.
MadVikingGod 5y ago
What I think is interesting and not talked about in the copilot cases is there is actually two different copyright actions that come from using it. The first is if GitHub had a license to distribute the code they have learned on. This is assuming that the code it produces is a derivative, which I don't think would be much of a stretch. For this I think most of the licenses that are used, GPL MIT etc, allow for someone to distribute in such a way. The second is the user of copilot. They would be getting and distributing code where they wouldn't know the original license, but that wouldn't be a viable defense of infringement. To actually comply with most licenses they would have to follow the requirements. In both cases I don't know if a fair use would really apply. Maybe GitHub could stretch the research aspect, but if you just use the code in your product there is no fair use.
- wlesieutre 5y ago
  > This is assuming that the code it produces is a derivative, which I don't think would be much of a stretch I don't think that's easily settled. Is the code I write a derivative work of all prior code that I've learned from? The difference (IMO) is that Copilot has a much more precise memory than I do; I couldn't recall the exact text of a code example that I've encountered repeatedly even if I wanted to.
  - ric2b 5y ago
    > I couldn't recall the exact text of a code example that I've encountered repeatedly even if I wanted to. If you could it would be copyright infringement anyway.
- orwin 5y ago
  easy solution: license all the code you write with copilot under AGPL or GPLv3, and you're good.
  - cassonmars 5y ago
    Not quite, there are far more aggressive licenses on repositories on GitHub, such as SSPL, which in theory by providing the SSPL licensed code, Co-Pilot should be open sourced under the same license to comply.

IshKebab 5y ago

Github did an analysis and found that it does do it, though very rarely, and usually when it has little context (e.g. at the start of a file). They're working on detecting those cases though so it doesn't happen accidentally, so it is unlikely to be a realistic problem.

dogecoinbase 5y ago

It's happily spitting out licenses and copyright notices with other people's names on them, it's pretty clearly half-baked.

sellyme 5y ago
While that's obviously a UX flaw that definitely shouldn't have made it to release, I find it hard to envision it ever actually being a problem. If someone's accepting auto-generated copyright notices and licenses that don't actually apply, they can't really point the finger at Github - it's called "Copilot", not "Pilot". The problem with full snippets of arbitrary obscure repositories being copy+pasted is that there's no realistic way for a user to know if that's happening without putting in more effort than just writing the code themselves, somewhat defeating the point. That's not really case when the first line of the file contains "(C) Someone Else 2003".
- abrokenpipe 5y ago
  Thats much more than just a UX flaw, it's clearly showing that copilot just regurgitates code snippets from other peoples opensource. Which draws back to the issue of "did copilot AI actually learn from other peoples code or is it just copy+pasting other peoples code?". I don't think people are concerned about copilot user's liability/productivity.
  - sellyme 5y ago
    > it's clearly showing that copilot just regurgitates code snippets from other peoples opensource You consider a license file to be code?
    - abrokenpipe 5y ago
      That's irrelevant, if copilot is copy+pasting random licenses then it's definitely copy+pasting code as well, I don't see how this could be hard to believe.

tyingq 5y ago

Is it clear how many people have access? If access is fairly limited, I'm not surprised at the low number of examples. Here's an example from their own docs: Compare: https://docs.github.com/assets/images/help/copilot/example_robot.png https://docs.github.com/assets/images/help/copilot/example_r... To: https://github.com/nilavghosh/OMSCS/blob/master/AI4Robotics-8803-001/ProblemSets/ProblemSet1/RunawayRobot/Two/robot.py https://github.com/nilavghosh/OMSCS/blob/master/AI4Robotics-...

sellyme 5y ago
> Is it clear how many people have access? If access is fairly limited, I'm not surprised at the low number of examples. I wouldn't be too surprised either, what surprised me was the large number of people who explicitly said words like "many examples" and then couldn't link more than one. > Here's an example from their own docs: Thank you! This is definitely a much stronger example than the Quake one. Looking at the git repository it seems like this is generic startup code for a university assignment, so it probably shows up a significant number of times in the training data. While it definitely makes sense to interpret a piece of code showing up several hundred times in the exact same format as being okay to straight-up copy+paste (e.g., imports, some boilerplate in more verbose languages, keyboard input switch statements), this does seem to highlight that Copilot can't distinguish between code snippets that are always identical because that's just the correct way to do it, and code snippets that are always identical because only one person did that thing, and it just happens to be in hundreds of repositories for one reason or another.
- tyingq 5y ago
  There's also the example of it spitting out valid sendgrid api keys, which wouldn't be a case of the code appearing in many places. https://news.ycombinator.com/item?id=27736460 https://news.ycombinator.com/item?id=27736460
geekraver 5y ago
I wonder how many of these verbatim examples are because the training data code itself occurs multiple times in GitHub because it in turn was copied from an upvoted answer in StackOverflow, LOL.

uberswe 5y ago

I have access and hints for entire functions have only appeared from code already in the same file. It uses the file you are in for context. It also says on their website that the AI may generate api keys that look real but it’s actually just a “fake” key that the AI generated as placeholder. I have had access for one day, spent my entire Saturday playing with while working on an addon for a game. I find it useful and most of the hints come from other code I have in the same files which saves me time or let’s me know when I’m too repetitive :)

tyingq 5y ago
"It also says on their website that the AI may generate api keys that look real but it’s actually just a “fake” key that the AI generated as placeholder." Maybe this is what you meant, but that's already been shown to be untrue. https://fossbytes.com/github-copilot-generating-functional-api-keys/ https://fossbytes.com/github-copilot-generating-functional-a...
- uberswe 5y ago
  So the article mentions an issue reported by dtjm annd the maintainer answers the issue explainig that the keys are fictional. I don't see any statement by the github CEO about this being an issue as the article says. If you have access, link to the maintainers answer: https://github.com/github/copilot-preview/discussions/45#discussioncomment-956912 https://github.com/github/copilot-preview/discussions/45#dis... The answer references the FAQ which is what I referred to: http://copilot.github.com/#faq-does-github-copilot-ever-output-personal-data http://copilot.github.com/#faq-does-github-copilot-ever-outp...

dwild 5y ago

> I've seen a lot of people ragging on Copilot for "copy+pasting" code - does anyone have links to cases where it has done this without the user intentionally trying to generate a specific (extremely famous) code snippet? It's not important that the examples are really specific, the issue is that it does happens. This tool has the potential to infringe copyright (sure it does seems legal at some place, doesn't make it more right though). Look at how EA reversed engineered the Genesis in the past [1]. They had 2 teams, one that did the reversing, and another that did the implementation. That made it safe to say that no infringing code was going through. Plenty of emulators developers try to avoid source code leaks for similar reason. Co-Pilot can't do that. The fact that the copyright status of the code is hard to determine, doesn't means it's not copyrighted code nonetheless. Personally I got nothing against this kind of technology, but I do have a pretty big issue with how it learn. Once people published their code on Github, they didn't know it could have been used to do machine learning and it's bad that it is. If Github asked for the copyright over the code to do Co-Pilot, I wouldn't mind. [1] https://www.youtube.com/watch?v=x0qe1FNqtCo&t=280s https://www.youtube.com/watch?v=x0qe1FNqtCo&t=280s