9 ms·

Research recitation: A first look at rote learning in GitHub Copilot suggestions

184 points by azhenley 5y ago

thunderbird120 5y ago
It should be noted that this kind of behavior is entirely expected from a GPT-style self-supervised sequence model. Rote memorization for this kind of model is indicative of correct training, not overfitting. The underlying training objective of these models ideally results in a representation of the training data which allows complete samples to be extracted by using partial samples as keys. Actual overfitting in this kind of model requires absurd parameter counts. See https://tilde.town/~fessus/reward_is_unnecessary.pdf https://tilde.town/~fessus/reward_is_unnecessary.pdf
amelius 5y ago
Does it substitute variables correctly? E.g. if I define max(a,b) or max(x,y) does it complete the definition with the right variable names?
- onionisafruit 5y ago
  Generally, yes. It’s not guaranteed to do that correctly, but I’ve not seen it get variable names wrong so far.
  - tyingq 5y ago
    There's an example called "fetch_tweets.py" at the bottom of page on https://copilot.github.com/ https://copilot.github.com/ that gets it wrong: def fetch_tweets_from_user(user_name): # deleted some lines here... # fetch tweets tweets = api.user_timeline(screen_name=user, count=200, include_rts=False) screen_name=user there isn't right. It's a nit, but it is interesting how many of the hand-picked examples on that page aren't right. Since they were hand-picked, presumably to show the product off.
    - deleted 5y ago
      [deleted]
  - the8472 5y ago
    > It’s not guaranteed to do that correctly Which is odd considering they could run this as beam search with the checking part of a compiler in the loop.
    - 0-_-0 5y ago
      Which would be extremely slow
      - the8472 5y ago
        Not necessarily. If you emit a batch of suggestions for 1 line and prune them with syntax checks you can already prune them and present them to the user while working on the next iterations. The real difficulty is getting good checking on partial source code, some languages have better tooling for that than others.
blamestross 5y ago
Take aways in summary: - Copilot does sometimes rote copy for nontrivial situations - Mostly this happens when there isn't much context to go on. - Provided an empty file, it proceeds to recommend to write the GPL - They will add a "recitation detector" to copilot to indicate non-novel recommendations By the standards of corp-speak this is pretty good, they admit there is a problem there and they intend to do something tractable to prevent it. This entire copilot situation is far enough outside my personal "mental ethics model" that I'm personally abstaining form taking a stance until I have had a lot more time to think and learn about it.
- temac 5y ago
  They quite misdirect on what the problem actually is. Verbatim or quasi-verbatim recitations are not the only things copyright protects.
ionwake 5y ago
Sorry for the basic question, but the code one builds on this platform is saved to githubs servers?
- k__ 5y ago
  yes
- rvz 5y ago
  Anything pasted or typed into that Copilot editor is sent to GitHub as 'telemetry'. > In order to generate suggestions, GitHub Copilot transmits part of the file you are editing to the service. So Yes.
  - astrange 5y ago
    That's not telemetry, it's the prompt to the model to generate the rest of the text. I would assume it's not saved.
    - onionisafruit 5y ago
      I wouldn't think to call that telemetry either, but it is addressed in this doc about telemetry: https://docs.github.com/en/github/copilot/telemetry-terms#additional-telemetry https://docs.github.com/en/github/copilot/telemetry-terms#ad...
- Hamuko 5y ago
  Don't know about saved, but definitely sent. I guess one risk after knowing that Github regards all source code regardless of its license to be fair game for training Copilot is that you probably can't know for sure that your new code is not being used to teach the model more.
  - onionisafruit 5y ago
    It's clear the author of this article had access to the code that triggered the copilot suggestions. They also say this was from an internal trial of copilot, so it might be that these trial users were told their code could be seen by their coworkers.
devinplatt 5y ago
The article is worth reading, but a good summary is at the bottom: > This investigation demonstrates that GitHub Copilot can quote a body of code verbatim, but that it rarely does so, and when it does, it mostly quotes code that everybody quotes, and mostly at the beginning of a file, as if to break the ice. But there’s still one big difference between GitHub Copilot reciting code and me reciting a poem: I know when I’m quoting. I would also like to know when Copilot is echoing existing code rather than coming up with its own ideas. That way, I’m able to look up background information about that code, and to include credit where credit is due. The answer is obvious: sharing the prefiltering solution we used in this analysis to detect overlap with the training set. When a suggestion contains snippets copied from the training set, the UI should simply tell you where it’s quoted from. You can then either include proper attribution or decide against using that code altogether. This duplication search is not yet integrated into the technical preview, but we plan to do so. And we will both continue to work on decreasing rates of recitation, and on making its detection more precise.
- sillysaurusx 5y ago
  The answer wasn’t obvious to me. Nice solution. It sounds like you’re a part of the Copilot team. If so, then I’m happy to see the Copilot team cares about these issues at all. I was expecting nothing but stonewall until the conversation died out, since realistically the chance of the EFF bringing or winning a lawsuit seems small. (And who else would try?) But when you anger the world and being so much attention to this delicate issue of copyright in AI, you risk every hobbyist. Suppose the world decides that AI models need to be restricted. Now every person who wants to get into AI will need to deal with it. I’m not sure anyone else cares, but I care, because it’s the difference between someone getting into woodworking (an unrestricted hobby) vs becoming a lawyer or doctor (the maximally restrictive hobby). The closer we are to the latter, the fewer ML practitioners we’ll see in the long run. And even though the world will go along fine — it always does — it’d be a sad outcome, since the only way it could happen is if gigantic corporations were flagrantly flying in the face of copyright spirit, daring it to punish you. My point is, please care about the right things. No one cared about language filters on ML models outside of a select vocal group, yet look how deeply OpenAI took those concerns to heart. Everybody cares whether their personal or professional work is being ripped off by an overfitted AI model, and it wasn’t obvious that GitHub or OpenAI gave it more than a passing thought. Backlinking to the training set should help. But it’s also going to catapult the concern of “holy moly, this code is GPL licensed!” to the front and center of anyone who works in corporate settings. Gamedev is particularly insular when it comes to GPL, and I can just imagine the conversations at various studios. “This thing might spit out GPL? We can’t use this.” My point is, when you launch that new feature to address people’s concerns, please ensure it’s working. You won’t be able to do exact string matches against the training set; you can’t rely on “well, it’s slightly different, so it’s not really the same thing.” If it’s substantially similar, it needs to be cited. And that seems like a much tougher problem than merely building an index of matching code fragments. If you launch it, and it doesn’t work, it’s going to stoke the flames. Careful not to roast.
toxik 5y ago
Uh, how about a disclaimer that this analysis is made /by Github/?
- onionisafruit 5y ago
  It's on https://docs.github.com https://docs.github.com. Of course it's by github.
  - notatoad 5y ago
    i don't think that's necessarily obvious. most links submitted to HN from the github.com domain aren't authored by github.
    - onionisafruit 5y ago
      That's understandable. I thought the GP was saying that the article should contain the disclaimer.
    - _tulpa 5y ago
      The ‘docs’ subdomain should make authorship pretty clear…
      - dannyw 5y ago
        No, it doesn't. I thought github launched a docs sharing product at first.
iudqnolq 5y ago
I continue to be surprised GitHub shows examples that wouldn't compile/run correctly. For example, the Wikipedia scraping example that the author claims is also the intuitive way to solve the problem assigns each row to the global variable cols instead of appending. Further, the following if statement appears to be mis-indented.
- qayxc 5y ago
  > I continue to be surprised GitHub shows examples that wouldn't compile/run correctly. Why is that surprising to you? CoPolit doesn't actually know how to code, it just generates symbols that match learned patterns. Sometimes these generated symbols don't represent valid code and since CoPilot doesn't actually perform filtering based on syntax checks or JIT, these results end up as suggestions. This is actually a point where future versions could greatly improve the usefulness, e.g. use the compiler infrastructure to verify and filter generated results. This includes auto-formatting and even result scoring by code metrics (conciseness, complexity, ...). Plenty of room for improvement even without touching the underlying model.
  - iudqnolq 5y ago
    I'm not surprised the actual results are flawed. I'm surprised the handful of hand-picked examples Github calls out are flawed. Usually handpicked examples of algorithms are the best-case performance.
    - pjot 5y ago
      Maybe it’s just a more honest representation of a typical result? Looking at it the other way around, had they chose _perfect_ examples there’d be similar criticism about “cherry picking”.
      - iudqnolq 5y ago
        Fair enough, although it makes me think that if these are the better examples the average is much worse.
tyingq 5y ago
The analysis seems to depend on sequences where the same exact words appear X times in the same order. If my understanding of how this works is right, they have the ability to globally change symbol names based on the prompt. And probably other things that make a literal match less likely, but what's different may be trivial. Like symbol names being swapped, use of equivalent operators (+=1 vs ++, etc), order swap where it doesn't matter, etc. Of course, I'm just speculating since I don't have access to the product, but I have seen GPT-3 output that is verbatim plus some synonym swapping.
fmakunbound 5y ago
At this point it seems Copilot is a hilarious failure, which we can learn from. I hope it’s doesn’t taint the profession further though as a bunch of amateurish cut and pasters.
- rvz 5y ago
  Well I guess it means that we must repeatedly verify such wild claims or contraptions made by others that deliberately hype and raise our expectations only to ultimately disappoint us later. I'm also going to call that the Copilot fanatics here will scream on HN about it being a paid service, when the day comes.
shakna 5y ago
> The answer is obvious: sharing the prefiltering solution we used in this analysis to detect overlap with the training set. When a suggestion contains snippets copied from the training set, the UI should simply tell you where it’s quoted from. You can then either include proper attribution or decide against using that code altogether. > This duplication search is not yet integrated into the technical preview, but we plan to do so. And we will both continue to work on decreasing rates of recitation, and on making its detection more precise. I'm afraid I don't think actually comes even close to resolving the legal implications of recitation. If Copilot is reciting pieces of GPL code (which we know it can), then not only does it need to point out where it has grabbed that code from, Copilot itself is (probably) required to be GPL-licensed. > 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. > 2b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. Copilot is distributing the GPL source code (through recitation), which means that it is, at least in part, derived from the GPL code (through learning), but it most definitely "contains" the GPL code.
- dtrailin 5y ago
  It's not clear why it would need to be if it runs as a online service. I would say this is analogous to a search engine index. Google for example likely has lots of GPL code in it's index in some transformed form and yet there is little dispute that this is legal.
  - shakna 5y ago
    > I would say this is analogous to a search engine index. Google for example likely has lots of GPL code in it's index in some transformed form and yet there is little dispute that this is legal. I'd disagree with the comparison, pretty vehemently. Copilot can generate new code. It isn't just some storage mechanism. It can create derived code, Google Search can't.
temac 5y ago
There is a huge problem for them to concentrate so much on verbatim recitations. That's not the criteria used for copyright... see for example Abstraction-Filtration-Comparison tests.
- deleted 5y ago
  [deleted]
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
varispeed 5y ago
Now the question is, whether Microsoft has a more advanced tool that they train on private repositories? How can you check whether Microsoft has infringed your private code? For example if they used an internal tool and part of my code is now in Windows 11? How do I check that?
- _ph_ 5y ago
  The question is more, did Microsoft train the copilot on their code repositories? If there are no copyright concerns, why not train it on the Windows code base? Should be a treasure trove of good code to train with.
  - frankling_ 5y ago
    That's a great point. If they didn't, it's clear what their actual risk assessment is.
al2o3cr 5y ago
(in re: "pieces of code where there's really only one way to do it" appearing verbatim) This doesn’t fit my idea of a quote either. That seems kinda cavalier, given that it took 10 years of intensive litigation and an appeal all the way to the Supreme Court to establish that copying things where there's LITERALLY no other way to do it (API declarations) wasn't copyright infringement: https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_Inc https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_....
- comex 5y ago
  There was another way to do it: Google could have designed its own API from scratch for Android, rather than reimplementing the Java standard library. Of course, this would have sacrificed two things: compatibility with existing Java code, and familiarity for existing Java developers. However, thanks to the typical game of telephone seen when lawyers and judges discuss technical subjects they're not deeply familiar with, the fact that Android was compatible with large bodies of existing Java code went essentially unrecognized; the courts focused solely on the fact that Android was not compatible with existing Java apps (in their entirety). Therefore the case was decided as if designing an API from scratch would have only sacrificed developer familiarity, not compatibility. Luckily, the Supreme Court decided that even developer familiarity alone is enough to justify reimplementing an API as a type of fair use. (And that is now precedent.) On the other hand, if you're using an API rather than reimplementing it, and it's very clear that compatibility is at issue (i.e. your code would not be compatible with the library you're trying to use if you wrote it any other way), I'd say the case for fair use is much clearer. That said, if some snippet isn't literally the only way to do something, be careful before declaring it too trivial to be copyrightable. In a different part of the same case, a trivial 9-line helper function [1] called rangeCheck, which Google had copied verbatim by mistake, was held to be an infringement. [1] https://majadhondt.wordpress.com/2012/05/16/googles-9-lines/ https://majadhondt.wordpress.com/2012/05/16/googles-9-lines/
sonofaragorn 5y ago
Did I understand correctly? There were 473 "useful/interesting" suggestions out of 453k?
- onionisafruit 5y ago
  I think it’s 473 suggestions that had interesting quotes of code from the training data
youareawesome 5y ago
Regardless of whether or not the output of Copilot is sufficiently "crow-like," the reality is that Copilot would not exist without its corpus of largely GPL-licensed code. Training data has value and many companies go to great lengths to hoard data for this exact reason. If Copilot is then offered for a price, GitHub is profiting from the work of thousands who never intended their code to be used in this way.
- nullc 5y ago
  The flip side is that if, from a policy perspective, we decide that the licensing doesn't matter-- then it's extremely likely that anyone else can make a copilot competitor. If we decide that licensing is required, it's likely that no one except github (or a few other huge players) could ever make something like this just due to access to the code. (Github would make it a requirement of the TOS that your code is licensed to allow copilot to use it, and require users to indemnify github from third party legal action arising from github using code you posted in copilot.) The permissive handling levels the playing field.
  - dannyw 5y ago
    no, if github does that the entire open source community will move to GitLab, much like the freenode to libera exodus.
    - nullc 5y ago
      Already their TOS gives them broad permissions, it just stops short of indemnification. Their TOS arguably already turns people posting projects containing third party copylefted code into license violators, -- yet people are still using them. So I wouldn't be so sure-- but moreover, I think my larger point remains: the entire world being fragmented into separate licensing is a huge moat.
yourabstraction 5y ago
This whole copyright thing reminds me of way back when Google bought Youtube. I remember at the time, my boss (and many others) were certain Google was going to get themselves into a shit storm by hosting copyrighted material. Well, it turns out a company with as many lawyers as google had/has knew plenty about copyright law and put automated systems in place to keep themselves out of legal trouble. To think that Microsoft isn't thinking deeply about license/copyright issues for Copilot and working on solutions seems pretty naive.
rasz 5y ago
TLDR: Microsoft: We have seen Antitrust (2001) https://www.imdb.com/title/tt0218817/ https://www.imdb.com/title/tt0218817/ and were finally able to implement it after 20 years, this time without murder and conspiracy, all in the open.
0-_-0 5y ago
Surprising how extremely rarely Copilot quotes text verbatim.

184 points by azhenley 5y ago

It should be noted that this kind of behavior is entirely expected from a GPT-style self-supervised sequence model. Rote memorization for this kind of model is indicative of correct training, not overfitting. The underlying training objective of these models ideally results in a representation of the training data which allows complete samples to be extracted by using partial samples as keys. Actual overfitting in this kind of model requires absurd parameter counts. See https://tilde.town/~fessus/reward_is_unnecessary.pdf https://tilde.town/~fessus/reward_is_unnecessary.pdf

amelius 5y ago

Does it substitute variables correctly? E.g. if I define max(a,b) or max(x,y) does it complete the definition with the right variable names?

onionisafruit 5y ago
Generally, yes. It’s not guaranteed to do that correctly, but I’ve not seen it get variable names wrong so far.
- tyingq 5y ago
  There's an example called "fetch_tweets.py" at the bottom of page on https://copilot.github.com/ https://copilot.github.com/ that gets it wrong: def fetch_tweets_from_user(user_name): # deleted some lines here... # fetch tweets tweets = api.user_timeline(screen_name=user, count=200, include_rts=False) screen_name=user there isn't right. It's a nit, but it is interesting how many of the hand-picked examples on that page aren't right. Since they were hand-picked, presumably to show the product off.
  - deleted 5y ago
    [deleted]
- the8472 5y ago
  > It’s not guaranteed to do that correctly Which is odd considering they could run this as beam search with the checking part of a compiler in the loop.
  - 0-_-0 5y ago
    Which would be extremely slow
    - the8472 5y ago
      Not necessarily. If you emit a batch of suggestions for 1 line and prune them with syntax checks you can already prune them and present them to the user while working on the next iterations. The real difficulty is getting good checking on partial source code, some languages have better tooling for that than others.

blamestross 5y ago

Take aways in summary: - Copilot does sometimes rote copy for nontrivial situations - Mostly this happens when there isn't much context to go on. - Provided an empty file, it proceeds to recommend to write the GPL - They will add a "recitation detector" to copilot to indicate non-novel recommendations By the standards of corp-speak this is pretty good, they admit there is a problem there and they intend to do something tractable to prevent it. This entire copilot situation is far enough outside my personal "mental ethics model" that I'm personally abstaining form taking a stance until I have had a lot more time to think and learn about it.

temac 5y ago
They quite misdirect on what the problem actually is. Verbatim or quasi-verbatim recitations are not the only things copyright protects.

ionwake 5y ago

Sorry for the basic question, but the code one builds on this platform is saved to githubs servers?

k__ 5y ago
yes
rvz 5y ago
Anything pasted or typed into that Copilot editor is sent to GitHub as 'telemetry'. > In order to generate suggestions, GitHub Copilot transmits part of the file you are editing to the service. So Yes.
- astrange 5y ago
  That's not telemetry, it's the prompt to the model to generate the rest of the text. I would assume it's not saved.
  - onionisafruit 5y ago
    I wouldn't think to call that telemetry either, but it is addressed in this doc about telemetry: https://docs.github.com/en/github/copilot/telemetry-terms#additional-telemetry https://docs.github.com/en/github/copilot/telemetry-terms#ad...
Hamuko 5y ago
Don't know about saved, but definitely sent. I guess one risk after knowing that Github regards all source code regardless of its license to be fair game for training Copilot is that you probably can't know for sure that your new code is not being used to teach the model more.
- onionisafruit 5y ago
  It's clear the author of this article had access to the code that triggered the copilot suggestions. They also say this was from an internal trial of copilot, so it might be that these trial users were told their code could be seen by their coworkers.

devinplatt 5y ago

The article is worth reading, but a good summary is at the bottom: > This investigation demonstrates that GitHub Copilot can quote a body of code verbatim, but that it rarely does so, and when it does, it mostly quotes code that everybody quotes, and mostly at the beginning of a file, as if to break the ice. But there’s still one big difference between GitHub Copilot reciting code and me reciting a poem: I know when I’m quoting. I would also like to know when Copilot is echoing existing code rather than coming up with its own ideas. That way, I’m able to look up background information about that code, and to include credit where credit is due. The answer is obvious: sharing the prefiltering solution we used in this analysis to detect overlap with the training set. When a suggestion contains snippets copied from the training set, the UI should simply tell you where it’s quoted from. You can then either include proper attribution or decide against using that code altogether. This duplication search is not yet integrated into the technical preview, but we plan to do so. And we will both continue to work on decreasing rates of recitation, and on making its detection more precise.

sillysaurusx 5y ago
The answer wasn’t obvious to me. Nice solution. It sounds like you’re a part of the Copilot team. If so, then I’m happy to see the Copilot team cares about these issues at all. I was expecting nothing but stonewall until the conversation died out, since realistically the chance of the EFF bringing or winning a lawsuit seems small. (And who else would try?) But when you anger the world and being so much attention to this delicate issue of copyright in AI, you risk every hobbyist. Suppose the world decides that AI models need to be restricted. Now every person who wants to get into AI will need to deal with it. I’m not sure anyone else cares, but I care, because it’s the difference between someone getting into woodworking (an unrestricted hobby) vs becoming a lawyer or doctor (the maximally restrictive hobby). The closer we are to the latter, the fewer ML practitioners we’ll see in the long run. And even though the world will go along fine — it always does — it’d be a sad outcome, since the only way it could happen is if gigantic corporations were flagrantly flying in the face of copyright spirit, daring it to punish you. My point is, please care about the right things. No one cared about language filters on ML models outside of a select vocal group, yet look how deeply OpenAI took those concerns to heart. Everybody cares whether their personal or professional work is being ripped off by an overfitted AI model, and it wasn’t obvious that GitHub or OpenAI gave it more than a passing thought. Backlinking to the training set should help. But it’s also going to catapult the concern of “holy moly, this code is GPL licensed!” to the front and center of anyone who works in corporate settings. Gamedev is particularly insular when it comes to GPL, and I can just imagine the conversations at various studios. “This thing might spit out GPL? We can’t use this.” My point is, when you launch that new feature to address people’s concerns, please ensure it’s working. You won’t be able to do exact string matches against the training set; you can’t rely on “well, it’s slightly different, so it’s not really the same thing.” If it’s substantially similar, it needs to be cited. And that seems like a much tougher problem than merely building an index of matching code fragments. If you launch it, and it doesn’t work, it’s going to stoke the flames. Careful not to roast.

toxik 5y ago

Uh, how about a disclaimer that this analysis is made /by Github/?

onionisafruit 5y ago
It's on https://docs.github.com https://docs.github.com. Of course it's by github.
- notatoad 5y ago
  i don't think that's necessarily obvious. most links submitted to HN from the github.com domain aren't authored by github.
  - onionisafruit 5y ago
    That's understandable. I thought the GP was saying that the article should contain the disclaimer.
  - _tulpa 5y ago
    The ‘docs’ subdomain should make authorship pretty clear…
    - dannyw 5y ago
      No, it doesn't. I thought github launched a docs sharing product at first.

iudqnolq 5y ago

I continue to be surprised GitHub shows examples that wouldn't compile/run correctly. For example, the Wikipedia scraping example that the author claims is also the intuitive way to solve the problem assigns each row to the global variable cols instead of appending. Further, the following if statement appears to be mis-indented.

qayxc 5y ago
> I continue to be surprised GitHub shows examples that wouldn't compile/run correctly. Why is that surprising to you? CoPolit doesn't actually know how to code, it just generates symbols that match learned patterns. Sometimes these generated symbols don't represent valid code and since CoPilot doesn't actually perform filtering based on syntax checks or JIT, these results end up as suggestions. This is actually a point where future versions could greatly improve the usefulness, e.g. use the compiler infrastructure to verify and filter generated results. This includes auto-formatting and even result scoring by code metrics (conciseness, complexity, ...). Plenty of room for improvement even without touching the underlying model.
- iudqnolq 5y ago
  I'm not surprised the actual results are flawed. I'm surprised the handful of hand-picked examples Github calls out are flawed. Usually handpicked examples of algorithms are the best-case performance.
  - pjot 5y ago
    Maybe it’s just a more honest representation of a typical result? Looking at it the other way around, had they chose _perfect_ examples there’d be similar criticism about “cherry picking”.
    - iudqnolq 5y ago
      Fair enough, although it makes me think that if these are the better examples the average is much worse.

tyingq 5y ago

The analysis seems to depend on sequences where the same exact words appear X times in the same order. If my understanding of how this works is right, they have the ability to globally change symbol names based on the prompt. And probably other things that make a literal match less likely, but what's different may be trivial. Like symbol names being swapped, use of equivalent operators (+=1 vs ++, etc), order swap where it doesn't matter, etc. Of course, I'm just speculating since I don't have access to the product, but I have seen GPT-3 output that is verbatim plus some synonym swapping.

fmakunbound 5y ago

At this point it seems Copilot is a hilarious failure, which we can learn from. I hope it’s doesn’t taint the profession further though as a bunch of amateurish cut and pasters.

rvz 5y ago
Well I guess it means that we must repeatedly verify such wild claims or contraptions made by others that deliberately hype and raise our expectations only to ultimately disappoint us later. I'm also going to call that the Copilot fanatics here will scream on HN about it being a paid service, when the day comes.

shakna 5y ago

> The answer is obvious: sharing the prefiltering solution we used in this analysis to detect overlap with the training set. When a suggestion contains snippets copied from the training set, the UI should simply tell you where it’s quoted from. You can then either include proper attribution or decide against using that code altogether. > This duplication search is not yet integrated into the technical preview, but we plan to do so. And we will both continue to work on decreasing rates of recitation, and on making its detection more precise. I'm afraid I don't think actually comes even close to resolving the legal implications of recitation. If Copilot is reciting pieces of GPL code (which we know it can), then not only does it need to point out where it has grabbed that code from, Copilot itself is (probably) required to be GPL-licensed. > 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. > 2b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. Copilot is distributing the GPL source code (through recitation), which means that it is, at least in part, derived from the GPL code (through learning), but it most definitely "contains" the GPL code.

dtrailin 5y ago
It's not clear why it would need to be if it runs as a online service. I would say this is analogous to a search engine index. Google for example likely has lots of GPL code in it's index in some transformed form and yet there is little dispute that this is legal.
- shakna 5y ago
  > I would say this is analogous to a search engine index. Google for example likely has lots of GPL code in it's index in some transformed form and yet there is little dispute that this is legal. I'd disagree with the comparison, pretty vehemently. Copilot can generate new code. It isn't just some storage mechanism. It can create derived code, Google Search can't.

temac 5y ago

There is a huge problem for them to concentrate so much on verbatim recitations. That's not the criteria used for copyright... see for example Abstraction-Filtration-Comparison tests.

deleted 5y ago
[deleted]

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

varispeed 5y ago

Now the question is, whether Microsoft has a more advanced tool that they train on private repositories? How can you check whether Microsoft has infringed your private code? For example if they used an internal tool and part of my code is now in Windows 11? How do I check that?

_ph_ 5y ago
The question is more, did Microsoft train the copilot on their code repositories? If there are no copyright concerns, why not train it on the Windows code base? Should be a treasure trove of good code to train with.
- frankling_ 5y ago
  That's a great point. If they didn't, it's clear what their actual risk assessment is.

al2o3cr 5y ago

(in re: "pieces of code where there's really only one way to do it" appearing verbatim) This doesn’t fit my idea of a quote either. That seems kinda cavalier, given that it took 10 years of intensive litigation and an appeal all the way to the Supreme Court to establish that copying things where there's LITERALLY no other way to do it (API declarations) wasn't copyright infringement: https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_Inc https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_....

comex 5y ago
There was another way to do it: Google could have designed its own API from scratch for Android, rather than reimplementing the Java standard library. Of course, this would have sacrificed two things: compatibility with existing Java code, and familiarity for existing Java developers. However, thanks to the typical game of telephone seen when lawyers and judges discuss technical subjects they're not deeply familiar with, the fact that Android was compatible with large bodies of existing Java code went essentially unrecognized; the courts focused solely on the fact that Android was not compatible with existing Java apps (in their entirety). Therefore the case was decided as if designing an API from scratch would have only sacrificed developer familiarity, not compatibility. Luckily, the Supreme Court decided that even developer familiarity alone is enough to justify reimplementing an API as a type of fair use. (And that is now precedent.) On the other hand, if you're using an API rather than reimplementing it, and it's very clear that compatibility is at issue (i.e. your code would not be compatible with the library you're trying to use if you wrote it any other way), I'd say the case for fair use is much clearer. That said, if some snippet isn't literally the only way to do something, be careful before declaring it too trivial to be copyrightable. In a different part of the same case, a trivial 9-line helper function [1] called rangeCheck, which Google had copied verbatim by mistake, was held to be an infringement. [1] https://majadhondt.wordpress.com/2012/05/16/googles-9-lines/ https://majadhondt.wordpress.com/2012/05/16/googles-9-lines/

sonofaragorn 5y ago

Did I understand correctly? There were 473 "useful/interesting" suggestions out of 453k?

onionisafruit 5y ago
I think it’s 473 suggestions that had interesting quotes of code from the training data

youareawesome 5y ago

Regardless of whether or not the output of Copilot is sufficiently "crow-like," the reality is that Copilot would not exist without its corpus of largely GPL-licensed code. Training data has value and many companies go to great lengths to hoard data for this exact reason. If Copilot is then offered for a price, GitHub is profiting from the work of thousands who never intended their code to be used in this way.

nullc 5y ago
The flip side is that if, from a policy perspective, we decide that the licensing doesn't matter-- then it's extremely likely that anyone else can make a copilot competitor. If we decide that licensing is required, it's likely that no one except github (or a few other huge players) could ever make something like this just due to access to the code. (Github would make it a requirement of the TOS that your code is licensed to allow copilot to use it, and require users to indemnify github from third party legal action arising from github using code you posted in copilot.) The permissive handling levels the playing field.
- dannyw 5y ago
  no, if github does that the entire open source community will move to GitLab, much like the freenode to libera exodus.
  - nullc 5y ago
    Already their TOS gives them broad permissions, it just stops short of indemnification. Their TOS arguably already turns people posting projects containing third party copylefted code into license violators, -- yet people are still using them. So I wouldn't be so sure-- but moreover, I think my larger point remains: the entire world being fragmented into separate licensing is a huge moat.

yourabstraction 5y ago

This whole copyright thing reminds me of way back when Google bought Youtube. I remember at the time, my boss (and many others) were certain Google was going to get themselves into a shit storm by hosting copyrighted material. Well, it turns out a company with as many lawyers as google had/has knew plenty about copyright law and put automated systems in place to keep themselves out of legal trouble. To think that Microsoft isn't thinking deeply about license/copyright issues for Copilot and working on solutions seems pretty naive.

rasz 5y ago

TLDR: Microsoft: We have seen Antitrust (2001) https://www.imdb.com/title/tt0218817/ https://www.imdb.com/title/tt0218817/ and were finally able to implement it after 20 years, this time without murder and conspiracy, all in the open.

0-_-0 5y ago

Surprising how extremely rarely Copilot quotes text verbatim.