5 ms·

I’m a broken record at this point, but the focus on GPL code seems secondary to the massive repetition of all-rights-reserved content. I recently tried entering

I’m a broken record at this point, but the focus on GPL code seems secondary to the massive repetition of all-rights-reserved content. I recently tried entering “chapter one: the boy who lived” as a prompt into gpt-3, and it was happy to spit out chapter one of Harry Potter, verbatim.

einarfd 5y ago
Someone should definitely tell J. K. Rowling about that. She might be in a position to do something about this, and she definitely needs something more useful to focus her unhappiness and frustration at.
kouteiheika 5y ago
Yep. So we either 1) declare this to not be copyright infringement, or 2) basically outright outlaw training of interesting ML models. I personally believe that (1) is the way to go, and I find the whole outrage about Copilot to be essentially akin to collectively shooting ourselves in the foot in the long run. I think we shouldn't forget why copyright exists in the first place. To quote the copyright clause of the US constitution (emphasis mine): > "*To promote the Progress of Science and useful Arts*, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries." If anything we already went too far with how draconian modern copyright enforcement is, up to the point that it often actively hampers our progress. Please don't make it even worse!
- michaelmrose 5y ago
  How would you go about 1 given existing law, precedent, and circumstances. Would you dramatically expand what you can legally borrow without infringing or just allow wholesale automated washing from one copyright to another? What do you do when random corp starts shutting down open source over ownership of code it swiped from project claiming infringement?
  - mjg59 5y ago
    What existing law, precedent, and circumstances demonstrate that the output of an ML model is a derived work of the training data?
    - krab 5y ago
      The problem is probably the distinguishing line. When is it a ML model doing something and when is it a human with tools doing the same thing? In other words, to what extent does the color of the bits matter? [1] And does the output of a ML model have a new "color"? [1]: https://ansuz.sooke.bc.ca/entry/23 https://ansuz.sooke.bc.ca/entry/23
    - michaelmrose 5y ago
      If I wrote a shell script to sample a subset of your code base and rename the identifiers I shouldn't think it would require new case law to figure that out.
  - kouteiheika 5y ago
    > Would you dramatically expand what you can legally borrow without infringing or just allow wholesale automated washing from one copyright to another? You're acting like we don't already have something similar. We do. It's called fair use. (: I'd probably just expand on that. If the portion of the work generated by a model is insubstantial, and/or doesn't harm the interests of the original rights holder then it should be okay. If your model can generate a single excerpt of Harry Potter does that constitute harm to J.K. Rowling? Can you use it to reasonably generate a substantial portion of the book with it? Is it a reasonable substitute for buying the book? No? Then it's okay. Some countries in fact do already have such exceptions, e.g. Japan amended its copyright law in 2018 to explicitly allow for this. To quote the relevant law: > "It is permissible to exploit work in any way and to the extent considered necessary ...where such exploitation is not for enjoying or causing another person to enjoy the ideas or emotions expressed in such work..." So if you want to train an ML model on a bunch of commercial novels, without the permission of their owners, and then exploit that commercially - you can do it, and it's completely legal.
- hetspookjee 5y ago
  I think this is a too black-and-white picture of the options that are there.
- rkeene2 5y ago
  The question I have is that if Microsoft really thinks #1 is true and the way, why didn't they train Copilot on Microsoft's code (in addition to other people's code on GitHub.com) ?
  - ThrowawayB7 5y ago
    Copilot only supports "Python, JavaScript, TypeScript, Ruby, and Go" but obviously the majority of Microsoft's product code is in C++ and C# and far fewer people use those two languages.
- 6gvONxR4sf7o 5y ago
  > 2) basically outright outlaw training of interesting ML models. Saying that it’s infringement doesn’t mean we’re outlawing interesting ML. These definitely aren’t the only two options. A few years ago, this kind of infringement wasn’t even possible, and now it’s just beginning. This stuff is in its infancy. Reliability and controllability is an issue our field has to confront as applications move from research to application. This is just another instance of that. We’re all eager to deploy this stuff as soon as it can do something cool, but that’s not enough, as all the discussion around copilot demonstrates. > I think we shouldn't forget why copyright exists in the first place… To promote the Progress of Science and useful Arts Right. It’s to encourage creators to create things. I wrote about this elsewhere [0] that the issue of generative ML models is that, as far as incentives go, you can pay a creator for use of their things, or you can throw money at AWS and get use of something like that creator’s things. The fair use equation is ‘$$$ + my stuff = your stuff,’ except that that money doesn’t go to the creators, it goes to nvidia and aws. I think we’ll need new copyright laws for this, to make a middle ground. I personally want to be able to exclude my works from being training data unless I license then openly enough. Just like with open code licenses where you opt in. That option doesn’t seem to exist right now. Wouldn’t it be great if a model benefiting from all this public data had to be public itself, or something along those lines? Wouldn’t that promote the arts and sciences even more? [0] https://news.ycombinator.com/item?id=27778042 https://news.ycombinator.com/item?id=27778042
- b3morales 5y ago
  There's a lot of excluded middle between your 1 and 2. Off the top of my head, some other options to consider... - We could extend copyright statute to explicitly allow training ML models only under particular conditions (for example, some kind of "non-commercial use" clause). - We could enact mechanical licensing for inputs to ML models, as exists for musical cover performances. Creators of the works used as inputs would get royalties. - We could require asking for permission to use a work as input, but standardized generous terms once permission is granted. - We could flip that and just require notice that you're about to use a work as input, but allow the creator a certain window to object. - We could declare that inputs are a free-for-all, automatically non-infringing, but that you (the creator of the model) are liable for extra damages if the outputs are proven to be able to infringe. Many, many different ways this could shake out. I'm not particularly optimistic that any of (what I see as) the better choices will come to pass, but they exist.
Hamuko 5y ago
Especially since all-rights-reserved content is apparently the majority on GitHub. About 80% of repositories were without a license in 2015 and I doubt it has changed drastically. https://github.blog/2015-03-09-open-source-license-usage-on-github-com/ https://github.blog/2015-03-09-open-source-license-usage-on-...
lacavao 5y ago
Any chance you could share the GPT-3 settings you used to get that result? I've been playing around to try and get the same thing -- closest I've gotten is having it spit out other (correct) chapter names in the same format.
- 6gvONxR4sf7o 5y ago
  With the default parameters it’ll do it fairly reliably with this CHAPTER ONE THE BOY WHO LIVED and it’ll do it super duper reliably with CHAPTER ONE THE BOY WHO LIVED Mr.
tiborsaas 5y ago
If I say "3.14" to you and then ask you to try to come up with two more numbers that's very likely to follow what would you say?
TazeTSchnitzel 5y ago
GitHub has quite a bit of source-available code (i.e. the source is public but it's not free software). Would be very interesting if Copilot repeats something verbatim from that.
mark_l_watson 5y ago
I am very surprised by this. Your experiment implies that GPT-3 has sufficient parameters to keep copies of training text. I have been effectively using GPT-3 a lot recently because I added chapters with OpenAI APIs examples to my Common Lisp and Clojure books (you can get free copies at [1] by setting the price to Free). At Capital One, I used LSTM models to generate AWS JSON logs for testing purposes. The original logs contained sensitive data that was not in the generated data. JSON structure and names of keys was learned and reproduced verbatim, but not the values. GPT-3 has massively more parameters than my models. BTW, I also generated spreadsheet data using GANs: similar idea, but different tech. [1] https://leanpub.com/u/markwatson https://leanpub.com/u/markwatson

einarfd 5y ago

Someone should definitely tell J. K. Rowling about that. She might be in a position to do something about this, and she definitely needs something more useful to focus her unhappiness and frustration at.

kouteiheika 5y ago

Yep. So we either 1) declare this to not be copyright infringement, or 2) basically outright outlaw training of interesting ML models. I personally believe that (1) is the way to go, and I find the whole outrage about Copilot to be essentially akin to collectively shooting ourselves in the foot in the long run. I think we shouldn't forget why copyright exists in the first place. To quote the copyright clause of the US constitution (emphasis mine): > "*To promote the Progress of Science and useful Arts*, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries." If anything we already went too far with how draconian modern copyright enforcement is, up to the point that it often actively hampers our progress. Please don't make it even worse!

michaelmrose 5y ago
How would you go about 1 given existing law, precedent, and circumstances. Would you dramatically expand what you can legally borrow without infringing or just allow wholesale automated washing from one copyright to another? What do you do when random corp starts shutting down open source over ownership of code it swiped from project claiming infringement?
- mjg59 5y ago
  What existing law, precedent, and circumstances demonstrate that the output of an ML model is a derived work of the training data?
  - krab 5y ago
    The problem is probably the distinguishing line. When is it a ML model doing something and when is it a human with tools doing the same thing? In other words, to what extent does the color of the bits matter? [1] And does the output of a ML model have a new "color"? [1]: https://ansuz.sooke.bc.ca/entry/23 https://ansuz.sooke.bc.ca/entry/23
  - michaelmrose 5y ago
    If I wrote a shell script to sample a subset of your code base and rename the identifiers I shouldn't think it would require new case law to figure that out.
- kouteiheika 5y ago
  > Would you dramatically expand what you can legally borrow without infringing or just allow wholesale automated washing from one copyright to another? You're acting like we don't already have something similar. We do. It's called fair use. (: I'd probably just expand on that. If the portion of the work generated by a model is insubstantial, and/or doesn't harm the interests of the original rights holder then it should be okay. If your model can generate a single excerpt of Harry Potter does that constitute harm to J.K. Rowling? Can you use it to reasonably generate a substantial portion of the book with it? Is it a reasonable substitute for buying the book? No? Then it's okay. Some countries in fact do already have such exceptions, e.g. Japan amended its copyright law in 2018 to explicitly allow for this. To quote the relevant law: > "It is permissible to exploit work in any way and to the extent considered necessary ...where such exploitation is not for enjoying or causing another person to enjoy the ideas or emotions expressed in such work..." So if you want to train an ML model on a bunch of commercial novels, without the permission of their owners, and then exploit that commercially - you can do it, and it's completely legal.
hetspookjee 5y ago
I think this is a too black-and-white picture of the options that are there.
rkeene2 5y ago
The question I have is that if Microsoft really thinks #1 is true and the way, why didn't they train Copilot on Microsoft's code (in addition to other people's code on GitHub.com) ?
- ThrowawayB7 5y ago
  Copilot only supports "Python, JavaScript, TypeScript, Ruby, and Go" but obviously the majority of Microsoft's product code is in C++ and C# and far fewer people use those two languages.
6gvONxR4sf7o 5y ago
> 2) basically outright outlaw training of interesting ML models. Saying that it’s infringement doesn’t mean we’re outlawing interesting ML. These definitely aren’t the only two options. A few years ago, this kind of infringement wasn’t even possible, and now it’s just beginning. This stuff is in its infancy. Reliability and controllability is an issue our field has to confront as applications move from research to application. This is just another instance of that. We’re all eager to deploy this stuff as soon as it can do something cool, but that’s not enough, as all the discussion around copilot demonstrates. > I think we shouldn't forget why copyright exists in the first place… To promote the Progress of Science and useful Arts Right. It’s to encourage creators to create things. I wrote about this elsewhere [0] that the issue of generative ML models is that, as far as incentives go, you can pay a creator for use of their things, or you can throw money at AWS and get use of something like that creator’s things. The fair use equation is ‘$$$ + my stuff = your stuff,’ except that that money doesn’t go to the creators, it goes to nvidia and aws. I think we’ll need new copyright laws for this, to make a middle ground. I personally want to be able to exclude my works from being training data unless I license then openly enough. Just like with open code licenses where you opt in. That option doesn’t seem to exist right now. Wouldn’t it be great if a model benefiting from all this public data had to be public itself, or something along those lines? Wouldn’t that promote the arts and sciences even more? [0] https://news.ycombinator.com/item?id=27778042 https://news.ycombinator.com/item?id=27778042
b3morales 5y ago
There's a lot of excluded middle between your 1 and 2. Off the top of my head, some other options to consider... - We could extend copyright statute to explicitly allow training ML models only under particular conditions (for example, some kind of "non-commercial use" clause). - We could enact mechanical licensing for inputs to ML models, as exists for musical cover performances. Creators of the works used as inputs would get royalties. - We could require asking for permission to use a work as input, but standardized generous terms once permission is granted. - We could flip that and just require notice that you're about to use a work as input, but allow the creator a certain window to object. - We could declare that inputs are a free-for-all, automatically non-infringing, but that you (the creator of the model) are liable for extra damages if the outputs are proven to be able to infringe. Many, many different ways this could shake out. I'm not particularly optimistic that any of (what I see as) the better choices will come to pass, but they exist.

Hamuko 5y ago

Especially since all-rights-reserved content is apparently the majority on GitHub. About 80% of repositories were without a license in 2015 and I doubt it has changed drastically. https://github.blog/2015-03-09-open-source-license-usage-on-github-com/ https://github.blog/2015-03-09-open-source-license-usage-on-...

lacavao 5y ago

Any chance you could share the GPT-3 settings you used to get that result? I've been playing around to try and get the same thing -- closest I've gotten is having it spit out other (correct) chapter names in the same format.

6gvONxR4sf7o 5y ago
With the default parameters it’ll do it fairly reliably with this CHAPTER ONE THE BOY WHO LIVED and it’ll do it super duper reliably with CHAPTER ONE THE BOY WHO LIVED Mr.

tiborsaas 5y ago

If I say "3.14" to you and then ask you to try to come up with two more numbers that's very likely to follow what would you say?

TazeTSchnitzel 5y ago

GitHub has quite a bit of source-available code (i.e. the source is public but it's not free software). Would be very interesting if Copilot repeats something verbatim from that.

mark_l_watson 5y ago

I am very surprised by this. Your experiment implies that GPT-3 has sufficient parameters to keep copies of training text. I have been effectively using GPT-3 a lot recently because I added chapters with OpenAI APIs examples to my Common Lisp and Clojure books (you can get free copies at [1] by setting the price to Free). At Capital One, I used LSTM models to generate AWS JSON logs for testing purposes. The original logs contained sensitive data that was not in the generated data. JSON structure and names of keys was learned and reproduced verbatim, but not the values. GPT-3 has massively more parameters than my models. BTW, I also generated spreadsheet data using GANs: similar idea, but different tech. [1] https://leanpub.com/u/markwatson https://leanpub.com/u/markwatson