4 ms·

> it doesn't mean every single line in isolation is copyrightable Microsoft did not just copy individual lines. They fed whole repositories into their model, i

> it doesn't mean every single line in isolation is copyrightable Microsoft did not just copy individual lines. They fed whole repositories into their model, ignoring the license (if it exists) even though they knew from the start that information generated by the model will be publicly available. Available usually out of context, but nonetheless - the scope of the input and intent are very clearly "everything" and "redistribution". Just adding a filter/ML model to the output shouldn't matter. I dare you to build a Copilot clone trained from leaked internal Microsoft code and then trying to argue the output is a bit mixed up. That is a clear violation imho.

google234123 5y ago
Copilot was trained on leaked internal Microsoft code that's on github at the moment. Anyway, everyone seems perfectly ok with training langauge models on copyright text.
- leereeves 5y ago
  If a trained language model exactly reproduces copyrighted text, is there any question about whether copyright still applies?
  - TchoBeer 5y ago
    This is a useless hypothetical, no language models do that
    - heavyset_go 5y ago
      And yet there are plenty of examples of Copilot reproducing copyrighted code verbatim, like is does in this example[1] that was posted on HN. [1] https://twitter.com/mitsuhiko/status/1410886329924194309 https://twitter.com/mitsuhiko/status/1410886329924194309
    - jen20 5y ago
      This is precisely what Copilot does, regularly.
  - benhurmarcel 5y ago
    But then the infringement is done by the person who publishes that output, not by the text editor that copies the code.
- dylan604 5y ago
  Everyone is not perfectly OK with training language models on copyrighted text. It's just that evilCorps do it anyways, and there's nothing anyone can do to stop them. I can't do anything. At best, I could get a Twitter account and complain to the ether. The copyright holders can't do anything against the might evilCorps, but that doesn't make them okay with it. The fact you believe this is just sad, and exactly what evilCorps want from you. This goes beyond fair use or satirical/comedic effect. They are training their models to output text in the style of the authors being absorbed. The style of is exactly the artistic effect that is being copyrighted.
  - gradys 5y ago
    Could you explain why you think training models on copyrighted text is illegal or copyright infringement or whatever else it might be?
    - klyrs 5y ago
      Training the models is fine. Applying the models, which reproduces copyrighted works without proper attribution, is where it gets sticky.
    - dylan604 5y ago
      My explanation will not be popular here on HN, but I'm never one to shy away. Especially when asked directly. Buying a book, buying an audio CD, or buying a DVD/Blu-ray is granting the holder permission to read,listen,view that product as a single instance. You can lend them out, but that's all you're really allowed to do with them. The text,audio/video is not owned by you to do with as you please. People obviously do not like that, and argue making copies/backups is their right. Maybe that's acceptable, but we can agree posting them on torrents and sharing in any other manner from a copy made from the thing you have is not. Saying that, training a model on someone's copyrighted text is not part of the agreement of the usage of said text whether it's a copyrighted magazine, newspaper, or book. If the people doing the training reach out to the copyright holders and get specific permission to use their copyrighted material in such a manner, then go ahead. The fact that people feel like they can do anything without the common courtesy of asking for permission is troubling to me that we've lost something as a society. There's no acknowledgment that someone has created something by their own work so that the creator can do with it as they please. A large portion of people believe that because it was created they deserve/should be able to/etc do what ever they want with someone else's creation. Including getting paid for derivitave works from the original creation.
sobellian 5y ago
The search engine on Github also calls up entire pages of GPL licensed code verbatim. Does it run afoul of copyright?

google234123 5y ago

Copilot was trained on leaked internal Microsoft code that's on github at the moment. Anyway, everyone seems perfectly ok with training langauge models on copyright text.

leereeves 5y ago
If a trained language model exactly reproduces copyrighted text, is there any question about whether copyright still applies?
- TchoBeer 5y ago
  This is a useless hypothetical, no language models do that
  - heavyset_go 5y ago
    And yet there are plenty of examples of Copilot reproducing copyrighted code verbatim, like is does in this example[1] that was posted on HN. [1] https://twitter.com/mitsuhiko/status/1410886329924194309 https://twitter.com/mitsuhiko/status/1410886329924194309
  - jen20 5y ago
    This is precisely what Copilot does, regularly.
- benhurmarcel 5y ago
  But then the infringement is done by the person who publishes that output, not by the text editor that copies the code.
dylan604 5y ago
Everyone is not perfectly OK with training language models on copyrighted text. It's just that evilCorps do it anyways, and there's nothing anyone can do to stop them. I can't do anything. At best, I could get a Twitter account and complain to the ether. The copyright holders can't do anything against the might evilCorps, but that doesn't make them okay with it. The fact you believe this is just sad, and exactly what evilCorps want from you. This goes beyond fair use or satirical/comedic effect. They are training their models to output text in the style of the authors being absorbed. The style of is exactly the artistic effect that is being copyrighted.
- gradys 5y ago
  Could you explain why you think training models on copyrighted text is illegal or copyright infringement or whatever else it might be?
  - klyrs 5y ago
    Training the models is fine. Applying the models, which reproduces copyrighted works without proper attribution, is where it gets sticky.
  - dylan604 5y ago
    My explanation will not be popular here on HN, but I'm never one to shy away. Especially when asked directly. Buying a book, buying an audio CD, or buying a DVD/Blu-ray is granting the holder permission to read,listen,view that product as a single instance. You can lend them out, but that's all you're really allowed to do with them. The text,audio/video is not owned by you to do with as you please. People obviously do not like that, and argue making copies/backups is their right. Maybe that's acceptable, but we can agree posting them on torrents and sharing in any other manner from a copy made from the thing you have is not. Saying that, training a model on someone's copyrighted text is not part of the agreement of the usage of said text whether it's a copyrighted magazine, newspaper, or book. If the people doing the training reach out to the copyright holders and get specific permission to use their copyrighted material in such a manner, then go ahead. The fact that people feel like they can do anything without the common courtesy of asking for permission is troubling to me that we've lost something as a society. There's no acknowledgment that someone has created something by their own work so that the creator can do with it as they please. A large portion of people believe that because it was created they deserve/should be able to/etc do what ever they want with someone else's creation. Including getting paid for derivitave works from the original creation.

sobellian 5y ago

The search engine on Github also calls up entire pages of GPL licensed code verbatim. Does it run afoul of copyright?