11 ms·

Surprised it's not mentioned already. OpenAI recently fed everything on GitHub into a language model which will cheerfully reproduce GPL'd software without lic

Surprised it's not mentioned already. OpenAI recently fed everything on GitHub into a language model which will cheerfully reproduce GPL'd software without license or attribution. I think that's the end of "free software" in the GPL sense. There is zero point to writing AGPL on a program if copying it into a language model and then back out into a text file without the license is a legitimate thing to do. I can't imagine this being countered effectively by lawsuit. Microsoft has built a weapon that they can use to disassemble open source software, I think they'll take the legal side of that seriously enough that GPL will be a distant memory by the time the courts conclude in either direction.

bayindirh 3y ago
People don't want to acknowledge it, because they deem that CoPilot and similar tools are too useful. I personally left the day CoPilot became GA. It's an open secret that xGPL licensed applications are fed into AI models, but opponents' voice is drowned under "We're doing something amazing, we don't need any permission. Plus, it's all fair use, anyway" choir. This also important for Source Available (for your eyes only) licenses. People don't care about them either. This whole AI thing is a boundless ethics violation in every category, yet people don't care, because "it's awesome!" for reasons I don't understand. Addendum: The reason why OpenAI has a non-profit part is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work.
- whaleofatw2022 3y ago
  We have been conditioned for decades to allow an increasingly slippery slope of dubious behavior in the name of 'cool'
  - lo_zamoyski 3y ago
    Cool, or convenient.
    - prox 3y ago
      Cool, convenient and coins
      - ableal 3y ago
        Don't miss collecting the coupons for a free Butlerian Jihad (e.g. https://dune.fandom.com/wiki/Butlerian_Jihad https://dune.fandom.com/wiki/Butlerian_Jihad )
  - jstummbillig 3y ago
    As a blanket statement? I don't think so. The reason we even think of anything as "dubious" now, is a moral compass, that has (at the large scale) been staggeringly readjusted for increased human well-being.
- myaccountonhn 3y ago
  It's also why these days I do not host any of my code on Github. Only then did I realize how much of an iron grip it has on the ecosystem. People expect Github Actions, Pull requests and well now people want to use CoPilot for all their work. It is extremely worrying.
  - bayindirh 3y ago
    Me neither. I moved to Source Hut. I'll be restarting my work on my previous research and pondering on whether I should keep its code at a Forgejo instance buried under a VPN, until it's ready to release.
  - palata 3y ago
    Same here, I moved to SourceHut. There are other alternatives that look much more like GitHub for those who like that (like Forgejo).
  - JonChesterfield 3y ago
    That's the "extend" part of the embrace, extend, extinguish game plan.
- blagie 3y ago
  As much as I agree with the conclusion (my AGPL code was stolen by OpenAI, as was a lot of other code), your logic is extreme. This is not good. A more reasoned argument is more convincing. 1. "Fair use" It is fair use to use snippets of my code in other projects. AGPL doesn't change that. If I have a 3-line sort routine, and Copilot copies that verbatim, it's okay. The place and reason Copilot is not okay is because I can ask it to spit out code which, in some versions, was almost verbatim code I wrote, and in others, a paraphrase. The same abstractions and high-level design I invented is copied by GPT. It's like writing a book set in the Star Trek universe and (not as a parody), changing "Spock" to "Spork" and "Kirk" to "Dirk." It's a problem for me because I compete with commercial systems, and chose the AGPL for very specific reasons: People have a right to know how their data is analyzed, and AGPL preserves that. Copilot allows companies to quickly replicate and whitewash my code. If Microsoft uses a snippet in Windows -- e.g. a small use in an unrelated context, fair use clearly allows that. As much as I don't want my code stolen, degrading fair use would be the far greater evil. 2. "The reason why OpenAI has a non-profit part is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work." No, OpenAI was set up as a genuine, if ill-formed, non-profit. When it clearly became big $$$, bad people (likely breaking the law) formed a for-profit arm to personally profit from it. The non-profit arm is there for historical reasons. Your statement is false, and there is ample historical record that what you're stating is not how it happened. That's dangerous, since the wrong lessons would be drawn. The lesson this reiterates, which I learned many times the hard way, is that for non-profits, checks-and-balances matter. If you don't want your non-profit going south, this means everything -- licenses, corporate charter, bylaws, contracts, etc. -- need to be properly structured to act as a permanent network of checks-and-balances so the organization cannot go south even if bad people are involved, and that incentives are structured to support good behavior. We do need a for-profit like the original OpenAI, but properly structured and resourced.
  - bayindirh 3y ago
    > It is fair use to use snippets of my code in other projects. AGPL doesn't change that. If I have a 3-line sort routine, and Copilot copies that verbatim, it's okay. That depends. You may have a 30 line function which can be paper-worthy. I'm not supporting software patents, but reasonable attribution. If somebody uses my GPL licensed novel algorithm (which is a single function) to get unfair commercial advantage and doesn't adhere to the GPL license, can we say it's fair use? Example? See [0]. > It's a problem for me because I compete with commercial systems, and chose the AGPL for very specific reasons: People have a right to know how their data is analyzed, and AGPL preserves that. Copilot allows companies to quickly replicate and whitewash my code. We're on the same page. I don't want my algorithms to be used by people verbatim w/o attribution or without proper license. Don't like the license? Reimplement it, paper is there. > No, OpenAI was set up as a genuine, if ill-formed, non-profit. ... [snipped for brevity] You're right. Let me rewrite my statement: "The reason why OpenAI keeps a non-profit part and has this convoluted many-company structure is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work." Hope that it's better now. > Your statement is false, and there is ample historical record that what you're stating is not how it happened. About OpenAI's structure, or GPL being fed to CoPilot? I didn't say anything about OpenAI's history AFAIK, but I have seen a tweet, containing a snapshot of an e-mail saying that "Copilot is indeed trained on every public repository on GitHub". The tweet is deleted later probably of threats. Just because of this tweet, I'm keeping this list [1]. Actually we're literally on the same page. Only difference is I keep my "plagiarism" threshold at "function" level, because they can contain some bigger magic than themselves, and you keep it at "design" level where the magic happens as a collection of functions + design. This is probably due to different domains we operate in, in mine, a single function can affect accuracy and speed so much that it can be considered secret sauce. Welcome to HPC / Scientific Computing. :) > We do need a for-profit like the original OpenAI, but properly structured and resourced. When you do for-profit, fair use goes out of window. See [2]. [0]: https://journals.tubitak.gov.tr/elektrik/vol29/iss2/45/ https://journals.tubitak.gov.tr/elektrik/vol29/iss2/45/ [1]: https://notes.bayindirh.io/notes/Lists/Discussions+about+Artificial+Intelligence https://notes.bayindirh.io/notes/Lists/Discussions+about+Art... [2]: https://news.ycombinator.com/item?id=39188979 https://news.ycombinator.com/item?id=39188979
- ghaff 3y ago
  Or, like many IP lawyers, they don't believe it's a copyright violation. It will come down to the courts and legislatures. Being non-profit, or non-commerciality in general, has relatively little to do with fair use.
- victor106 3y ago
  It seems like we need a new licensing model to prevent AI from training on open source code?
  - mnau 3y ago
    That's useless. Only courts can help. NYTimes is suing OpenAI because it siphoned paid articles and reproduced in verbatim. That's basically the same thing it does to open source code, so if NYTimes wins, there might be some hope.
deleted 3y ago
[deleted]
anonymousDan 3y ago
Whether it violates the GPL aside, is there a need for a new license that explicitly forbids what OpenAI has done?
- deleted 3y ago
  [deleted]
- bluGill 3y ago
  would such a license even be possiple?
- DougBTX 3y ago
  No, either this is fair use, so no license is required, or the current licences must be followed (including attribution which seems like the first thing that gets forgotten).
  - palata 3y ago
    But if this is fair use, doesn't it mean that copyright is essentially dead? As in, LLMs are then a big copyright-laundering machine?
pseudocomposer 3y ago
This should mean that all code written with the use of an LLM that was fed AGPL code must be released under the AGPL.
- hatenberg 3y ago
  Technically you can’t copyright it. Proving that is the issue
- jppittma 3y ago
  That makes sense to me. If you use GPL code to train your model, then everything that model produces is a derivative work of that GPL code.
  - josephg 3y ago
    This would be a fascinating argument to make in court.
  - lo_zamoyski 3y ago
    But what is derivative? When does being derivative w.r.t. the GPL mean? Can I copy a function from a GPL’ed project? What if I rename the variables? What if I re-order some of the lines in an inconsequential way? This starts looking like the software patent problem, or the ship of Theseus.
    - anon291 3y ago
      > Can I copy a function from a GPL’ed project? What if I rename the variables? What if I re-order some of the lines in an inconsequential way? All of these are GPL violations if it can be shown you looked at a GPL function while copying this. On the other hand, if you came up with it on your own and it happened to look similar, then it's fine.
      - lo_zamoyski 3y ago
        So, if I looked at a function, understood it, then reproduced the function from my grasp (not necessarily verbatim, but by understanding the logical flow), would that be subject to the GPL? What if it varied? How much does it need to vary not to be considered derivative? If there is a threshold, then it is possible to rewrite the original code in ways that do not resemble the original code verbatim, but are effectively the same.
fredsmith219 3y ago
I disagree. Somewhere out there is an ambitious and voracious law firm just chomping at the bit for a chance at Microsoft’s or OpenAI’s billions. They will sue and the situation will get clarification.
- JonChesterfield 3y ago
  Definitely. It'll take a decade or two and cost millions of dollars. Doesn't actually matter which side wins, GPL will no longer be a thing either way after that long. It's a spectacular play by Microsoft.
- bayindirh 3y ago
  They already started: https://githubcopilotlitigation.com/ https://githubcopilotlitigation.com/
Tadpole9181 3y ago
It's not mentioned because it's neither relevant to this article or true at all? As far as courts are concerned right now, LLMs are simply tools and all existing rules and procedures around copyright still apply: a human is liable for it's use, only the work and skill of a human is copyrightable, etc. Nor are LLMs novel in any way when it comes to copyright violation. Microsoft is already rich enough they could ask developer(s) to reproduce a GPL code base but slightly different and then make an argument of coincidental, parallel thought. Or just pinky promise "we totally didn't steal this." Heck, it'd just be easier to copy and paste without license headers and then just run some basic human-readable obfuscation. Furthermore, AI still isn't even close to being able to pull this off for even remotely complex projects. The context sizes are too small, the models aren't sophisticated enough, etc. They save time as an assistant, but novel output is extremely finicky and prone to errors. Getting a model that can be trusted to do what you're saying would be approaching AGI, in which case the economic ramifications are far more concerning than those on OSS. Finally, how exactly would this "disassemble" OSS? The GPL has been violated for forever, especially behind closed doors. If this would kill it, the GPL is already dead and capitalism killed it. After all, if Microsoft - a company with a revenue stream larger than countries - decides to steal from virtually any OSS projext and the developers - a gaggle of volunteers with few liquid assets - try to take them to court, they're about to engage in a battle of attrition with legal fees. The legal system notoriously favors the rich, which is a far greater threat.
- hnfong 3y ago
  > The context sizes are too small, the models aren't sophisticated enough, etc. Yeah, I wonder what GPL-ed code is actually infringed upon. I vividly remember the time Oracle claimed Google infringed their copyrights because they copied a couple small functions, and the tech community just laughed saying the functions were too trivial. (Mind you, Java was licensed under the GPL.) People harping that the GPL is somehow more affected by LLMs is kind of misguided. If there is copyright infringement, then almost all other licenses would be infringed too, since the vast majority of OSS licenses require keeping the copyright notice at the very least. And the enmity towards Microsoft, whose "fault" is just having acquired shares of OpenAI, as opposed to almost every other Big Tech company who also released LLMs (and obviously injested a lot of github code), feels like the early 2000s again.
concordDance 3y ago
> OpenAI recently fed everything on GitHub into a language model which will cheerfully reproduce GPL'd software without license or attribution. Can you give me a prompt that will do this? I'm a tad skeptical this is possible atm.
- pierat 3y ago
  Use beginning text from https://www.gnu.org/licenses/gpl-3.0.txt https://www.gnu.org/licenses/gpl-3.0.txt
  - concordDance 3y ago
    Tried few differently sized chunks of this. Couldn't reproduce.
newswasboring 3y ago
> model which will cheerfully reproduce GPL'd software without license or attribution. Can I get a citation for this? I have seen it produce code similar to GPL code for trivial things but never a direct reproduction.
- endgame 3y ago
  https://codeium.com/blog/copilot-trains-on-gpl-codeium-does-not https://codeium.com/blog/copilot-trains-on-gpl-codeium-does-...
  - DougBTX 3y ago
    > To prove that GitHub Copilot trains on non permissive licenses, we just disable any post-generation filters and see what GPL code we can generate with minimal context This seems like a key line. I can get my browser to output GPL code with a carefully crafted Google search query, and there might even be unlicensed use of GPL code in my browser itself. But as I understand it, as long as I don’t copy GPL code, I should be fine. In this case they’ve disabled the GPL filter, and gotten GPL’d outputs. Fair enough. The question though: when the filter is on, can I trust it? Not sure!
    - fkyoureadthedoc 3y ago
      Not to mention it seems like most of the blog posts and tweets about this are also very purposefully prompting to get GPL code. Secondly, if you take this seriously you should already being doing some kind of static analysis to detect it because what if one of your employees copy/pastes some GPL code from somewhere? It's practically a non issue that people are blowing out of proportion.
      - palata 3y ago
        > It's practically a non issue that people are blowing out of proportion. I don't think it is. If it is legal to input copyrighted stuff into an LLM and have the LLM magically remove the copyright, then the very concept of copyright is dead. LLMs are then a copyright-laundering machine. Worse: if it is considered "fair use", then I don't think that I could even sell my copyrighted material with a contract that says "you are not allowed to feed it to your copyright-laundering machine". If it is fair use, anyone can do it, right?
tinco 3y ago
Why do you assume it's a legitimate thing to do? Nothing changed from a copyright perspective. You can copy a GPL file directly, or take a photo of it, then decode the JPG, OCR the bitmap and put the result in a file, or you can train a LLM to reproduce it. It's all the same, if a judge feels you've copied a substantial part of the file then it's an infringement. The same way an LLM would generate that code without the license, another LLM could find that code and determine that it infringes. If anything, the emergence of LLM powered agents will empower the free software movement to find infringements. The only way free software would "end", is if judges would not be able to enforce copyright, because for example the copyright infringement is obfuscated too much. For example if LLM powered agents are going to both generate and run the code without ever saving it to disk. We're not quite there yet.
- marshray 3y ago
  I can train a human (such as myself) to reproduce entire song lyrics, verbatim. We have entire professions (musicians, actors) who develop this ability. Is that, in your view, 'illegitimate' too? Why or why not?
  - kwhitefoot 3y ago
    It's legitimate because the law says what is and isn't legitimate. LLMs/Copilot/etc. copying other people's code is not legitimate for precisely the same reason. But big and powerful beats small and week.
    - berkes 3y ago
      Also, those laws say that remembering a song is fine. And playing it in your living room too. But that you'll have to get the licensing right if you play that song in front of an audience, in a pub or on street. At least in most countries. To translate that to LLMs: training an LLM on songs (or code) is fine. It's the output and what you do with that output that might be problematic.
      - 3y ago
lo_zamoyski 3y ago
> OpenAI recently fed everything on GitHub into a language model which will cheerfully reproduce GPL'd software without license or attribution. Currently, when does reproducing code become a violation of the GPL? If I see an interesting function in a project under the GPL, can I copy it? What if I rename the variables? Or what if I reimplement the function exactly? Approximately? Add perturbations Etc, etc. I get ship of Theseus vibes.
anon291 3y ago
> I think that's the end of "free software" in the GPL sense. There is zero point to writing AGPL on a program if copying it into a language model and then back out into a text file without the license is a legitimate thing to do. Why would you think this is 'legitimate'? Firstly, LLMs are not legal agents, only humans are, so any human caught doing this is violating the GPL, whether or not the creation of the LLM is legal or not. Bits have color
- palata 3y ago
  > Why would you think this is 'legitimate'? They say "IF it is legitimate to do X, then ...". They don't say it is legitimate.
  - anon291 3y ago
    Sorry, to be clear, I am saying, regardless of whether it is legitimate to do X, it is still illegitimate to distribute copyrighted code intentionally without following the terms of the license.
    - palata 3y ago
      I tend to agree with the parent: if it is legitimate to take copyrighted code, pass it through an LLM, output it in a new text file, and say "it doesn't have any copyright anymore because it was generated by an LLM", then copyright is essentially dead as a concept. If copyright is dead, then no need for licenses anymore: anyway people can remove your copyright by laundering your code through an LLM.
      - anon291 3y ago
        > : if it is legitimate to take copyrighted code, pass it through an LLM, output it in a new text file, and say "it doesn't have any copyright anymore because it was generated by an LLM", then copyright is essentially dead as a concept. But it's not legitimate. That's what I said. If you (a human) take copyrighted code and redistribute it against the terms of the license, you are guilty of a crime in most places. That was my point, whether the language model was built legally or not, if you have control of such a language model and use it to produce copyrighted code and then distribute that, you will be held as violating copyright. I think there's a lot of confusion here because some on the thread seem to believe a large language model can in and of itself be guilty of a crime. It can't. The trainer may be guilty, or the user could be guilty, or both could be guilty. The model simply is. It might not be redistributable depending on how courts interpret the training procedure (remains to be seen).
federico_c 3y ago
> I think that's the end of "free software" in the GPL sense If anything it's the end of OpenAI
jpambrun 3y ago
Copilot does small-ish snippets. It does not recreate a whole project. I am on the fence about the morality, but it won't recreate elastic search for Amazon to resell as a service.
- berkes 3y ago
  I find It fascinating to see the same-ish group of people arguing that "Algorithms cannot be patented" or "information wants to be free" or "free software, free access" and such, now turning 180° and demanding to control where, how and when information is used. At the very least it's somewhat ironic. At worst it's plain hypocrite.
  - palata 3y ago
    It feels like you completely miss the point of copyleft licenses. They definitely do impose constraints. Saying "I believe that LLMs are abusing my license" is absolutely not hypocritical.
    - jpambrun 3y ago
      If I read your code, learn from it and later write something inspired and based on the knowledge I internalized, am I violating your license?
      - palata 3y ago
        I keep reading this argument, and it's hard for me to grasp how people think it makes sense. Why do people keep comparing an LLM to a human? If I copy-paste your code, can I say that my computer "read your code, learned from it and later wrote something inspired and based on the knowledge it internalized"? If I take a photography of a photography, can I sell it as mine, given that my camera "observed the original and learned from it"? My opinion is that if you read my code, learn from it and later write something inspired and based on the knowledge you internalized, then that is fine. Because you are a human. But if you do anything that it functionally an obfuscated copy-paste and pretend that you actually created that, then that is not fine.
breadwinner 3y ago
Microsoft has indemnified customers who use its Copilot to generate code. If you use code generated by copilot and you're found to be in copyright violation, Microsoft will pay the fines for you. "if a third party sues a commercial customer for copyright infringement for using Microsoft’s Copilots or the output they generate, we will defend the customer and pay the amount of any adverse judgments or settlements that result from the lawsuit, as long as the customer used the guardrails and content filters we have built into our products." https://blogs.microsoft.com/on-the-issues/2023/09/07/copilot-copyright-commitment-ai-legal-concerns/ https://blogs.microsoft.com/on-the-issues/2023/09/07/copilot...
- Dagonfly 3y ago
  As this case shows, it takes over a decade to successfully litigate a pretty obvious copyright violation. So Microsoft feels quite confident in paying for you.
deleted 3y ago
[deleted]

bayindirh 3y ago

People don't want to acknowledge it, because they deem that CoPilot and similar tools are too useful. I personally left the day CoPilot became GA. It's an open secret that xGPL licensed applications are fed into AI models, but opponents' voice is drowned under "We're doing something amazing, we don't need any permission. Plus, it's all fair use, anyway" choir. This also important for Source Available (for your eyes only) licenses. People don't care about them either. This whole AI thing is a boundless ethics violation in every category, yet people don't care, because "it's awesome!" for reasons I don't understand. Addendum: The reason why OpenAI has a non-profit part is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work.

whaleofatw2022 3y ago
We have been conditioned for decades to allow an increasingly slippery slope of dubious behavior in the name of 'cool'
- lo_zamoyski 3y ago
  Cool, or convenient.
  - prox 3y ago
    Cool, convenient and coins
    - ableal 3y ago
      Don't miss collecting the coupons for a free Butlerian Jihad (e.g. https://dune.fandom.com/wiki/Butlerian_Jihad https://dune.fandom.com/wiki/Butlerian_Jihad )
- jstummbillig 3y ago
  As a blanket statement? I don't think so. The reason we even think of anything as "dubious" now, is a moral compass, that has (at the large scale) been staggeringly readjusted for increased human well-being.
myaccountonhn 3y ago
It's also why these days I do not host any of my code on Github. Only then did I realize how much of an iron grip it has on the ecosystem. People expect Github Actions, Pull requests and well now people want to use CoPilot for all their work. It is extremely worrying.
- bayindirh 3y ago
  Me neither. I moved to Source Hut. I'll be restarting my work on my previous research and pondering on whether I should keep its code at a Forgejo instance buried under a VPN, until it's ready to release.
- palata 3y ago
  Same here, I moved to SourceHut. There are other alternatives that look much more like GitHub for those who like that (like Forgejo).
- JonChesterfield 3y ago
  That's the "extend" part of the embrace, extend, extinguish game plan.
blagie 3y ago
As much as I agree with the conclusion (my AGPL code was stolen by OpenAI, as was a lot of other code), your logic is extreme. This is not good. A more reasoned argument is more convincing. 1. "Fair use" It is fair use to use snippets of my code in other projects. AGPL doesn't change that. If I have a 3-line sort routine, and Copilot copies that verbatim, it's okay. The place and reason Copilot is not okay is because I can ask it to spit out code which, in some versions, was almost verbatim code I wrote, and in others, a paraphrase. The same abstractions and high-level design I invented is copied by GPT. It's like writing a book set in the Star Trek universe and (not as a parody), changing "Spock" to "Spork" and "Kirk" to "Dirk." It's a problem for me because I compete with commercial systems, and chose the AGPL for very specific reasons: People have a right to know how their data is analyzed, and AGPL preserves that. Copilot allows companies to quickly replicate and whitewash my code. If Microsoft uses a snippet in Windows -- e.g. a small use in an unrelated context, fair use clearly allows that. As much as I don't want my code stolen, degrading fair use would be the far greater evil. 2. "The reason why OpenAI has a non-profit part is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work." No, OpenAI was set up as a genuine, if ill-formed, non-profit. When it clearly became big $$$, bad people (likely breaking the law) formed a for-profit arm to personally profit from it. The non-profit arm is there for historical reasons. Your statement is false, and there is ample historical record that what you're stating is not how it happened. That's dangerous, since the wrong lessons would be drawn. The lesson this reiterates, which I learned many times the hard way, is that for non-profits, checks-and-balances matter. If you don't want your non-profit going south, this means everything -- licenses, corporate charter, bylaws, contracts, etc. -- need to be properly structured to act as a permanent network of checks-and-balances so the organization cannot go south even if bad people are involved, and that incentives are structured to support good behavior. We do need a for-profit like the original OpenAI, but properly structured and resourced.
- bayindirh 3y ago
  > It is fair use to use snippets of my code in other projects. AGPL doesn't change that. If I have a 3-line sort routine, and Copilot copies that verbatim, it's okay. That depends. You may have a 30 line function which can be paper-worthy. I'm not supporting software patents, but reasonable attribution. If somebody uses my GPL licensed novel algorithm (which is a single function) to get unfair commercial advantage and doesn't adhere to the GPL license, can we say it's fair use? Example? See [0]. > It's a problem for me because I compete with commercial systems, and chose the AGPL for very specific reasons: People have a right to know how their data is analyzed, and AGPL preserves that. Copilot allows companies to quickly replicate and whitewash my code. We're on the same page. I don't want my algorithms to be used by people verbatim w/o attribution or without proper license. Don't like the license? Reimplement it, paper is there. > No, OpenAI was set up as a genuine, if ill-formed, non-profit. ... [snipped for brevity] You're right. Let me rewrite my statement: "The reason why OpenAI keeps a non-profit part and has this convoluted many-company structure is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work." Hope that it's better now. > Your statement is false, and there is ample historical record that what you're stating is not how it happened. About OpenAI's structure, or GPL being fed to CoPilot? I didn't say anything about OpenAI's history AFAIK, but I have seen a tweet, containing a snapshot of an e-mail saying that "Copilot is indeed trained on every public repository on GitHub". The tweet is deleted later probably of threats. Just because of this tweet, I'm keeping this list [1]. Actually we're literally on the same page. Only difference is I keep my "plagiarism" threshold at "function" level, because they can contain some bigger magic than themselves, and you keep it at "design" level where the magic happens as a collection of functions + design. This is probably due to different domains we operate in, in mine, a single function can affect accuracy and speed so much that it can be considered secret sauce. Welcome to HPC / Scientific Computing. :) > We do need a for-profit like the original OpenAI, but properly structured and resourced. When you do for-profit, fair use goes out of window. See [2]. [0]: https://journals.tubitak.gov.tr/elektrik/vol29/iss2/45/ https://journals.tubitak.gov.tr/elektrik/vol29/iss2/45/ [1]: https://notes.bayindirh.io/notes/Lists/Discussions+about+Artificial+Intelligence https://notes.bayindirh.io/notes/Lists/Discussions+about+Art... [2]: https://news.ycombinator.com/item?id=39188979 https://news.ycombinator.com/item?id=39188979
ghaff 3y ago
Or, like many IP lawyers, they don't believe it's a copyright violation. It will come down to the courts and legislatures. Being non-profit, or non-commerciality in general, has relatively little to do with fair use.
victor106 3y ago
It seems like we need a new licensing model to prevent AI from training on open source code?
- mnau 3y ago
  That's useless. Only courts can help. NYTimes is suing OpenAI because it siphoned paid articles and reproduced in verbatim. That's basically the same thing it does to open source code, so if NYTimes wins, there might be some hope.

deleted 3y ago

[deleted]

anonymousDan 3y ago

Whether it violates the GPL aside, is there a need for a new license that explicitly forbids what OpenAI has done?

deleted 3y ago
[deleted]
bluGill 3y ago
would such a license even be possiple?
DougBTX 3y ago
No, either this is fair use, so no license is required, or the current licences must be followed (including attribution which seems like the first thing that gets forgotten).
- palata 3y ago
  But if this is fair use, doesn't it mean that copyright is essentially dead? As in, LLMs are then a big copyright-laundering machine?

pseudocomposer 3y ago

This should mean that all code written with the use of an LLM that was fed AGPL code must be released under the AGPL.

hatenberg 3y ago
Technically you can’t copyright it. Proving that is the issue
jppittma 3y ago
That makes sense to me. If you use GPL code to train your model, then everything that model produces is a derivative work of that GPL code.
- josephg 3y ago
  This would be a fascinating argument to make in court.
- lo_zamoyski 3y ago
  But what is derivative? When does being derivative w.r.t. the GPL mean? Can I copy a function from a GPL’ed project? What if I rename the variables? What if I re-order some of the lines in an inconsequential way? This starts looking like the software patent problem, or the ship of Theseus.
  - anon291 3y ago
    > Can I copy a function from a GPL’ed project? What if I rename the variables? What if I re-order some of the lines in an inconsequential way? All of these are GPL violations if it can be shown you looked at a GPL function while copying this. On the other hand, if you came up with it on your own and it happened to look similar, then it's fine.
    - lo_zamoyski 3y ago
      So, if I looked at a function, understood it, then reproduced the function from my grasp (not necessarily verbatim, but by understanding the logical flow), would that be subject to the GPL? What if it varied? How much does it need to vary not to be considered derivative? If there is a threshold, then it is possible to rewrite the original code in ways that do not resemble the original code verbatim, but are effectively the same.

fredsmith219 3y ago

I disagree. Somewhere out there is an ambitious and voracious law firm just chomping at the bit for a chance at Microsoft’s or OpenAI’s billions. They will sue and the situation will get clarification.

JonChesterfield 3y ago
Definitely. It'll take a decade or two and cost millions of dollars. Doesn't actually matter which side wins, GPL will no longer be a thing either way after that long. It's a spectacular play by Microsoft.
bayindirh 3y ago
They already started: https://githubcopilotlitigation.com/ https://githubcopilotlitigation.com/

Tadpole9181 3y ago

It's not mentioned because it's neither relevant to this article or true at all? As far as courts are concerned right now, LLMs are simply tools and all existing rules and procedures around copyright still apply: a human is liable for it's use, only the work and skill of a human is copyrightable, etc. Nor are LLMs novel in any way when it comes to copyright violation. Microsoft is already rich enough they could ask developer(s) to reproduce a GPL code base but slightly different and then make an argument of coincidental, parallel thought. Or just pinky promise "we totally didn't steal this." Heck, it'd just be easier to copy and paste without license headers and then just run some basic human-readable obfuscation. Furthermore, AI still isn't even close to being able to pull this off for even remotely complex projects. The context sizes are too small, the models aren't sophisticated enough, etc. They save time as an assistant, but novel output is extremely finicky and prone to errors. Getting a model that can be trusted to do what you're saying would be approaching AGI, in which case the economic ramifications are far more concerning than those on OSS. Finally, how exactly would this "disassemble" OSS? The GPL has been violated for forever, especially behind closed doors. If this would kill it, the GPL is already dead and capitalism killed it. After all, if Microsoft - a company with a revenue stream larger than countries - decides to steal from virtually any OSS projext and the developers - a gaggle of volunteers with few liquid assets - try to take them to court, they're about to engage in a battle of attrition with legal fees. The legal system notoriously favors the rich, which is a far greater threat.

hnfong 3y ago
> The context sizes are too small, the models aren't sophisticated enough, etc. Yeah, I wonder what GPL-ed code is actually infringed upon. I vividly remember the time Oracle claimed Google infringed their copyrights because they copied a couple small functions, and the tech community just laughed saying the functions were too trivial. (Mind you, Java was licensed under the GPL.) People harping that the GPL is somehow more affected by LLMs is kind of misguided. If there is copyright infringement, then almost all other licenses would be infringed too, since the vast majority of OSS licenses require keeping the copyright notice at the very least. And the enmity towards Microsoft, whose "fault" is just having acquired shares of OpenAI, as opposed to almost every other Big Tech company who also released LLMs (and obviously injested a lot of github code), feels like the early 2000s again.

concordDance 3y ago

> OpenAI recently fed everything on GitHub into a language model which will cheerfully reproduce GPL'd software without license or attribution. Can you give me a prompt that will do this? I'm a tad skeptical this is possible atm.

pierat 3y ago
Use beginning text from https://www.gnu.org/licenses/gpl-3.0.txt https://www.gnu.org/licenses/gpl-3.0.txt
- concordDance 3y ago
  Tried few differently sized chunks of this. Couldn't reproduce.

newswasboring 3y ago

> model which will cheerfully reproduce GPL'd software without license or attribution. Can I get a citation for this? I have seen it produce code similar to GPL code for trivial things but never a direct reproduction.

endgame 3y ago
https://codeium.com/blog/copilot-trains-on-gpl-codeium-does-not https://codeium.com/blog/copilot-trains-on-gpl-codeium-does-...
- DougBTX 3y ago
  > To prove that GitHub Copilot trains on non permissive licenses, we just disable any post-generation filters and see what GPL code we can generate with minimal context This seems like a key line. I can get my browser to output GPL code with a carefully crafted Google search query, and there might even be unlicensed use of GPL code in my browser itself. But as I understand it, as long as I don’t copy GPL code, I should be fine. In this case they’ve disabled the GPL filter, and gotten GPL’d outputs. Fair enough. The question though: when the filter is on, can I trust it? Not sure!
  - fkyoureadthedoc 3y ago
    Not to mention it seems like most of the blog posts and tweets about this are also very purposefully prompting to get GPL code. Secondly, if you take this seriously you should already being doing some kind of static analysis to detect it because what if one of your employees copy/pastes some GPL code from somewhere? It's practically a non issue that people are blowing out of proportion.
    - palata 3y ago
      > It's practically a non issue that people are blowing out of proportion. I don't think it is. If it is legal to input copyrighted stuff into an LLM and have the LLM magically remove the copyright, then the very concept of copyright is dead. LLMs are then a copyright-laundering machine. Worse: if it is considered "fair use", then I don't think that I could even sell my copyrighted material with a contract that says "you are not allowed to feed it to your copyright-laundering machine". If it is fair use, anyone can do it, right?

tinco 3y ago

Why do you assume it's a legitimate thing to do? Nothing changed from a copyright perspective. You can copy a GPL file directly, or take a photo of it, then decode the JPG, OCR the bitmap and put the result in a file, or you can train a LLM to reproduce it. It's all the same, if a judge feels you've copied a substantial part of the file then it's an infringement. The same way an LLM would generate that code without the license, another LLM could find that code and determine that it infringes. If anything, the emergence of LLM powered agents will empower the free software movement to find infringements. The only way free software would "end", is if judges would not be able to enforce copyright, because for example the copyright infringement is obfuscated too much. For example if LLM powered agents are going to both generate and run the code without ever saving it to disk. We're not quite there yet.

marshray 3y ago
I can train a human (such as myself) to reproduce entire song lyrics, verbatim. We have entire professions (musicians, actors) who develop this ability. Is that, in your view, 'illegitimate' too? Why or why not?
- kwhitefoot 3y ago
  It's legitimate because the law says what is and isn't legitimate. LLMs/Copilot/etc. copying other people's code is not legitimate for precisely the same reason. But big and powerful beats small and week.
  - berkes 3y ago
    Also, those laws say that remembering a song is fine. And playing it in your living room too. But that you'll have to get the licensing right if you play that song in front of an audience, in a pub or on street. At least in most countries. To translate that to LLMs: training an LLM on songs (or code) is fine. It's the output and what you do with that output that might be problematic.
    - 3y ago

lo_zamoyski 3y ago

> OpenAI recently fed everything on GitHub into a language model which will cheerfully reproduce GPL'd software without license or attribution. Currently, when does reproducing code become a violation of the GPL? If I see an interesting function in a project under the GPL, can I copy it? What if I rename the variables? Or what if I reimplement the function exactly? Approximately? Add perturbations Etc, etc. I get ship of Theseus vibes.

anon291 3y ago

> I think that's the end of "free software" in the GPL sense. There is zero point to writing AGPL on a program if copying it into a language model and then back out into a text file without the license is a legitimate thing to do. Why would you think this is 'legitimate'? Firstly, LLMs are not legal agents, only humans are, so any human caught doing this is violating the GPL, whether or not the creation of the LLM is legal or not. Bits have color

palata 3y ago
> Why would you think this is 'legitimate'? They say "IF it is legitimate to do X, then ...". They don't say it is legitimate.
- anon291 3y ago
  Sorry, to be clear, I am saying, regardless of whether it is legitimate to do X, it is still illegitimate to distribute copyrighted code intentionally without following the terms of the license.
  - palata 3y ago
    I tend to agree with the parent: if it is legitimate to take copyrighted code, pass it through an LLM, output it in a new text file, and say "it doesn't have any copyright anymore because it was generated by an LLM", then copyright is essentially dead as a concept. If copyright is dead, then no need for licenses anymore: anyway people can remove your copyright by laundering your code through an LLM.
    - anon291 3y ago
      > : if it is legitimate to take copyrighted code, pass it through an LLM, output it in a new text file, and say "it doesn't have any copyright anymore because it was generated by an LLM", then copyright is essentially dead as a concept. But it's not legitimate. That's what I said. If you (a human) take copyrighted code and redistribute it against the terms of the license, you are guilty of a crime in most places. That was my point, whether the language model was built legally or not, if you have control of such a language model and use it to produce copyrighted code and then distribute that, you will be held as violating copyright. I think there's a lot of confusion here because some on the thread seem to believe a large language model can in and of itself be guilty of a crime. It can't. The trainer may be guilty, or the user could be guilty, or both could be guilty. The model simply is. It might not be redistributable depending on how courts interpret the training procedure (remains to be seen).

federico_c 3y ago

> I think that's the end of "free software" in the GPL sense If anything it's the end of OpenAI

jpambrun 3y ago

Copilot does small-ish snippets. It does not recreate a whole project. I am on the fence about the morality, but it won't recreate elastic search for Amazon to resell as a service.

berkes 3y ago
I find It fascinating to see the same-ish group of people arguing that "Algorithms cannot be patented" or "information wants to be free" or "free software, free access" and such, now turning 180° and demanding to control where, how and when information is used. At the very least it's somewhat ironic. At worst it's plain hypocrite.
- palata 3y ago
  It feels like you completely miss the point of copyleft licenses. They definitely do impose constraints. Saying "I believe that LLMs are abusing my license" is absolutely not hypocritical.
  - jpambrun 3y ago
    If I read your code, learn from it and later write something inspired and based on the knowledge I internalized, am I violating your license?
    - palata 3y ago
      I keep reading this argument, and it's hard for me to grasp how people think it makes sense. Why do people keep comparing an LLM to a human? If I copy-paste your code, can I say that my computer "read your code, learned from it and later wrote something inspired and based on the knowledge it internalized"? If I take a photography of a photography, can I sell it as mine, given that my camera "observed the original and learned from it"? My opinion is that if you read my code, learn from it and later write something inspired and based on the knowledge you internalized, then that is fine. Because you are a human. But if you do anything that it functionally an obfuscated copy-paste and pretend that you actually created that, then that is not fine.

breadwinner 3y ago

Microsoft has indemnified customers who use its Copilot to generate code. If you use code generated by copilot and you're found to be in copyright violation, Microsoft will pay the fines for you. "if a third party sues a commercial customer for copyright infringement for using Microsoft’s Copilots or the output they generate, we will defend the customer and pay the amount of any adverse judgments or settlements that result from the lawsuit, as long as the customer used the guardrails and content filters we have built into our products." https://blogs.microsoft.com/on-the-issues/2023/09/07/copilot-copyright-commitment-ai-legal-concerns/ https://blogs.microsoft.com/on-the-issues/2023/09/07/copilot...

Dagonfly 3y ago
As this case shows, it takes over a decade to successfully litigate a pretty obvious copyright violation. So Microsoft feels quite confident in paying for you.

deleted 3y ago

[deleted]