5 ms·

We don't need Open Source licenses to change to deal with AI. We need AI to respect Open Source licenses, or not use code under those licenses. I sincerely hop

We don't need Open Source licenses to change to deal with AI. We need AI to respect Open Source licenses, or not use code under those licenses. I sincerely hope that one of the many court cases produces a verdict that says AI-generated code is, in fact, subject to the licenses of the inputs. Then there will be a lot of screaming and wailing, as people go "but how can we train AI if we have to respect licenses?!". And then people will figure out how to actually respect Open Source software licenses (and, for that matter, proprietary ones).

est31 3y ago
I think if this existed, then it would benefit the current owners of the large IP pools the most. Currently, yes, many think they can use models trained on OSS code to create proprietary software. In general, proprietary software is bad but it's way worse to have a scarcity of models because they are owned by large IP holders. In Github's case for example, Github's TOS already includes a clause that if you upload code there, you grant Github a license to use the content to run Github's services... and copilot is one of them. https://docs.github.com/en/site-policy/github-terms/github-terms-of-service#4-license-grant-to-us https://docs.github.com/en/site-policy/github-terms/github-t... Such clauses are commonly found in social media where users can upload content. Think of imgur, instagam, reddit, etc. OpenAI might buy reddit, and declare ChatGPT a product of the reddit service, then all discussions on reddit could be used for the training of ChatGPT... while open models can't access the data.
- JoshTriplett 3y ago
  > In Github's case for example, Github's TOS already includes a clause that if you upload code there, you grant Github a license to use the content to run Github's services... and copilot is one of them. You can't grant permission for something you don't own. Uploading a copy of a GPLed work to GitHub does not grant GitHub permission to ignore the GPL. (It might grant GitHub permission to ignore your copyrights in that work, maybe, though it seems like a stretch to argue that "run Github's services" includes "give other people derivative works of all your code"; arguably that ought to be too broad for a contract of adhesion. There's case law about what you can and can't do in a unilaterally imposed contract such as a ToS; a ToS can't say "you owe us $100 if you browse more than twelve pages" either, and codebases can be worth far more than that.) If GitHub started saying "one of our services is to give people access to copies of your code with the licenses and copyright notices removed", the GitHub ToS wouldn't suddenly make it acceptable to run that "service". But I agree with part of your underlying point. All AI models should respect Open Source licenses. It's a problem if some try to work around that.
  - est31 3y ago
    Fair points, but note that Microsoft charges a lot more than $100 to inspect the source code of Windows (but it is available, at least to really large customers, think governments).
grumbel 3y ago
> I sincerely hope that one of the many court cases produces a verdict that says AI-generated code is, in fact, subject to the licenses of the inputs. Such a strict interpretation of copyright would kill Open Source, as any attempt at reverse engineering would be disallowed by that. Furthermore it would mean that only big and rich companies would have AI, as they can do behind closest doors whatever they want with AI, Open Source licenses only cover redistribution after all, which internal use doesn't fall under. Meanwhile any attempt at publicly available open AI models would instantly get killed by copyright claims.
- JoshTriplett 3y ago
  > Such a strict interpretation of copyright would kill Open Source, as any attempt at reverse engineering would be disallowed by that. Not at all, for multiple reasons. Reverse-engineering already has that problem: if any of the people who do the reverse engineering also work on the code, it's entirely possible for some of the reverse-engineered code to end up in the new code, making the new code a derivative work of the original. There are standard ways to carefully avoid that: https://en.wikipedia.org/wiki/Clean_room_design https://en.wikipedia.org/wiki/Clean_room_design (Also, reverse engineering is a small fraction of Open Source.) > Furthermore it would mean that only big and rich companies would have AI, as they can do behind closest doors whatever they want with AI, Open Source licenses only cover redistribution after all, which internal use doesn't fall under. The moment they distribute anything written by the model, the same problem applies. And if they don't redistribute anything written by the model, then sure, they can do anything they like, just as you're free to internally combine GPLed and proprietary code if you never ship the result. (Note, though, that many companies have figured out it's a bad idea to do this, because it creates a combination you cannot ever distribute, and circumstances might change in the future to lead you to want to distribute it.) > Meanwhile any attempt at publicly available open AI models would instantly get killed by copyright claims. No, they just need to actually pay attention to the licenses of work they train on. Train on permissively licensed code, document every codebase trained on, and record the licenses and copyright notices.
clintfred 3y ago
Genuine question here; not trying to be snarky. How is AI "reading" code different from me reading code? Is the difference the AI's ability for perfect memory? I can read open source code, (even GPL) and not have all future code I independently write be subject to that license. I don't think anyone would argue that I immediately "forget" any OSS code that I read, so it's becoming part of the structure of my brain (and potentially influencing future code I wrote), but unless I'm linking to the code or copying pieces out, verbatim, I'm generally in the clear. Of course there are some sticky situations clean-room, reverse engineering, but those seem like pretty narrow examples.
- hgs3 3y ago
  Because many programmers who open sourced their code intended it to be read by humans, not AI. They don't want some centralized super computer owned by a mega-corporation reading their code. At the very least, if the models were Free and Open Source, the reaction might be different.
  - g_delgado14 3y ago
    If you host your code on GitHub and the like, then your code is already being read by a centralized mega-corp.
- cgearhart 3y ago
  Part of what LLMs do is compress their training dataset into the weights, often with character-perfect recall later. For example, I would be shocked if any sufficiently large LLM failed when prompted “write the quake fast inverse square root algorithm verbatim”. (I’m not really interested in arguing whether that’s all they do, or whether it’s the purpose of LLMs—those details are just a distraction from the original question: what makes LLM training different than a human reading code.) If the model has memorized the training set and can reproduce it verbatim when prompted, then it should be incumbent on the AI owner to prove that it does not reproduce copyrighted code when it is not explicitly prompted.
  - IshKebab 3y ago
kouteiheika 3y ago
> I sincerely hope that one of the many court cases produces a verdict that says AI-generated code is, in fact, subject to the licenses of the inputs. Then there will be a lot of screaming and wailing, as people go "but how can we train AI if we have to respect licenses?!". And then people will figure out how to actually respect Open Source software licenses (and, for that matter, proprietary ones). I think this is an incredibly short sighted take. Training a modern state-of-art LLM needs terabytes of training data. It's probably not going to be practically possible to actually license this much data. And even if we assume it will be then the only entities which will be able to do it are the world's biggest corporations. So if AI models are a derivative work of their training data then as a consequence the whole field will mostly die overnight, with perhaps only a few of the world's richest corporations being able to play with this space. I'm sure you're aware of the slew of really cool open/free models that are out there, which you can download today and play with on your local machine. Like Stable Diffusion. It was trained on all rights reserved data. Now it's going to be illegal. Or OpenLLaMA/Falcon/MPT/etc. Also trained on all rights reserved data. Illegal too. Have you heard about the Pile dataset[1]? It's the most popular open dataset for training LLMs, and essentially every non-proprietary LLM is trained on it, or on parts of it. Do you know that it contains 100GB of all-rights-reserved pirated ebooks? If AI models are a derivative work of their training data then all of those models are now illegal. [1] - https://pile.eleuther.ai https://pile.eleuther.ai This would completely kill any chance of having good free/open models. Sure, you could then grab all of GPL'd code, train a model on that, and maybe have a decent working GPL'd model that can emit GPL'd code. There might be enough data for that. But that's it. What about other kinds of models? Image generation, chat bots, personal assistants, story writers, etc. There's just not enough freely licensed data (and probably will never be) to train those. We must democratize this space. It's already insanely expensive to train state-of-art LLMs; we don't need to make it even more expensive. It's not going to stop OpenAI. It's not going to stop Microsoft. They'll figure it out. What it'll stop is everyone else, and will make this technology completely out of reach for everyone who isn't an insanely rich multinational corporation. With all due respect, to everyone who's complaining about Microsoft using your code to train Codex, I'll be blunt: you're advocating for collectively shooting all of us in the foot. I don't want to live in a dystopia where only huge corporations will have access to cutting edge AI technology, so please stop pushing in that direction by advocating for more draconian copyright just because you're butthurt that Microsoft/OpenAI used your code as training data without asking for permission. Big corporations are not the only people doing this; the little guys (e.g. EleutherAI) who release free and open source models do it too. If you try to block the big bad evil corporation from doing it you'll also block the little guys. The major difference here is that the little guys will get completely screwed by this while the corporation has the cash to try to get around it.

est31 3y ago

I think if this existed, then it would benefit the current owners of the large IP pools the most. Currently, yes, many think they can use models trained on OSS code to create proprietary software. In general, proprietary software is bad but it's way worse to have a scarcity of models because they are owned by large IP holders. In Github's case for example, Github's TOS already includes a clause that if you upload code there, you grant Github a license to use the content to run Github's services... and copilot is one of them. https://docs.github.com/en/site-policy/github-terms/github-terms-of-service#4-license-grant-to-us https://docs.github.com/en/site-policy/github-terms/github-t... Such clauses are commonly found in social media where users can upload content. Think of imgur, instagam, reddit, etc. OpenAI might buy reddit, and declare ChatGPT a product of the reddit service, then all discussions on reddit could be used for the training of ChatGPT... while open models can't access the data.

JoshTriplett 3y ago
> In Github's case for example, Github's TOS already includes a clause that if you upload code there, you grant Github a license to use the content to run Github's services... and copilot is one of them. You can't grant permission for something you don't own. Uploading a copy of a GPLed work to GitHub does not grant GitHub permission to ignore the GPL. (It might grant GitHub permission to ignore your copyrights in that work, maybe, though it seems like a stretch to argue that "run Github's services" includes "give other people derivative works of all your code"; arguably that ought to be too broad for a contract of adhesion. There's case law about what you can and can't do in a unilaterally imposed contract such as a ToS; a ToS can't say "you owe us $100 if you browse more than twelve pages" either, and codebases can be worth far more than that.) If GitHub started saying "one of our services is to give people access to copies of your code with the licenses and copyright notices removed", the GitHub ToS wouldn't suddenly make it acceptable to run that "service". But I agree with part of your underlying point. All AI models should respect Open Source licenses. It's a problem if some try to work around that.
- est31 3y ago
  Fair points, but note that Microsoft charges a lot more than $100 to inspect the source code of Windows (but it is available, at least to really large customers, think governments).

grumbel 3y ago

> I sincerely hope that one of the many court cases produces a verdict that says AI-generated code is, in fact, subject to the licenses of the inputs. Such a strict interpretation of copyright would kill Open Source, as any attempt at reverse engineering would be disallowed by that. Furthermore it would mean that only big and rich companies would have AI, as they can do behind closest doors whatever they want with AI, Open Source licenses only cover redistribution after all, which internal use doesn't fall under. Meanwhile any attempt at publicly available open AI models would instantly get killed by copyright claims.

JoshTriplett 3y ago
> Such a strict interpretation of copyright would kill Open Source, as any attempt at reverse engineering would be disallowed by that. Not at all, for multiple reasons. Reverse-engineering already has that problem: if any of the people who do the reverse engineering also work on the code, it's entirely possible for some of the reverse-engineered code to end up in the new code, making the new code a derivative work of the original. There are standard ways to carefully avoid that: https://en.wikipedia.org/wiki/Clean_room_design https://en.wikipedia.org/wiki/Clean_room_design (Also, reverse engineering is a small fraction of Open Source.) > Furthermore it would mean that only big and rich companies would have AI, as they can do behind closest doors whatever they want with AI, Open Source licenses only cover redistribution after all, which internal use doesn't fall under. The moment they distribute anything written by the model, the same problem applies. And if they don't redistribute anything written by the model, then sure, they can do anything they like, just as you're free to internally combine GPLed and proprietary code if you never ship the result. (Note, though, that many companies have figured out it's a bad idea to do this, because it creates a combination you cannot ever distribute, and circumstances might change in the future to lead you to want to distribute it.) > Meanwhile any attempt at publicly available open AI models would instantly get killed by copyright claims. No, they just need to actually pay attention to the licenses of work they train on. Train on permissively licensed code, document every codebase trained on, and record the licenses and copyright notices.

clintfred 3y ago

Genuine question here; not trying to be snarky. How is AI "reading" code different from me reading code? Is the difference the AI's ability for perfect memory? I can read open source code, (even GPL) and not have all future code I independently write be subject to that license. I don't think anyone would argue that I immediately "forget" any OSS code that I read, so it's becoming part of the structure of my brain (and potentially influencing future code I wrote), but unless I'm linking to the code or copying pieces out, verbatim, I'm generally in the clear. Of course there are some sticky situations clean-room, reverse engineering, but those seem like pretty narrow examples.

hgs3 3y ago
Because many programmers who open sourced their code intended it to be read by humans, not AI. They don't want some centralized super computer owned by a mega-corporation reading their code. At the very least, if the models were Free and Open Source, the reaction might be different.
- g_delgado14 3y ago
  If you host your code on GitHub and the like, then your code is already being read by a centralized mega-corp.
cgearhart 3y ago
Part of what LLMs do is compress their training dataset into the weights, often with character-perfect recall later. For example, I would be shocked if any sufficiently large LLM failed when prompted “write the quake fast inverse square root algorithm verbatim”. (I’m not really interested in arguing whether that’s all they do, or whether it’s the purpose of LLMs—those details are just a distraction from the original question: what makes LLM training different than a human reading code.) If the model has memorized the training set and can reproduce it verbatim when prompted, then it should be incumbent on the AI owner to prove that it does not reproduce copyrighted code when it is not explicitly prompted.
- IshKebab 3y ago

kouteiheika 3y ago

> I sincerely hope that one of the many court cases produces a verdict that says AI-generated code is, in fact, subject to the licenses of the inputs. Then there will be a lot of screaming and wailing, as people go "but how can we train AI if we have to respect licenses?!". And then people will figure out how to actually respect Open Source software licenses (and, for that matter, proprietary ones). I think this is an incredibly short sighted take. Training a modern state-of-art LLM needs terabytes of training data. It's probably not going to be practically possible to actually license this much data. And even if we assume it will be then the only entities which will be able to do it are the world's biggest corporations. So if AI models are a derivative work of their training data then as a consequence the whole field will mostly die overnight, with perhaps only a few of the world's richest corporations being able to play with this space. I'm sure you're aware of the slew of really cool open/free models that are out there, which you can download today and play with on your local machine. Like Stable Diffusion. It was trained on all rights reserved data. Now it's going to be illegal. Or OpenLLaMA/Falcon/MPT/etc. Also trained on all rights reserved data. Illegal too. Have you heard about the Pile dataset[1]? It's the most popular open dataset for training LLMs, and essentially every non-proprietary LLM is trained on it, or on parts of it. Do you know that it contains 100GB of all-rights-reserved pirated ebooks? If AI models are a derivative work of their training data then all of those models are now illegal. [1] - https://pile.eleuther.ai https://pile.eleuther.ai This would completely kill any chance of having good free/open models. Sure, you could then grab all of GPL'd code, train a model on that, and maybe have a decent working GPL'd model that can emit GPL'd code. There might be enough data for that. But that's it. What about other kinds of models? Image generation, chat bots, personal assistants, story writers, etc. There's just not enough freely licensed data (and probably will never be) to train those. We must democratize this space. It's already insanely expensive to train state-of-art LLMs; we don't need to make it even more expensive. It's not going to stop OpenAI. It's not going to stop Microsoft. They'll figure it out. What it'll stop is everyone else, and will make this technology completely out of reach for everyone who isn't an insanely rich multinational corporation. With all due respect, to everyone who's complaining about Microsoft using your code to train Codex, I'll be blunt: you're advocating for collectively shooting all of us in the foot. I don't want to live in a dystopia where only huge corporations will have access to cutting edge AI technology, so please stop pushing in that direction by advocating for more draconian copyright just because you're butthurt that Microsoft/OpenAI used your code as training data without asking for permission. Big corporations are not the only people doing this; the little guys (e.g. EleutherAI) who release free and open source models do it too. If you try to block the big bad evil corporation from doing it you'll also block the little guys. The major difference here is that the little guys will get completely screwed by this while the corporation has the cash to try to get around it.