5 ms·
> I sincerely hope that one of the many court cases produces a verdict that says AI-generated code is, in fact, subject to the licenses of the inputs. Then ther
by kouteiheika 3y ago
> I sincerely hope that one of the many court cases produces a verdict that says AI-generated code is, in fact, subject to the licenses of the inputs. Then there will be a lot of screaming and wailing, as people go "but how can we train AI if we have to respect licenses?!". And then people will figure out how to actually respect Open Source software licenses (and, for that matter, proprietary ones).
I think this is an incredibly short sighted take.
Training a modern state-of-art LLM needs terabytes of training data. It's probably not going to be practically possible to actually license this much data. And even if we assume it will be then the only entities which will be able to do it are the world's biggest corporations.
So if AI models are a derivative work of their training data then as a consequence the whole field will mostly die overnight, with perhaps only a few of the world's richest corporations being able to play with this space.
I'm sure you're aware of the slew of really cool open/free models that are out there, which you can download today and play with on your local machine. Like Stable Diffusion. It was trained on all rights reserved data. Now it's going to be illegal. Or OpenLLaMA/Falcon/MPT/etc. Also trained on all rights reserved data. Illegal too.
Have you heard about the Pile dataset[1]? It's the most popular open dataset for training LLMs, and essentially every non-proprietary LLM is trained on it, or on parts of it. Do you know that it contains 100GB of all-rights-reserved pirated ebooks? If AI models are a derivative work of their training data then all of those models are now illegal.
[1] - https://pile.eleuther.ai https://pile.eleuther.ai
This would completely kill any chance of having good free/open models. Sure, you could then grab all of GPL'd code, train a model on that, and maybe have a decent working GPL'd model that can emit GPL'd code. There might be enough data for that. But that's it. What about other kinds of models? Image generation, chat bots, personal assistants, story writers, etc. There's just not enough freely licensed data (and probably will never be) to train those.
We must democratize this space. It's already insanely expensive to train state-of-art LLMs; we don't need to make it even more expensive. It's not going to stop OpenAI. It's not going to stop Microsoft. They'll figure it out. What it'll stop is everyone else, and will make this technology completely out of reach for everyone who isn't an insanely rich multinational corporation.
With all due respect, to everyone who's complaining about Microsoft using your code to train Codex, I'll be blunt: you're advocating for collectively shooting all of us in the foot. I don't want to live in a dystopia where only huge corporations will have access to cutting edge AI technology, so please stop pushing in that direction by advocating for more draconian copyright just because you're butthurt that Microsoft/OpenAI used your code as training data without asking for permission.
Big corporations are not the only people doing this; the little guys (e.g. EleutherAI) who release free and open source models do it too. If you try to block the big bad evil corporation from doing it you'll also block the little guys. The major difference here is that the little guys will get completely screwed by this while the corporation has the cash to try to get around it.
- spongebobstoes 3y agoI completely agree that expanding copyright only hurts us all. The big companies will be able to work around it. I think if anything, this helps to demonstrate the absurdity of ancient copyright law in our modern world. I would rather move in the direction of abolishing copyright rather than in the direction of strengthening it.
- JoshTriplett 3y ago> I completely agree that expanding copyright only hurts us all. As do I, but this is not expanding copyright. This is saying that as long as copyright exists, AI model training has to respect it too, and can't be a laundering operation for license violations. Otherwise, you're giving companies building AI extra permission to ignore Open Source licenses and small copyright holders (e.g. independent artists, authors of stories/comments/text on the Internet), which is an asymmetry in favor of companies with the resources to train huge models. I'm all for abolishing copyright, and I think it is absurd. What I'm against here is the asymmetry of keeping all the harms of copyright around while letting AI training violate it.
- kouteiheika 3y ago> Otherwise, you're giving companies building AI extra permission to ignore Open Source licenses and small copyright holders (e.g. independent artists, authors of stories/comments/text on the Internet), which is an asymmetry in favor of companies with the resources to train huge models. From my experience this is incorrect; it's not an asymmetry in favor of companies; if anything it's an asymmetry in favor of small players. How do I know? Because I myself am one of those small players, and because I train machine learning models myself. Not huge ones of course, but ones which need data in the range of e.g. 10~25GB range. As an individual it would be completely impossible to explicitly license this data for training (most of which is owned, mind you, by huge corporations!), while a corporation wouldn't have a problem with it. > What I'm against here is the asymmetry of keeping all the harms of copyright around while letting AI training violate it. This is fair enough. But please beware that the blast radius of entities your proposal would harm is not limited to huge corporations. I would prefer to fundamentally reduce the scope of copyright too instead of carving out special cases for AI training, but we both know that isn't going to happen.