3 ms·

Realistically, the copyrighted works that are most valuable to training machine learning models, at least if we go by The Pile as typical of training data [1] i

Realistically, the copyrighted works that are most valuable to training machine learning models, at least if we go by The Pile as typical of training data [1] is: - Web pages; hard to argue that royalties are due since these are publicly available for free - Scientific papers; these do cost money but the copyright is typically owned by scientific publishers - Github, Stack Exchange, HN (yes); these are freely available, sometimes by license, so hard to argue for royalties - Wikipedia, Project Gutenberg; these are also free by license So the actual consequence of what you're proposing (or at least the realistically-enactable version of it) is the big AI firms paying scientific publishers a lot of money. Is this actually good? Is Elsevier, a basically pure rent-seeker, really more worthy than AI labs, which maybe you don't like but at least do something valuable? [1]: https://en.wikipedia.org/wiki/The_Pile_(dataset) https://en.wikipedia.org/wiki/The_Pile_(dataset)

kevingadd 2y ago
Scientific papers are available on web pages so they're publicly available for free too right? I can download an ISO or installer of almost anything if I go to the right website, so all software is already free? If you're going to ignore the existence of copyright and licenses we should extend it to everything that's ever been posted on the internet, not just "web pages". Why shouldn't all books and films count as free too? I'm actually open to the idea of just abolishing copyright but it's kind of silly to act like it's only about Elsevier. Lots of creatives depend on copyright in order to earn a living, similarly to how patents fund a lot of important research despite how noxious the patent system has become. If we fixate on examples like Elsevier or Martin Shkreli in order to argue for completely abolishing the copyright or patent systems we risk destroying the framework that enables valuable creative works or new technologies to be developed in the first place. This is part of why people are so upset by AI companies arguing that they should just be able to ignore the whole framework in order to enrich themselves; once you allow the for-profit AI companies to do it, other groups are going to line up to also demand a free ride.
- Kim_Bruning 2y ago
  > We risk destroying the framework that enables valuable creative works or new technologies to be developed in the first place I think we are starting to see not just historical empirical data but also strategic intelligence that this assumption might be somewhat inaccurate. - which is what Anna's archive is pointing out here in the context of AI development and copyright restrictions.
- martin-t 2y ago
  I am surprised by how quickly I get downvoted to hell any time I try to propose that people should be compensated for their work fairly. And I get replies like the above, full of fallacies, which basically feel like trolling. So first, thank you for a sane reply. Now, it boils down to the typical issue of a current system being bad and clearly not serving its stated purpose so the choice is whether to abolish it or whether to reform it (and how). And just like any other time people have to agree on something, the simplest-to-describe solution (abolishing) will get a huge following because the more fair solutions (reform so it actually protects and rewards creators) are way, way more complex. I fundamentally think if you build on other people's work, they deserve 1) credit 2) compensation according to what percentage of your work is based on theirs. We could for example compare the performance of the same model depending on which parts of the training data you omit. If you omit all copyrighted work and the result is useless, we know the copyrighted work is responsible a large part of the value.
  - Kim_Bruning 2y ago
    > I fundamentally think if you build on other people's work, they deserve 1) credit 2) compensation according to what percentage of your work is based on theirs. That’s not a naturally occurring principle—it’s a legal and social construct that people have come to see as an inherent entitlement. If the current system isn’t working, maybe that's the actual bit that's broken and needs reform?
    - martin-t 2y ago
      What I describe isn't the current system, it's the reform I propose. The current system sadly often is you can't self-publihsh because "lol no marketing" so you sell your rights for a one-time fixed payment and the new owner profits for years or decades. The new owner is also usually a large company which has fundamentally more bargaining power than an individual so even if you get a propertional payment, the ratio of effort to reward of the company vs individual is unfair (if you put a 1000 man-days of work into your game and Steam takes 5%, do you think it's actually putting 50 man-days of work into selling it?). As for naturally occurring principles... there are some, ehm, methods of negotiating, which you could consider natural but the state you live in would generally not approve.
martin-t 2y ago
I have trouble believing this argument is in good faith. 1) (minor nitpick) I don't see how HN is in the same category as GitHub or Stack Overflow. 2) "sometimes by license" or "free by license" imply you don't understand how copyright works. Code that is not accompanied by a license is proprietary. Period. [0] And if it has a license, then you have to follow it. If the license says that derivative works have to give credit and use the same license, then LLMs using that code for training and anything generated by those models is derivative and has to respect the license. 3) Arguably i didn't say this in the OP but the idea that the publisher owns copyright is absolutely nuts and only possible through extensive lobbying and basically extortion. It should be illegal. Copyright should always belong to the person doing the actual work. Fuck rent-seekers. 4) If western ML companies thought they can produce models of the same quality without for example stealing and laundering the entirety of Github, they would have. They don't so clearly GH is a very important input and its builders should either be compensated or the models and ML companies should only use the subset that they can without breaking the license. Please don't get offended but I've seen this argument multiple times by proponents of A"I" and their entire argument hinged on the idea that the current ML models are so large that nobody can understand them and therefore a form of "intelligence" which they are clearly not (unless proven otherwise, at which point, they should get their own personhood but the fact no big ML company is arguing for that makes it obvious nobody really considers them intelligent). [0]: https://opensource.stackexchange.com/questions/1720/what-can-i-assume-if-a-publicly-published-project-has-no-license https://opensource.stackexchange.com/questions/1720/what-can...

kevingadd 2y ago

Scientific papers are available on web pages so they're publicly available for free too right? I can download an ISO or installer of almost anything if I go to the right website, so all software is already free? If you're going to ignore the existence of copyright and licenses we should extend it to everything that's ever been posted on the internet, not just "web pages". Why shouldn't all books and films count as free too? I'm actually open to the idea of just abolishing copyright but it's kind of silly to act like it's only about Elsevier. Lots of creatives depend on copyright in order to earn a living, similarly to how patents fund a lot of important research despite how noxious the patent system has become. If we fixate on examples like Elsevier or Martin Shkreli in order to argue for completely abolishing the copyright or patent systems we risk destroying the framework that enables valuable creative works or new technologies to be developed in the first place. This is part of why people are so upset by AI companies arguing that they should just be able to ignore the whole framework in order to enrich themselves; once you allow the for-profit AI companies to do it, other groups are going to line up to also demand a free ride.

Kim_Bruning 2y ago
> We risk destroying the framework that enables valuable creative works or new technologies to be developed in the first place I think we are starting to see not just historical empirical data but also strategic intelligence that this assumption might be somewhat inaccurate. - which is what Anna's archive is pointing out here in the context of AI development and copyright restrictions.
martin-t 2y ago
I am surprised by how quickly I get downvoted to hell any time I try to propose that people should be compensated for their work fairly. And I get replies like the above, full of fallacies, which basically feel like trolling. So first, thank you for a sane reply. Now, it boils down to the typical issue of a current system being bad and clearly not serving its stated purpose so the choice is whether to abolish it or whether to reform it (and how). And just like any other time people have to agree on something, the simplest-to-describe solution (abolishing) will get a huge following because the more fair solutions (reform so it actually protects and rewards creators) are way, way more complex. I fundamentally think if you build on other people's work, they deserve 1) credit 2) compensation according to what percentage of your work is based on theirs. We could for example compare the performance of the same model depending on which parts of the training data you omit. If you omit all copyrighted work and the result is useless, we know the copyrighted work is responsible a large part of the value.
- Kim_Bruning 2y ago
  > I fundamentally think if you build on other people's work, they deserve 1) credit 2) compensation according to what percentage of your work is based on theirs. That’s not a naturally occurring principle—it’s a legal and social construct that people have come to see as an inherent entitlement. If the current system isn’t working, maybe that's the actual bit that's broken and needs reform?
  - martin-t 2y ago
    What I describe isn't the current system, it's the reform I propose. The current system sadly often is you can't self-publihsh because "lol no marketing" so you sell your rights for a one-time fixed payment and the new owner profits for years or decades. The new owner is also usually a large company which has fundamentally more bargaining power than an individual so even if you get a propertional payment, the ratio of effort to reward of the company vs individual is unfair (if you put a 1000 man-days of work into your game and Steam takes 5%, do you think it's actually putting 50 man-days of work into selling it?). As for naturally occurring principles... there are some, ehm, methods of negotiating, which you could consider natural but the state you live in would generally not approve.

martin-t 2y ago

I have trouble believing this argument is in good faith. 1) (minor nitpick) I don't see how HN is in the same category as GitHub or Stack Overflow. 2) "sometimes by license" or "free by license" imply you don't understand how copyright works. Code that is not accompanied by a license is proprietary. Period. [0] And if it has a license, then you have to follow it. If the license says that derivative works have to give credit and use the same license, then LLMs using that code for training and anything generated by those models is derivative and has to respect the license. 3) Arguably i didn't say this in the OP but the idea that the publisher owns copyright is absolutely nuts and only possible through extensive lobbying and basically extortion. It should be illegal. Copyright should always belong to the person doing the actual work. Fuck rent-seekers. 4) If western ML companies thought they can produce models of the same quality without for example stealing and laundering the entirety of Github, they would have. They don't so clearly GH is a very important input and its builders should either be compensated or the models and ML companies should only use the subset that they can without breaking the license. Please don't get offended but I've seen this argument multiple times by proponents of A"I" and their entire argument hinged on the idea that the current ML models are so large that nobody can understand them and therefore a form of "intelligence" which they are clearly not (unless proven otherwise, at which point, they should get their own personhood but the fact no big ML company is arguing for that makes it obvious nobody really considers them intelligent). [0]: https://opensource.stackexchange.com/questions/1720/what-can-i-assume-if-a-publicly-published-project-has-no-license https://opensource.stackexchange.com/questions/1720/what-can...