4 ms·

> There is lots of out of copyright content available OpenAI's quote, from which the headline is derived, briefly mentions public domain content: > > Because

by Ukv 3y ago

> There is lots of out of copyright content available OpenAI's quote, from which the headline is derived, briefly mentions public domain content: > > Because copyright today covers virtually every sort of human expression including blog posts, photographs, forum posts, scraps of software code, and government documents – it would be impossible to train today’s leading AI models without using copyrighted materials. Limiting training data to public domain books and drawings created more than a century ago might yield an interesting experiment, but would not provide AI systems that meet the needs of today’s citizens. I do think they're right in that you're not going to get remotely the same performance as modern LLMs if you train with solely public domain content. Licensing a sufficient amount of content could be feasible for OpenAI/Meta, but hopefully we get something that still works for smaller companies, FOSS groups, etc.

NewJazz 3y ago
So OpenAI license the content they train on... They just admitted it has value.
Terr_ 3y ago
Their quote sounds a bit like: "Paying for what I want is too much of a hassle so I should get it for free." Even being in favor of massive copyright reform, I don't think I can get behind that argument.
- persolb 3y ago
  I think this is an edge case where intuition breaks down. I'm not sure if my intuition is broke or yours. My intuition is that, if you publish something online and let me read it, I can use that knowledge. I can NOT flat out duplicate it for others, but I can still use the knowledge. Training OpenAI is akin to me using something I read. (Hell, most of what I know I obtained this way.) To me, the only issue is if the model is reproducing text verbatim en masse. I think my intuition above is basically where my intuition lands for search engines too. The can use datas found on the web, and can show me snippets, but I'd draw a line at them showing me all/most of an article.
  - johnnyanmac 3y ago
    We go straight back to the arguments of last decade's scraping legalities. Think of my intuition as an all you can eat buffet. In spirit, you can only eat so much as one person and you may bring home some scraps of food you already touched. In theory (if we ignore fine print), you can very much eat out half the buffet, and then take the other half to go. But this latter "approach" breaks down the model of a buffet. Tragedy of the commons. Everyone is a glutton, the buffet can't keep up with making food, business shuts down, no more food for anyone. The internet here is a buffet. Just because you have access to terabytes, petabytes or more of information without paying doesn't mean you can download all those terabytes of info. Everyone starts doing that and the model of the commons that are ad-supported servers breaks down. Sites start to close up behind more anti-copy protection or literal paywalls to make things not-so-freely accessible. And that ends the free internet as we know it. We are indeed starting to see pieces of this happen in real time. >To me, the only issue is if the model is reproducing text verbatim en masse. That's the other issue as well that was discovered with that exploit. You can reguritate the source with enough prompting. This seems to imply that some server out there is in fact just storing hoards of source material on itself. If that server is commercialized and has TBs of copyright material, that seems to go past the fair use argument of "limited use".
    - Ukv 3y ago
      > Sites start to close up behind more anti-copy protection or literal paywalls to make things not-so-freely accessible. And that ends the free internet as we know it. We are indeed starting to see pieces of this happen in real time Sites have been pushing towards "to continue reading, download our app" or "you've reached your free article limit, subscribe for just $X" for a long time, because it's profitable. I don't think there's sense in retroactively placing the blame for this behavior on crawlers - it's a tactic designed for humans, to the extent that it's often implemented in ways entirely ineffective against bots such as putting an element in front of the content, using cookies to track viewed article count, or even intentionally letting bot user-agents through the paywall. Sites are capable of restricting GPTBot/CCBot/etc., or putting a rate limit on crawling, without impacting human readers. > You can reguritate the source with enough prompting. This seems to imply that some server out there is in fact just storing hoards of source material on itself. If that server is commercialized and has TBs of copyright material, that seems to go past the fair use argument of "limited use". The model isn't retrieving content from some storage server during inference if that's what you're thinking, but consider Google Books which does actually do that - storing a huge number of in-copyright books internally on Google's servers and retrieving snippets when searched. As found in Authors Guild v. Google, this is Fair Use - in particular: > > What matters in such cases is not so much "the amount and substantiality of the portion used" in making a copy, but rather the amount and substantiality of what is thereby made accessible to a public for which it may serve as a competing substitute.
protocolture 3y ago
A chatbot with the prejudices and knowledge of 1930s pulp authors would be fun, but likely the same fairweather copyright supporters would likely be also be upset about learning about "degenerate races" and phrenology from a chatbot.

by Ukv 3y ago

NewJazz 3y ago

So OpenAI license the content they train on... They just admitted it has value.

Terr_ 3y ago

Their quote sounds a bit like: "Paying for what I want is too much of a hassle so I should get it for free." Even being in favor of massive copyright reform, I don't think I can get behind that argument.

persolb 3y ago
I think this is an edge case where intuition breaks down. I'm not sure if my intuition is broke or yours. My intuition is that, if you publish something online and let me read it, I can use that knowledge. I can NOT flat out duplicate it for others, but I can still use the knowledge. Training OpenAI is akin to me using something I read. (Hell, most of what I know I obtained this way.) To me, the only issue is if the model is reproducing text verbatim en masse. I think my intuition above is basically where my intuition lands for search engines too. The can use datas found on the web, and can show me snippets, but I'd draw a line at them showing me all/most of an article.
- johnnyanmac 3y ago
  We go straight back to the arguments of last decade's scraping legalities. Think of my intuition as an all you can eat buffet. In spirit, you can only eat so much as one person and you may bring home some scraps of food you already touched. In theory (if we ignore fine print), you can very much eat out half the buffet, and then take the other half to go. But this latter "approach" breaks down the model of a buffet. Tragedy of the commons. Everyone is a glutton, the buffet can't keep up with making food, business shuts down, no more food for anyone. The internet here is a buffet. Just because you have access to terabytes, petabytes or more of information without paying doesn't mean you can download all those terabytes of info. Everyone starts doing that and the model of the commons that are ad-supported servers breaks down. Sites start to close up behind more anti-copy protection or literal paywalls to make things not-so-freely accessible. And that ends the free internet as we know it. We are indeed starting to see pieces of this happen in real time. >To me, the only issue is if the model is reproducing text verbatim en masse. That's the other issue as well that was discovered with that exploit. You can reguritate the source with enough prompting. This seems to imply that some server out there is in fact just storing hoards of source material on itself. If that server is commercialized and has TBs of copyright material, that seems to go past the fair use argument of "limited use".
  - Ukv 3y ago
    > Sites start to close up behind more anti-copy protection or literal paywalls to make things not-so-freely accessible. And that ends the free internet as we know it. We are indeed starting to see pieces of this happen in real time Sites have been pushing towards "to continue reading, download our app" or "you've reached your free article limit, subscribe for just $X" for a long time, because it's profitable. I don't think there's sense in retroactively placing the blame for this behavior on crawlers - it's a tactic designed for humans, to the extent that it's often implemented in ways entirely ineffective against bots such as putting an element in front of the content, using cookies to track viewed article count, or even intentionally letting bot user-agents through the paywall. Sites are capable of restricting GPTBot/CCBot/etc., or putting a rate limit on crawling, without impacting human readers. > You can reguritate the source with enough prompting. This seems to imply that some server out there is in fact just storing hoards of source material on itself. If that server is commercialized and has TBs of copyright material, that seems to go past the fair use argument of "limited use". The model isn't retrieving content from some storage server during inference if that's what you're thinking, but consider Google Books which does actually do that - storing a huge number of in-copyright books internally on Google's servers and retrieving snippets when searched. As found in Authors Guild v. Google, this is Fair Use - in particular: > > What matters in such cases is not so much "the amount and substantiality of the portion used" in making a copy, but rather the amount and substantiality of what is thereby made accessible to a public for which it may serve as a competing substitute.

protocolture 3y ago

A chatbot with the prejudices and knowledge of 1930s pulp authors would be fun, but likely the same fairweather copyright supporters would likely be also be upset about learning about "degenerate races" and phrenology from a chatbot.