3 ms·

Summary of their arguments for why training on copyrighted datasets is OK: (1) Training AI systems on large copyrighted datasets constitutes fair use under cur

by Palmik 3y ago

Summary of their arguments for why training on copyrighted datasets is OK: (1) Training AI systems on large copyrighted datasets constitutes fair use under current copyright law. They believe it is highly transformative, does not harm the market for the original works, and serves a different purpose than the original works. (2) If it were an infringement, it would severely hinder AI progress, which has many societal benefits. Copyright law should promote creativity and innovation. (3) Other legal tools like infringement suits on AI outputs, anti-scraping laws, etc can address valid concerns instead of rigid copyright laws. Expanding on (1), they say: (1.a) The corpora used for training are not made publicly accessible - they are only consumed by machines during training. So authors do not lose any potential audience. (1.b) The outputs of AI systems do not regenerate significant portions of the original works verbatim. The systems learn general patterns, not the specific expression of any particular work. (1.c) Well-constructed AI systems do not overfit on any particular work. Each work contributes very minimally to the overall model. --- I think we should not be stopping progress, but this document 100% minimizes the impact/harm. Soon AI writers and programmers will be so good, that even if they do not reproduce copyright content verbatim, they will be able to replace large % of the future work of the people that produced the copyrighted content the AI was trained on. This seems like a clear harm. Given that the datasets are the product of the whole population, the whole population should be able to reap the benefits. As things are heading, only the largest corporations will be able to squeeze profit from the most powerful models. This goes against (1.a), if one company can use the data for training, it should be made easily accessible to everyone. Side note for (1.b), many companies implement this as a dumb filter on top of the LLM output, making sure that no N character substring appears in some copyrighted text.

judge2020 3y ago
> Soon AI writers and programmers will be so good, that even if they do not reproduce copyright content verbatim, they will be able to replace large % of the future work of the people that produced the copyrighted content the AI was trained on. This seems like a clear harm. Note that "harm the market for the original works" is only a part of Fair Use and harming the original work does not necessarily make your use of it 'not fair use'. Movie critics using movie clips can absolutely destroy the market for the original work while still being fair use criticism. Now whatever the USPTO says/does is important for now, but it's a backdrop to the legal challenges that will arise. Because the USPTO has to adhere to new laws and must follow whatever the courts rule regarding how copyright is defined, if the USPTO tries to say "ai is not fair use" then the courts can disagree with them and issue a Court Order to recognize certain copyrights, whenever someone challenges the USPTO's actions.
- Palmik 3y ago
  Movie critics harm the original work / producers of the work in different way: they dissuade the consumption of the work, rather than providing a 1:1 replacement of it.
  - judge2020 3y ago
    I'd argue that a lot of people love to watch a movie only to find its flaws, and CinemaSins might remove their need to go buy/watch the film legally. Although that's probably 1/100th of the market for a movie, if that.
deeviant 3y ago
> Soon AI writers and programmers will be so good, that even if they do not reproduce copyright content verbatim, they will be able to replace large % of the future work of the people that produced the copyrighted content the AI was trained on. This seems like a clear harm. That is not the harm that copyright is meant to protect. It's not there to keep lamp lighters employed.
- Palmik 3y ago
  I agree we should not try to preserve jobs that are no longer needed. Though I don't think there's any copyright involved in your analogy :) I just think we should be clear about the potential short-to-medium term effects / harm. And given how transformative this technology can be, we may need to come up with solutions that don't fit into existing frameworks -- e.g. making sure that the datasets and models are open access.
yieldcrv 3y ago
“clear harm” meaning what role of the government involvement, to you? as it stands I’m not sure that observation or rebuttal has any bearing on anything aside from “oh well”
- Palmik 3y ago
  Well, I am definitely against the kind of involvement these large companies, that are set to benefit the most, are argumenting for, which is limiting / gate-keeping access. Rather, I could imagine the opposite -- some sort of guarantee of access to everyone, at fair prices. There are different ways to do this.
  - yieldcrv 3y ago
    okay, yeah definitely watch out for that I see the same arguments applied to already free, open source, copyleft licensed models
bloppe 3y ago
I cannot abide the argument that obviating work is a harm. That's luddism, which has proven every single time to be more harmful in the long term than embracing new technology, which may have harms but only in the short term. I also think you're overestimating the short term harms. I believe human writing will stay valuable, especially the premium creative variety. It's the un-creative, mechanical writing that's becoming somewhat obsolete.
- Palmik 3y ago
  I think we should not try to forcefully preserve jobs that are no longer needed, AND I think it could cause harm in the short term, and lead to further transfers of wealth.
  - Avicebron 3y ago
    I'm curious as to your solution to providing comparable (better?) options for those people who have been displaced? Agreed this will cause further transfers of wealth, but with the caveat that this will probably be upwards (it always seems to be) and in denser concentrations leading to long term harm.
    - jazzyjackson 3y ago
      (not the parent poster): Treat data like oil, a public resource, and redistribute the profits to everyone equally. I don't love the idea of taxing profits and letting the government send checks out, so being creative, how about we force dilution of the stockholders of whoever wins the AI wars and make every citizen a beneficiary of dividends.
      - Avicebron 3y ago
        I see no problem with this as long as the dividends are equal to the potential of anyone displaced or otherwise having their marketshare impacted. EDIT: otherwise, if I lose a 120k+ (or whatever) job a year programming because one engineer and a cadre of llms is 'good enough' for management, and all I get is $15 a month for my portion of the collective dole, it's not really a solution.
jazzyjackson 3y ago
> dumb filter on top of the LLM output Just to provide an example here, I know the Dune quote was a popular example of a request that crashes the session, but in my case I wasn't even trying to get GPT4 to trip, I just wanted to know what a song was about, so I asked "what is ballroom blitz about", and it described the song accurately but went on to quote the lyrics - it interrupts itself and starts over mid lyrics, before trying again and quoting the lyrics inside a code block instead, which also gets cut off. Very clumsy implementation, you'd think they could be fine tuned to not quote source material (that is, unless they want to offer a version that does quote source material to a different market segment without bothering to fine-tune again) https://poe.com/s/eW2tp487r1pa1RiVyDbH https://poe.com/s/eW2tp487r1pa1RiVyDbH
rvz 3y ago
> (2) If it were an infringement, it would severely hinder AI progress, which has many societal benefits. This is one of the most horrific and weakest arguments supporting the unlicensed training of images / data for commercial purposes as 'fair use' from OpenAI I have ever seen. There is more to AI than LLMs and nothing is being hindered. It is just them inventing excuses in avoiding to pay for the data they are using for training. The fact that OpenAI obtained permission from Shutterstock for its training data for DALL-E 2 suggests that they knew that Shutterstock would sue them if they didn't and commercialized it, just like how Getty unsurprisingly caught Stable Diffusion generated images containing their watermark. Either use public domain images for free, or just pay for the sources for your training data if you're commercializing it. OpenAI already did that once already and they have plenty of money to do it again. No excuses.

by Palmik 3y ago

judge2020 3y ago

> Soon AI writers and programmers will be so good, that even if they do not reproduce copyright content verbatim, they will be able to replace large % of the future work of the people that produced the copyrighted content the AI was trained on. This seems like a clear harm. Note that "harm the market for the original works" is only a part of Fair Use and harming the original work does not necessarily make your use of it 'not fair use'. Movie critics using movie clips can absolutely destroy the market for the original work while still being fair use criticism. Now whatever the USPTO says/does is important for now, but it's a backdrop to the legal challenges that will arise. Because the USPTO has to adhere to new laws and must follow whatever the courts rule regarding how copyright is defined, if the USPTO tries to say "ai is not fair use" then the courts can disagree with them and issue a Court Order to recognize certain copyrights, whenever someone challenges the USPTO's actions.

Palmik 3y ago
Movie critics harm the original work / producers of the work in different way: they dissuade the consumption of the work, rather than providing a 1:1 replacement of it.
- judge2020 3y ago
  I'd argue that a lot of people love to watch a movie only to find its flaws, and CinemaSins might remove their need to go buy/watch the film legally. Although that's probably 1/100th of the market for a movie, if that.

deeviant 3y ago

Palmik 3y ago
I agree we should not try to preserve jobs that are no longer needed. Though I don't think there's any copyright involved in your analogy :) I just think we should be clear about the potential short-to-medium term effects / harm. And given how transformative this technology can be, we may need to come up with solutions that don't fit into existing frameworks -- e.g. making sure that the datasets and models are open access.

yieldcrv 3y ago

“clear harm” meaning what role of the government involvement, to you? as it stands I’m not sure that observation or rebuttal has any bearing on anything aside from “oh well”

Palmik 3y ago
Well, I am definitely against the kind of involvement these large companies, that are set to benefit the most, are argumenting for, which is limiting / gate-keeping access. Rather, I could imagine the opposite -- some sort of guarantee of access to everyone, at fair prices. There are different ways to do this.
- yieldcrv 3y ago
  okay, yeah definitely watch out for that I see the same arguments applied to already free, open source, copyleft licensed models

bloppe 3y ago

I cannot abide the argument that obviating work is a harm. That's luddism, which has proven every single time to be more harmful in the long term than embracing new technology, which may have harms but only in the short term. I also think you're overestimating the short term harms. I believe human writing will stay valuable, especially the premium creative variety. It's the un-creative, mechanical writing that's becoming somewhat obsolete.

Palmik 3y ago
I think we should not try to forcefully preserve jobs that are no longer needed, AND I think it could cause harm in the short term, and lead to further transfers of wealth.
- Avicebron 3y ago
  I'm curious as to your solution to providing comparable (better?) options for those people who have been displaced? Agreed this will cause further transfers of wealth, but with the caveat that this will probably be upwards (it always seems to be) and in denser concentrations leading to long term harm.
  - jazzyjackson 3y ago
    (not the parent poster): Treat data like oil, a public resource, and redistribute the profits to everyone equally. I don't love the idea of taxing profits and letting the government send checks out, so being creative, how about we force dilution of the stockholders of whoever wins the AI wars and make every citizen a beneficiary of dividends.
    - Avicebron 3y ago
      I see no problem with this as long as the dividends are equal to the potential of anyone displaced or otherwise having their marketshare impacted. EDIT: otherwise, if I lose a 120k+ (or whatever) job a year programming because one engineer and a cadre of llms is 'good enough' for management, and all I get is $15 a month for my portion of the collective dole, it's not really a solution.

jazzyjackson 3y ago

> dumb filter on top of the LLM output Just to provide an example here, I know the Dune quote was a popular example of a request that crashes the session, but in my case I wasn't even trying to get GPT4 to trip, I just wanted to know what a song was about, so I asked "what is ballroom blitz about", and it described the song accurately but went on to quote the lyrics - it interrupts itself and starts over mid lyrics, before trying again and quoting the lyrics inside a code block instead, which also gets cut off. Very clumsy implementation, you'd think they could be fine tuned to not quote source material (that is, unless they want to offer a version that does quote source material to a different market segment without bothering to fine-tune again) https://poe.com/s/eW2tp487r1pa1RiVyDbH https://poe.com/s/eW2tp487r1pa1RiVyDbH

rvz 3y ago

> (2) If it were an infringement, it would severely hinder AI progress, which has many societal benefits. This is one of the most horrific and weakest arguments supporting the unlicensed training of images / data for commercial purposes as 'fair use' from OpenAI I have ever seen. There is more to AI than LLMs and nothing is being hindered. It is just them inventing excuses in avoiding to pay for the data they are using for training. The fact that OpenAI obtained permission from Shutterstock for its training data for DALL-E 2 suggests that they knew that Shutterstock would sue them if they didn't and commercialized it, just like how Getty unsurprisingly caught Stable Diffusion generated images containing their watermark. Either use public domain images for free, or just pay for the sources for your training data if you're commercializing it. OpenAI already did that once already and they have plenty of money to do it again. No excuses.