7 ms·

I think we should not stop innovation, but let's not minimize the impact on the producers of the content (the whole of humanity). First of all, even if the mod

by Palmik 3y ago

I think we should not stop innovation, but let's not minimize the impact on the producers of the content (the whole of humanity). First of all, even if the models never reproduce any of the copyrighted content verbatim, they will soon be good enough that they will be able to replace the work of the people [1] that produced the content the AI was trained on, like writers and programmers. Second of all, the nature of the use of the copyrighted content is purely for-profit, and the ability to squeeze the profit out of the best models will stay with the largest corporations -- effectively transferring wealth from the people that produced the content, to the corporations. We need to find way to make sure the access to this data the models were trained on, and the resulting models, is open and fair. Who wrote which paper [2], or who supplied the initial GPU training, should not really matter that much in the grand scheme of things. [1] or at least cause downward pressure on their earning potential [2] you could use similar arguments here they use for using the copyrighted content -- each paper (except maybe the Attention is all you need paper) contributed only marginally

skerit 3y ago
> We need to find way to make sure the access to this data the models were trained on, and the resulting models, is open and fair Agreed. The irony is that all the current anti-machine-learning lawsuits will make it so only big companies with deep pockets will be able to continue to make & use them.
Pet_Ant 3y ago
I mean we never seem to care about the downward pressures we put on others, so this seems a contrived place to do it now. My take on the matter is that facts aren't copyrightable. A LLM is basically recording facts. I have a background in stylometry where you basically determine authorship by recording statistics on known pieces of writing to try to find the author of another piece of writing. So just extracting facts. You could easily now generate random sentences and filter ones that pass a certain threshold. LLMs are basically this on steroids. So it's just facts and data, not copyright infringement. Otherwise, you'd have to argue that software reading the text contents of a book are copyright infringement.
- thfuran 3y ago
  >Otherwise, you'd have to argue that software reading the text contents of a book are copyright infringement. How so?
  - horsawlarway 3y ago
    Because that's all the ai model is doing. It's reading the books. How an entity (human or not) changes after reading those books is not in the purview of copyright. It has not shared them (copied). It has read them, learned from them, and changed itself in response.
    - SirMaster 3y ago
      Why is an AI model allowed to read a book without paying for access to it, but I as a human have to pay for it to read it?
      - Tadpole9181 3y ago
        That's not the argument being made, is it? If the AI is trained on pirated data, it's a completely different case than the underlying principle of AI meaning all generated content is copyright infringement.
        SirMaster 3y ago
        But I thought they were being trained on pirated data... Seems like a problem to me.
        Tadpole9181 3y ago
        They're being trained on copyrighted data that was publicly accessible. The lawsuits focus on whether or not this is copyright violation, not the legality of accessing the material (AFAICT).
sillysaurusx 3y ago
> We need to find way to make sure the access to this data the models were trained on, and the resulting models, is open and fair. Hear, hear. Unfortunately this is more or less impossible given current copyright law. Suppose you scrape libgen and turn it into training data, then you release the training data publicly. Since the vast majority of every book appears verbatim in the training data, is this sufficiently transformative? I think yes, it is, because nobody is going to read those books from the training data. When I made books3, I felt it was important to render each book into high quality text. But it turns out that when you convert Jurassic Park into a text file, there's no good way to read it anymore. Good luck trying to bookmark wherever you left off -- it's all one gigantic file. But nobody seems to agree. The Danish Rights Alliance (https://rettighedsalliancen.com/ https://rettighedsalliancen.com/) aggressively DMCA'ed anyone that hosted books3, even going so far as to DMCA The Pile from academictorrents: https://academictorrents.com/details/0d366035664fdf51cfbe9f733953ba325776e667 https://academictorrents.com/details/0d366035664fdf51cfbe9f7... with the justification that ~100 copyrighted books appear in the training data, so therefore they have the right to DMCA. Right now most of the world seems to agree, but I'm hoping that opinion will shift as the years tick by. Surely no one can believe that a plain text document poses a serious threat of economic harm to the original author. So the question is whether the original author should be allowed to deny everyone else the right to transform their work into a form that machines can read. For my part, I've been planning a books4 dataset, but this time similar to LAION: it's a script that spiders libgen torrents (https://libgen.gs/torrents/libgen/ https://libgen.gs/torrents/libgen/) and converts all the epubs into text files. That way, if LAION isn't infringing, then books4 can't be infringing either. (Of course, hosting the actual training data anywhere is pretty hard nowadays, but it should only take a few days to convert 38TB of libgen into ~2TB of plain text.) This is the only way to create an open source competitor to ChatGPT.
- Al-Khwarizmi 3y ago
  I'm actually on your side because I think copyright laws should be radically nerfed and things like books3 are greatly beneficial to society, but I wouldn't buy your argument about "no good way to read plain text files". The text file contents all the text in the book in an (at the very least) machine-readable way, it is perfectly feasible to write a program to put it in ebook form or to play it as an audiobook, and then the text file becomes desirable for laypeople to read.
skybrian 3y ago
Technically, the money comes from people who pay to use AI (customers), or alternatively to buy art from artists. To the extent they are out-competing artists, it's diverting a money flow from artists to somewhere else. But a lot of that money might be from people who would never have bought that art to begin with? In any case, the money never belonged to the artists. It never got to them. The people it actually belongs to (the customers) get to make the decision about what to buy.
- jazzyjackson 3y ago
  I think I agree but that never stopped the trade associations from coming after pirates for lost revenue. Then again, that wasn't really a winning strategy since they spent more on legal fees than they ever recovered from broke college kids and single moms.
mensetmanusman 3y ago
Just tax the use of the technology and give back to humanity in terms of less work/ubi/etc.
- jazzyjackson 3y ago
  I'm really curious about the political expediancy of this - the right wing tends to hate UBI but they also resent Big Tech and are probably not fans of algorithms taking their jobs, so maybe we can come to a compromise.
troupe 3y ago
> they will soon be good enough that they will be able to replace the work of the people that produced the content the AI was trained on Let assume this is true. It then follows that if we train LLM on all books that are out of copyright, the LLM would be able to replace the "work of the people that produced the content" that is just about to go out of copyright. Feed this body of work back into the LLM and you should end up with something just as advance as what you get by feeding in works that are under copyright. I'm not sure the assumption is true. But if it is true, using copyrighted material or only non-copyrighted material shouldn't result in a different situation.
- Certhas 3y ago
  So we need new legislation. Edit: To be more explicit, on my view both the parent and the op are right, and they don't contradict each other: We don't need new copyright legislation. We need new legal concepts. I think both people are right: There is an overwhelming need to address the situation of artists and writers ("content creators"). Copyright is not being used because it's the right tool to do so, but because it's the only even vaguely plausible one. It's also clear that it's not sufficient.
  - troupe 3y ago
    We have legal concepts that fit. Ideas cannot be copyrighted. The particular expression of an idea can be copyrighted, but an idea itself is in the public domain. The ideas you or an LLM get from reading a book are in the public domain while the book (the expression of those ideas) is not. In what way is that not sufficient?
    - pclmulqdq 3y ago
      There is no legal concept for a machine to have ideas. I agree with you that the current legal framework is clear, but I disagree as to the outcome: without new legislation, the creation of an LLM on unlicensed content seems to be the creation of a derived work without permission of the owners whose work you are deriving from, so the creation of the LLM itself is copyright infringement.
mywittyname 3y ago
I'm okay with the idea of no copyrights at all on works that are fully or partially AI derived. Of course, that introduces even more thorny arguments (what legally constitutes "AI"), but the courts can work that out. It's clear already that companies are going to be want to have their cake and eat it too. When they leverage AI to produce content, they don't want to be forced to pay licensing fees on the inputs to their models. But on the flip side, they absolutely don't want their content being used in models leveraged by other people. Not allowing copyrights on wholly or partially AI-generated work is probably the lesser evil path to take.

by Palmik 3y ago

skerit 3y ago

> We need to find way to make sure the access to this data the models were trained on, and the resulting models, is open and fair Agreed. The irony is that all the current anti-machine-learning lawsuits will make it so only big companies with deep pockets will be able to continue to make & use them.

Pet_Ant 3y ago

I mean we never seem to care about the downward pressures we put on others, so this seems a contrived place to do it now. My take on the matter is that facts aren't copyrightable. A LLM is basically recording facts. I have a background in stylometry where you basically determine authorship by recording statistics on known pieces of writing to try to find the author of another piece of writing. So just extracting facts. You could easily now generate random sentences and filter ones that pass a certain threshold. LLMs are basically this on steroids. So it's just facts and data, not copyright infringement. Otherwise, you'd have to argue that software reading the text contents of a book are copyright infringement.

thfuran 3y ago
>Otherwise, you'd have to argue that software reading the text contents of a book are copyright infringement. How so?
- horsawlarway 3y ago
  Because that's all the ai model is doing. It's reading the books. How an entity (human or not) changes after reading those books is not in the purview of copyright. It has not shared them (copied). It has read them, learned from them, and changed itself in response.
  - SirMaster 3y ago
    Why is an AI model allowed to read a book without paying for access to it, but I as a human have to pay for it to read it?
    - Tadpole9181 3y ago
      That's not the argument being made, is it? If the AI is trained on pirated data, it's a completely different case than the underlying principle of AI meaning all generated content is copyright infringement.
      - SirMaster 3y ago
        But I thought they were being trained on pirated data... Seems like a problem to me.
        Tadpole9181 3y ago
        They're being trained on copyrighted data that was publicly accessible. The lawsuits focus on whether or not this is copyright violation, not the legality of accessing the material (AFAICT).

sillysaurusx 3y ago

> We need to find way to make sure the access to this data the models were trained on, and the resulting models, is open and fair. Hear, hear. Unfortunately this is more or less impossible given current copyright law. Suppose you scrape libgen and turn it into training data, then you release the training data publicly. Since the vast majority of every book appears verbatim in the training data, is this sufficiently transformative? I think yes, it is, because nobody is going to read those books from the training data. When I made books3, I felt it was important to render each book into high quality text. But it turns out that when you convert Jurassic Park into a text file, there's no good way to read it anymore. Good luck trying to bookmark wherever you left off -- it's all one gigantic file. But nobody seems to agree. The Danish Rights Alliance (https://rettighedsalliancen.com/ https://rettighedsalliancen.com/) aggressively DMCA'ed anyone that hosted books3, even going so far as to DMCA The Pile from academictorrents: https://academictorrents.com/details/0d366035664fdf51cfbe9f733953ba325776e667 https://academictorrents.com/details/0d366035664fdf51cfbe9f7... with the justification that ~100 copyrighted books appear in the training data, so therefore they have the right to DMCA. Right now most of the world seems to agree, but I'm hoping that opinion will shift as the years tick by. Surely no one can believe that a plain text document poses a serious threat of economic harm to the original author. So the question is whether the original author should be allowed to deny everyone else the right to transform their work into a form that machines can read. For my part, I've been planning a books4 dataset, but this time similar to LAION: it's a script that spiders libgen torrents (https://libgen.gs/torrents/libgen/ https://libgen.gs/torrents/libgen/) and converts all the epubs into text files. That way, if LAION isn't infringing, then books4 can't be infringing either. (Of course, hosting the actual training data anywhere is pretty hard nowadays, but it should only take a few days to convert 38TB of libgen into ~2TB of plain text.) This is the only way to create an open source competitor to ChatGPT.

Al-Khwarizmi 3y ago
I'm actually on your side because I think copyright laws should be radically nerfed and things like books3 are greatly beneficial to society, but I wouldn't buy your argument about "no good way to read plain text files". The text file contents all the text in the book in an (at the very least) machine-readable way, it is perfectly feasible to write a program to put it in ebook form or to play it as an audiobook, and then the text file becomes desirable for laypeople to read.

skybrian 3y ago

Technically, the money comes from people who pay to use AI (customers), or alternatively to buy art from artists. To the extent they are out-competing artists, it's diverting a money flow from artists to somewhere else. But a lot of that money might be from people who would never have bought that art to begin with? In any case, the money never belonged to the artists. It never got to them. The people it actually belongs to (the customers) get to make the decision about what to buy.

jazzyjackson 3y ago
I think I agree but that never stopped the trade associations from coming after pirates for lost revenue. Then again, that wasn't really a winning strategy since they spent more on legal fees than they ever recovered from broke college kids and single moms.

mensetmanusman 3y ago

Just tax the use of the technology and give back to humanity in terms of less work/ubi/etc.

jazzyjackson 3y ago
I'm really curious about the political expediancy of this - the right wing tends to hate UBI but they also resent Big Tech and are probably not fans of algorithms taking their jobs, so maybe we can come to a compromise.

troupe 3y ago

> they will soon be good enough that they will be able to replace the work of the people that produced the content the AI was trained on Let assume this is true. It then follows that if we train LLM on all books that are out of copyright, the LLM would be able to replace the "work of the people that produced the content" that is just about to go out of copyright. Feed this body of work back into the LLM and you should end up with something just as advance as what you get by feeding in works that are under copyright. I'm not sure the assumption is true. But if it is true, using copyrighted material or only non-copyrighted material shouldn't result in a different situation.

Certhas 3y ago
So we need new legislation. Edit: To be more explicit, on my view both the parent and the op are right, and they don't contradict each other: We don't need new copyright legislation. We need new legal concepts. I think both people are right: There is an overwhelming need to address the situation of artists and writers ("content creators"). Copyright is not being used because it's the right tool to do so, but because it's the only even vaguely plausible one. It's also clear that it's not sufficient.
- troupe 3y ago
  We have legal concepts that fit. Ideas cannot be copyrighted. The particular expression of an idea can be copyrighted, but an idea itself is in the public domain. The ideas you or an LLM get from reading a book are in the public domain while the book (the expression of those ideas) is not. In what way is that not sufficient?
  - pclmulqdq 3y ago
    There is no legal concept for a machine to have ideas. I agree with you that the current legal framework is clear, but I disagree as to the outcome: without new legislation, the creation of an LLM on unlicensed content seems to be the creation of a derived work without permission of the owners whose work you are deriving from, so the creation of the LLM itself is copyright infringement.

mywittyname 3y ago

I'm okay with the idea of no copyrights at all on works that are fully or partially AI derived. Of course, that introduces even more thorny arguments (what legally constitutes "AI"), but the courts can work that out. It's clear already that companies are going to be want to have their cake and eat it too. When they leverage AI to produce content, they don't want to be forced to pay licensing fees on the inputs to their models. But on the flip side, they absolutely don't want their content being used in models leveraged by other people. Not allowing copyrights on wholly or partially AI-generated work is probably the lesser evil path to take.