4 ms·

A more correct analogy is if you bought Vonnegut’s books and learned from them. Or checked them out from the library. Vonnegut gets no money from you learning,

A more correct analogy is if you bought Vonnegut’s books and learned from them. Or checked them out from the library. Vonnegut gets no money from you learning, but nothing bad or illegal happened.

iudqnolq 3y ago
> A more correct analogy is if you bought Vonnegut’s books But they didn't do that > Or checked them out from the library. But they didn't do that either.
- DonsDiscountGas 3y ago
  This is really a nitpick. If I can write books in the style of Vonnegut and he loses his entire income (who knows how many millions of dollars), it doesn't matter if I first bought some of his books for $100 or just stole them. For AI, they pulled massive amounts of data off the internet. For any copyrighted material, I would consider this fair use, seeing as they used trillions of words of training, any individual work is a tiny fraction. Plus if you ask ChatGPT to give you the text of Cats Cradle, it won't do it, so there is no direct copying happening. Maybe the analogy is walking into a bookstore, reading his books, and then leaving without paying. But it doesn't really matter, because they can retrain without any of that without affecting the model quality all the much, the real magic is transformers + RLHF.
  - ElevenLathe 3y ago
    Also Kurt Vonnegut is dead so we don't have to worry about his income anymore.
- prepend 3y ago
  What makes you think they didn’t buy a copy of Vonnegut’s books? I suspect they definitely did as this is just an analogy we invented in this thread. And in real life, I’m guessing that people training LLMs are definitely buying ebooks and not pirating them.
  - iudqnolq 3y ago
    > A.I. companies have acknowledged in research papers that they rely on shadow libraries. OpenAI's GPT-1 was trained on BookCorpus, which has over 7,000 unpublished titles scraped from the self-publishing platform Smashwords. To train GPT-3, OpenAI said that about 16 percent of the data it used came from two "internet-based books corpora" that it called "Books1" and "Books2." According to a lawsuit by the comedian Sarah Silverman and two other authors against OpenAI, Books2 is most likely a "flagrantly illegal" shadow library. https://www.nytimes.com/2023/07/22/business/dealbook/extreme-heat-climate-investment.html https://www.nytimes.com/2023/07/22/business/dealbook/extreme...
stavros 3y ago
Does that mean that artists would be ok with LLMs if they got one book sale for each book the LLM trained with? Not to mention that the article's author called it "stealing" even though the articles in question are free to read.
- prepend 3y ago
  If “be ok” means being comfortable with not having the income to them from people paying them to write that stuff, then no, I don’t think the artist cares whether the AI pirated or paid the $500 to buy all 50 of their books (or whatever). If “be ok” means that legally they are limited because someone bought their book and chose to use it by feeding into a training algorithm, then yes. If I steal a book from your house, it doesn’t matter if I stole to read myself or to feed into an algorithm. But if I buy a book, or borrow it from a library, then it’s legal for me to have a computer scan it and feed into a training program. I just can’t distribute copies or otherwise infringe on the copyright.

A more correct analogy is if you bought Vonnegut’s books and learned from them. Or checked them out from the library. Vonnegut gets no money from you learning, but nothing bad or illegal happened.

iudqnolq 3y ago

> A more correct analogy is if you bought Vonnegut’s books But they didn't do that > Or checked them out from the library. But they didn't do that either.

DonsDiscountGas 3y ago
This is really a nitpick. If I can write books in the style of Vonnegut and he loses his entire income (who knows how many millions of dollars), it doesn't matter if I first bought some of his books for $100 or just stole them. For AI, they pulled massive amounts of data off the internet. For any copyrighted material, I would consider this fair use, seeing as they used trillions of words of training, any individual work is a tiny fraction. Plus if you ask ChatGPT to give you the text of Cats Cradle, it won't do it, so there is no direct copying happening. Maybe the analogy is walking into a bookstore, reading his books, and then leaving without paying. But it doesn't really matter, because they can retrain without any of that without affecting the model quality all the much, the real magic is transformers + RLHF.
- ElevenLathe 3y ago
  Also Kurt Vonnegut is dead so we don't have to worry about his income anymore.
prepend 3y ago
What makes you think they didn’t buy a copy of Vonnegut’s books? I suspect they definitely did as this is just an analogy we invented in this thread. And in real life, I’m guessing that people training LLMs are definitely buying ebooks and not pirating them.
- iudqnolq 3y ago
  > A.I. companies have acknowledged in research papers that they rely on shadow libraries. OpenAI's GPT-1 was trained on BookCorpus, which has over 7,000 unpublished titles scraped from the self-publishing platform Smashwords. To train GPT-3, OpenAI said that about 16 percent of the data it used came from two "internet-based books corpora" that it called "Books1" and "Books2." According to a lawsuit by the comedian Sarah Silverman and two other authors against OpenAI, Books2 is most likely a "flagrantly illegal" shadow library. https://www.nytimes.com/2023/07/22/business/dealbook/extreme-heat-climate-investment.html https://www.nytimes.com/2023/07/22/business/dealbook/extreme...

stavros 3y ago

Does that mean that artists would be ok with LLMs if they got one book sale for each book the LLM trained with? Not to mention that the article's author called it "stealing" even though the articles in question are free to read.

prepend 3y ago
If “be ok” means being comfortable with not having the income to them from people paying them to write that stuff, then no, I don’t think the artist cares whether the AI pirated or paid the $500 to buy all 50 of their books (or whatever). If “be ok” means that legally they are limited because someone bought their book and chose to use it by feeding into a training algorithm, then yes. If I steal a book from your house, it doesn’t matter if I stole to read myself or to feed into an algorithm. But if I buy a book, or borrow it from a library, then it’s legal for me to have a computer scan it and feed into a training program. I just can’t distribute copies or otherwise infringe on the copyright.