3 ms·

> A more correct analogy is if you bought Vonnegut’s books But they didn't do that > Or checked them out from the library. But they didn't do that either.

DonsDiscountGas 3y ago
This is really a nitpick. If I can write books in the style of Vonnegut and he loses his entire income (who knows how many millions of dollars), it doesn't matter if I first bought some of his books for $100 or just stole them. For AI, they pulled massive amounts of data off the internet. For any copyrighted material, I would consider this fair use, seeing as they used trillions of words of training, any individual work is a tiny fraction. Plus if you ask ChatGPT to give you the text of Cats Cradle, it won't do it, so there is no direct copying happening. Maybe the analogy is walking into a bookstore, reading his books, and then leaving without paying. But it doesn't really matter, because they can retrain without any of that without affecting the model quality all the much, the real magic is transformers + RLHF.
- ElevenLathe 3y ago
  Also Kurt Vonnegut is dead so we don't have to worry about his income anymore.
prepend 3y ago
What makes you think they didn’t buy a copy of Vonnegut’s books? I suspect they definitely did as this is just an analogy we invented in this thread. And in real life, I’m guessing that people training LLMs are definitely buying ebooks and not pirating them.
- iudqnolq 3y ago
  > A.I. companies have acknowledged in research papers that they rely on shadow libraries. OpenAI's GPT-1 was trained on BookCorpus, which has over 7,000 unpublished titles scraped from the self-publishing platform Smashwords. To train GPT-3, OpenAI said that about 16 percent of the data it used came from two "internet-based books corpora" that it called "Books1" and "Books2." According to a lawsuit by the comedian Sarah Silverman and two other authors against OpenAI, Books2 is most likely a "flagrantly illegal" shadow library. https://www.nytimes.com/2023/07/22/business/dealbook/extreme-heat-climate-investment.html https://www.nytimes.com/2023/07/22/business/dealbook/extreme...

DonsDiscountGas 3y ago

This is really a nitpick. If I can write books in the style of Vonnegut and he loses his entire income (who knows how many millions of dollars), it doesn't matter if I first bought some of his books for $100 or just stole them. For AI, they pulled massive amounts of data off the internet. For any copyrighted material, I would consider this fair use, seeing as they used trillions of words of training, any individual work is a tiny fraction. Plus if you ask ChatGPT to give you the text of Cats Cradle, it won't do it, so there is no direct copying happening. Maybe the analogy is walking into a bookstore, reading his books, and then leaving without paying. But it doesn't really matter, because they can retrain without any of that without affecting the model quality all the much, the real magic is transformers + RLHF.

ElevenLathe 3y ago
Also Kurt Vonnegut is dead so we don't have to worry about his income anymore.

prepend 3y ago

What makes you think they didn’t buy a copy of Vonnegut’s books? I suspect they definitely did as this is just an analogy we invented in this thread. And in real life, I’m guessing that people training LLMs are definitely buying ebooks and not pirating them.

iudqnolq 3y ago
> A.I. companies have acknowledged in research papers that they rely on shadow libraries. OpenAI's GPT-1 was trained on BookCorpus, which has over 7,000 unpublished titles scraped from the self-publishing platform Smashwords. To train GPT-3, OpenAI said that about 16 percent of the data it used came from two "internet-based books corpora" that it called "Books1" and "Books2." According to a lawsuit by the comedian Sarah Silverman and two other authors against OpenAI, Books2 is most likely a "flagrantly illegal" shadow library. https://www.nytimes.com/2023/07/22/business/dealbook/extreme-heat-climate-investment.html https://www.nytimes.com/2023/07/22/business/dealbook/extreme...