6 ms·

>On information and belief, the reason ChatGPT can accurately summarize a certain copyrighted book is because that book was copied by OpenAI and ingested by the

by Imnimo 3y ago

>On information and belief, the reason ChatGPT can accurately summarize a certain copyrighted book is because that book was copied by OpenAI and ingested by the underlying OpenAI Language Model (either GPT-3.5 or GPT-4) as part of its training data. While it strikes me as perfectly plausible that the Books2 dataset contains Silverman's book, this quote from the complaint seems obviously false. First, even if the model never saw a single word of the book's text during training, it could still learn to summarize it from reading other summaries which are publicly available. Such as the book's Wikipedia page. Second, it's not even clear to me that a model which only saw the text of a book, but not any descriptions or summaries of it, during training would even be particular good at producing a summary. We can test this by asking for a summary of a book which is available through Project Gutenberg (which the complaint asserts is Books1 and therefore part of ChatGPT's training data) but for which there is little discussion online. If the source of the ability to summarize is having the book itself during training, the model should be equally able to summarize the rare book as it is Silverman's book. I chose "The Ruby of Kishmoor" at random. It was added to PG in 2003. ChatGPT with GPT-3.5 hallucinates a summary that doesn't even identify the correct main characters. The GPT-4 model refuses to even try, saying it doesn't know anything about the story and it isn't part of its training data. If ChatGPT's ability to summarize Silverman's book comes from the book itself being part of the training data, why can it not do the same for other books?

seanthemon 3y ago
Accessibility? I've heard of Silverman but never Ruby of Kishmoor More people discuss it, more people summarize on their personal or other sites, etc
- buildbot 3y ago
  Right that is the point of the parent comment - it’s not the book, it’s the amalgamation of all the discussions and content about the book. This case is probably dead in the water.
  - krisoft 3y ago
    > This case is probably dead in the water. Is that a fact? I’m no lawyer, but if they can get it in front of a jurry is it impossible that they will find a human author more relatable and the technical counter arguments goblydook?
  - moffkalast 3y ago
    I'm not entirely up to speed on US law, but wouldn't OpenAI have to provide the court some kind of proof that they didn't use it in the training data during discovery?
    - buildbot 3y ago
      Not a layer, but I believe the plaintiff (the author) would need to prove that it regurgitates their copyrighted work - otherwise it is possibly fair use. OpenAI does not need to prove anything, just defend their position at a reasonable level. It’s not been decided if training a model on copyrighted works is “okay” or not as far as I know, but I expect it to be so, given that literally everyone does so at this point. It’s not like imagenet is copyright free, many of the images were/are.
      - pkilgore 3y ago
        No. Fair use is an affirmative defense to infringement. If they admit to infringement or in the alternative want to argue fair use, the burden is on OpenAI to demonstrate their use falls within the relevant standard for fair use.
    - pkilgore 3y ago
      No. Burden is on the plaintiff (Silverman) to prove infringement.
ec109685 3y ago
The playground summarizes it as this via GPT-4: Prompt: Please summarize the following book found on Project Gutenberg The Ruby of Kishmoor Response: "The Ruby of Kishmoor" is a short adventure story written by Howard Pyle. The narrative revolves around the life of Jonathan Rugg, a young man who is enticed by a mysterious stranger to come to the Caribbean to secure a valuable relic, the Ruby of Kishmoor. Once Jonathan reaches the Caribbean, he learns of the ruby's grave curse. Despite this, his innate curiosity and the prospect of considerable wealth drive him to risk his life and go after the highly prized gem. After facing a slew of challenges and trials, he surprisingly stumbles upon the ruby through sheer luck rather than organized search efforts. In the course of his journey, Jonathan delves into the depths of human greed and the lengths to which individuals go for materialistic gains. The story encapsulates an amalgamation of high-tide adventures, brushes with supernatural elements, noteworthy instances of bravery, and deep moral reflections, demanding readers to rethink about the real value of materialistic pursuits. Jonathan finally manages to escape with the ruby but at great personal cost. The story ends on a sobering note, making readers wonder if the adventures and trials were worth the prize, thus engaging them in a discourse about the true value of life and material pursuits.
- CydeWeys 3y ago
  Right, but that's useless without knowing how much (if any!) of it is actually correct. Is this completely hallucinated garbage?
  - pmontra 3y ago
    How is it different from asking to me to summarize anything? I could have bought the book, or read the Wikipedia page, or listened people talking about it, or downloaded the torrent. In all those cases my summary could be right or could be wrong. If the rights holders know that I dowloaded the torrent they could sue me. In the other cases they can't. What if it turns out that OpenAI bought a copy of every book ingested be ChatGPT?
    - CydeWeys 3y ago
      > In all those cases my summary could be right or could be wrong. Well that's incredibly nihilistic. Whether the summary is correct or not matters a great deal! And if someone I knew said they read a book, even a very obscure one, and then summarized it to me, I'd have great confidence that they would get such simple facts as "who are the characters" and "what are the major plot points" correct. But ChatGPT? Who the hell knows? You can't trust a thing it says, especially about obscure topics. The summary is useless if you have to do a bunch of verification to see if any of it is even true, a problem that summaries even by moderately competent human writers don't have!
monocasa 3y ago
I mean, that's the way you state facts that your suit is based on in order to start setting the bounds of discovery. They're asserting that they have reason to believe it's true, and now with a suit, they can look for themselves to be sure.
jmull 3y ago
> this quote from the complaint seems obviously false I notice you go on to provide an argument only for why it might not be true. Also, seeing the other post on this, I asked chatgpt-4 for a summary of “ The Ruby of Kishmoor” as well, and it provided one to me, though I had to ask twice. I don’t know anything about that book, so I can’t tell if its summary is accurate, but so much for your test. It seems pretty naive to me to just kind of assume chatgpt must be respecting copyright, and hasn’t scanned copyrighted material without obtaining authorization. Perhaps discovery will settle it, though. Logs of what they scanned should exist. (IMO, a better argument is that this is fair use.)
- NoZebra120vClip 3y ago
  > IMO, a better argument is that this is fair use There is no way in Hell that this is fair use! Fair use defenses rest on the fact that a limited excerpt was used for limited distribution, among other criteria. For example, if I'm a teacher and I make 30 copies of one page of a 300-page novel and I hand that out to my students, that's a brief excerpt for a fairly limited distribution. Now if I'm a social media influencer and I copy all 300 pages of a 300-page book and then I send it out to all 3,000 of my followers, that's not fair use! Also if I'm a teacher, and I find a one-page infographic and I make 30 copies of that, that's not fair use, because I didn't make an excerpt but I've copied 100% of the original work. That's infringement now. So if LLMs went through en masse in thousands of copyrighted works in their entirety and ingested every byte of them, no copyright judge on the planet would call that fair use. For reference, the English Wikipedia has a policy that allows some fair-use content of copyrighted works: https://en.wikipedia.org/wiki/Wikipedia:Non-free_content_criteria https://en.wikipedia.org/wiki/Wikipedia:Non-free_content_cri...
  - NLPaep 3y ago
    You are assuming the LLM spits out exact copies of everything they’ve read.
    - throwaway290 3y ago
      It doesn't matter what goes out, it matters what goes in because that's what's required for LLMs to work
pkilgore 3y ago
Plausible is literally the standard to clear a motion to dismiss. Plausible gets you discovery. Discovery gets you closer to the what the actual facts are.
vmh1928 3y ago
Isn't part of the problem that some of the training data is retained by the model and used during response generation? In that case it's not just that the copyrighted book was used as training data but that some part of the book has been retained by the model. So now my model is using copyrighted material while it runs. Here's an example of a model that retained enough image data to reconstruct a reasonable facsimile of the training image. https://www.theregister.com/2023/02/06/uh_oh_attackers_can_extract/ https://www.theregister.com/2023/02/06/uh_oh_attackers_can_e...

by Imnimo 3y ago

seanthemon 3y ago

Accessibility? I've heard of Silverman but never Ruby of Kishmoor More people discuss it, more people summarize on their personal or other sites, etc

buildbot 3y ago
Right that is the point of the parent comment - it’s not the book, it’s the amalgamation of all the discussions and content about the book. This case is probably dead in the water.
- krisoft 3y ago
  > This case is probably dead in the water. Is that a fact? I’m no lawyer, but if they can get it in front of a jurry is it impossible that they will find a human author more relatable and the technical counter arguments goblydook?
- moffkalast 3y ago
  I'm not entirely up to speed on US law, but wouldn't OpenAI have to provide the court some kind of proof that they didn't use it in the training data during discovery?
  - buildbot 3y ago
    Not a layer, but I believe the plaintiff (the author) would need to prove that it regurgitates their copyrighted work - otherwise it is possibly fair use. OpenAI does not need to prove anything, just defend their position at a reasonable level. It’s not been decided if training a model on copyrighted works is “okay” or not as far as I know, but I expect it to be so, given that literally everyone does so at this point. It’s not like imagenet is copyright free, many of the images were/are.
    - pkilgore 3y ago
      No. Fair use is an affirmative defense to infringement. If they admit to infringement or in the alternative want to argue fair use, the burden is on OpenAI to demonstrate their use falls within the relevant standard for fair use.
  - pkilgore 3y ago
    No. Burden is on the plaintiff (Silverman) to prove infringement.

ec109685 3y ago

The playground summarizes it as this via GPT-4: Prompt: Please summarize the following book found on Project Gutenberg The Ruby of Kishmoor Response: "The Ruby of Kishmoor" is a short adventure story written by Howard Pyle. The narrative revolves around the life of Jonathan Rugg, a young man who is enticed by a mysterious stranger to come to the Caribbean to secure a valuable relic, the Ruby of Kishmoor. Once Jonathan reaches the Caribbean, he learns of the ruby's grave curse. Despite this, his innate curiosity and the prospect of considerable wealth drive him to risk his life and go after the highly prized gem. After facing a slew of challenges and trials, he surprisingly stumbles upon the ruby through sheer luck rather than organized search efforts. In the course of his journey, Jonathan delves into the depths of human greed and the lengths to which individuals go for materialistic gains. The story encapsulates an amalgamation of high-tide adventures, brushes with supernatural elements, noteworthy instances of bravery, and deep moral reflections, demanding readers to rethink about the real value of materialistic pursuits. Jonathan finally manages to escape with the ruby but at great personal cost. The story ends on a sobering note, making readers wonder if the adventures and trials were worth the prize, thus engaging them in a discourse about the true value of life and material pursuits.

CydeWeys 3y ago
Right, but that's useless without knowing how much (if any!) of it is actually correct. Is this completely hallucinated garbage?
- pmontra 3y ago
  How is it different from asking to me to summarize anything? I could have bought the book, or read the Wikipedia page, or listened people talking about it, or downloaded the torrent. In all those cases my summary could be right or could be wrong. If the rights holders know that I dowloaded the torrent they could sue me. In the other cases they can't. What if it turns out that OpenAI bought a copy of every book ingested be ChatGPT?
  - CydeWeys 3y ago
    > In all those cases my summary could be right or could be wrong. Well that's incredibly nihilistic. Whether the summary is correct or not matters a great deal! And if someone I knew said they read a book, even a very obscure one, and then summarized it to me, I'd have great confidence that they would get such simple facts as "who are the characters" and "what are the major plot points" correct. But ChatGPT? Who the hell knows? You can't trust a thing it says, especially about obscure topics. The summary is useless if you have to do a bunch of verification to see if any of it is even true, a problem that summaries even by moderately competent human writers don't have!

monocasa 3y ago

I mean, that's the way you state facts that your suit is based on in order to start setting the bounds of discovery. They're asserting that they have reason to believe it's true, and now with a suit, they can look for themselves to be sure.

jmull 3y ago

> this quote from the complaint seems obviously false I notice you go on to provide an argument only for why it might not be true. Also, seeing the other post on this, I asked chatgpt-4 for a summary of “ The Ruby of Kishmoor” as well, and it provided one to me, though I had to ask twice. I don’t know anything about that book, so I can’t tell if its summary is accurate, but so much for your test. It seems pretty naive to me to just kind of assume chatgpt must be respecting copyright, and hasn’t scanned copyrighted material without obtaining authorization. Perhaps discovery will settle it, though. Logs of what they scanned should exist. (IMO, a better argument is that this is fair use.)

NoZebra120vClip 3y ago
> IMO, a better argument is that this is fair use There is no way in Hell that this is fair use! Fair use defenses rest on the fact that a limited excerpt was used for limited distribution, among other criteria. For example, if I'm a teacher and I make 30 copies of one page of a 300-page novel and I hand that out to my students, that's a brief excerpt for a fairly limited distribution. Now if I'm a social media influencer and I copy all 300 pages of a 300-page book and then I send it out to all 3,000 of my followers, that's not fair use! Also if I'm a teacher, and I find a one-page infographic and I make 30 copies of that, that's not fair use, because I didn't make an excerpt but I've copied 100% of the original work. That's infringement now. So if LLMs went through en masse in thousands of copyrighted works in their entirety and ingested every byte of them, no copyright judge on the planet would call that fair use. For reference, the English Wikipedia has a policy that allows some fair-use content of copyrighted works: https://en.wikipedia.org/wiki/Wikipedia:Non-free_content_criteria https://en.wikipedia.org/wiki/Wikipedia:Non-free_content_cri...
- NLPaep 3y ago
  You are assuming the LLM spits out exact copies of everything they’ve read.
  - throwaway290 3y ago
    It doesn't matter what goes out, it matters what goes in because that's what's required for LLMs to work

pkilgore 3y ago

Plausible is literally the standard to clear a motion to dismiss. Plausible gets you discovery. Discovery gets you closer to the what the actual facts are.

vmh1928 3y ago

Isn't part of the problem that some of the training data is retained by the model and used during response generation? In that case it's not just that the copyrighted book was used as training data but that some part of the book has been retained by the model. So now my model is using copyrighted material while it runs. Here's an example of a model that retained enough image data to reconstruct a reasonable facsimile of the training image. https://www.theregister.com/2023/02/06/uh_oh_attackers_can_extract/ https://www.theregister.com/2023/02/06/uh_oh_attackers_can_e...