3 ms·

> So will you owe life long compensation for all the knowledge you got from books too? You're just falling into the trap of anthropomorphizing the phrase "trai

> So will you owe life long compensation for all the knowledge you got from books too? You're just falling into the trap of anthropomorphizing the phrase "training" in the context of LLMs, which is not the same things as what humans do. There is no evidence they are the same thing and there is nothing to support the notion that what an LLM does when it "trains" on a book is equivalent to a human reading it.

flir 3mo ago
Whatever "training" is, if you can't persuade the machine to spit substantially the same text back out verbatim, it's clearly not something that falls under copyright law either, because there's no copy. Yes, for some texts that's possible. But for the vast majority, it is not.
- tarnith 3mo ago
  Can you cite any information on this not being possible for the vast majority? Or is it simply that the correct prompt hasn't been written for all possible cases? I also fail to see the difference if logic/harnessing is added around a vector database that can output the complete corpus, but simply is instructed not to. It very clearly is still compressing the information into the vector weights, and then recovering that information, thus the information is encoded. Why is a vector database somehow completely different from maintaining a library of the text itself?
  - satvikpendem 3mo ago
    Information entropy. The amount of data an LLM ingests cannot be compressed to the size of the weights even at maximum theoretical compression.
    - insanitybit 3mo ago
      Is that relevant? I can use a lossy compression algorithm such that the original could never be recovered from the image I've produced, but that derived image would surely be under copyright. LLMs are obviously capable of producing "exact" phrases as well. Ask it to give you famous quotes, it can do it. Ask it to read a paper for you and cite it, it can do it.
      - flir 3mo ago
        > but that derived image would surely be under copyright. I wouldn't bet on that. https://en.wikipedia.org/wiki/Campbell%27s_Soup_Cans https://en.wikipedia.org/wiki/Campbell%27s_Soup_Cans
        insanitybit 3mo ago
        I don't know that this really challenges anything relating to compression.
        flir 3mo ago
        Ok, reductio ad absurdum. Here's a highly compressed representation of The Lord of The Rings (all three volumes): 1 Obviously, fidelity when uncompressing it is not great, but I can assure you it was lossily compressed from the original text. Is it infringing the original's copyright? I have to assume you'd agree that the answer is "no". If I had compressed it by removing the letters x y and z, I'd agree with you that my "compressed" version is infringing. So what we've got here is a spectrum with two ridiculous extremes, and a question: When has the artifact been compressed so heavily that it no longer infringes the copyright of the original? I suggest "irretrievability" is a pretty good threshold for that question. Otherwise you're into "we know it infringes our copyright. Don't ask us to prove it, we just know it, ok?" Given the sheer volume of text that an LLM gets trained on, and how small the output is, it seems obvious that 99% of it can no longer be recovered - the process is "lossy" to the point of irretrievability, and only a statistical smear is left behind. That's why I think only the copyright claims that can show infringement in court (Harpy Potter, et al.) have merit. And a court will still have to decide "how much is too much" but at least there's case law for that. (Incidentally, I compressed the Mona Lisa to a single pixel. It was #3D3526).
        insanitybit 3mo ago
        My entire point is that compression is irrelevant. A lossily compressed image can absolutely still be in breach of the original's copyright. Showing that some kind of compressed artifact may not be doesn't change that.
  - consensus1 3mo ago
    You are asking to prove a negative. But even assuming that the model is capable of returning every bit of its training data verbatim (a mathematical impossibility) that would not be enough as mere capability is insufficient here. If capability alone were the standard any library that also has a photocopier / scanner would be in violation. To prove distribution of copyrighted materials it would have to be practical and actually used in the wild by people to circumvent copyright and generate copies of those works. Again, I can't prove a negative, but that isn't the standard, and nobody has shown a practical exploit here.
    - flir 3mo ago
      There was a paper a while back where (from memory) they managed to coax 75% of the original text of some internet-popular books out of an LLM. Harry Potter, 1984, etc. That's why I said it was possible for some texts. My assumption is that multiple copies in the training data "wear a deeper groove". I believe those are infringing, and should be dealt with on a case-by-case basis. But the vast majority of text doesn't wear that groove. (Edit: Think it was this one https://arxiv.org/abs/2601.02671 https://arxiv.org/abs/2601.02671)
  - flir 3mo ago
    Nah, I can't prove a negative. But Common Crawl is 12 petabytes and is not the largest part of what these models get trained on. DeepSeek v4 Pro is, what, 865GB? That's one hell of a compression ratio, if it can do what you claim.
- ripe 3mo ago
  > spitting out verbatim text The New York Times lawsuit is resting on the point that large chunks of undigested articles can be vomited out. OpenAI tried to have the lawsuit thrown out but the courts permitted it to continue. The Times... alleged that OpenAI's ChatGPT and Microsoft's Copilot had produced near-verbatim replicas of copyrighted articles, that the chatbots generated hallucinated content falsely attributed to the Times, ... https://en.wikipedia.org/wiki/The_New_York_Times_v._Microsoft_and_OpenAI https://en.wikipedia.org/wiki/The_New_York_Times_v._Microsof...
  - nomel 3mo ago
    > spitting out verbatim text > had produced near-verbatim replicas
    - flir 3mo ago
      Don't get too hung up on the preciseness of the copy - the courts won't. I doubt that spitting out an existing article with a few adjectives changed would be considered transformative.
      - SR2Z 3mo ago
        What's almost certainly going to happen is that the courts will decide the infringing party in that situation is the one who willfully caused the violation. Since most users of LLMs are not using them to run around copyright and this copyright stuff is aggressively trained out of models or blocked whenever possible, the models themselves are still transformative works not intended to facilitate infringement.
  - flir 3mo ago
    It's possible. Would be interesting to see their evidence, and to know whether they can reproduce it for arbitrary articles, not just ones that have been endlessly republished on the net.
- freejazz 3mo ago
  > if you can't persuade the machine to spit substantially the same text back out verbatim That's exactly what they've done in a number of the lawsuits, so I'm not sure why you think that hasn't occurred.
  - flir 3mo ago
    I think it's occurred. That's why I wrote this in the very next paragraph: "Yes, for some texts that's possible." https://arxiv.org/abs/2601.02671 https://arxiv.org/abs/2601.02671 The point is that for most texts, it is not possible. It's not able to recall what I wrote on Geocities in 1995, even though there's a good chance it was trained on it.
    - freejazz 3mo ago
      No one is suing over what you wrote on Geocities in 1995...
      - SR2Z 3mo ago
        No but it's still copyrighted content and subject to the same protections.
SR2Z 3mo ago
Ok, what about a human summarizing it or taking notes? What about a human indexing it for later searches? Does it matter if they use notecards or if they do it on a computer? What if they retouch a photo you've taken as a political message? Does it matter if the do it with Sharpie, Photoshop, or by feeding it into an LLM? When you publish, you give up some control over your work. Other people are allowed to do things with it and IMO it does not matter if it's in their head, on paper, or in a computer.

flir 3mo ago

Whatever "training" is, if you can't persuade the machine to spit substantially the same text back out verbatim, it's clearly not something that falls under copyright law either, because there's no copy. Yes, for some texts that's possible. But for the vast majority, it is not.

tarnith 3mo ago
Can you cite any information on this not being possible for the vast majority? Or is it simply that the correct prompt hasn't been written for all possible cases? I also fail to see the difference if logic/harnessing is added around a vector database that can output the complete corpus, but simply is instructed not to. It very clearly is still compressing the information into the vector weights, and then recovering that information, thus the information is encoded. Why is a vector database somehow completely different from maintaining a library of the text itself?
- satvikpendem 3mo ago
  Information entropy. The amount of data an LLM ingests cannot be compressed to the size of the weights even at maximum theoretical compression.
  - insanitybit 3mo ago
    Is that relevant? I can use a lossy compression algorithm such that the original could never be recovered from the image I've produced, but that derived image would surely be under copyright. LLMs are obviously capable of producing "exact" phrases as well. Ask it to give you famous quotes, it can do it. Ask it to read a paper for you and cite it, it can do it.
    - flir 3mo ago
      > but that derived image would surely be under copyright. I wouldn't bet on that. https://en.wikipedia.org/wiki/Campbell%27s_Soup_Cans https://en.wikipedia.org/wiki/Campbell%27s_Soup_Cans
      - insanitybit 3mo ago
        I don't know that this really challenges anything relating to compression.
        flir 3mo ago
        Ok, reductio ad absurdum. Here's a highly compressed representation of The Lord of The Rings (all three volumes): 1 Obviously, fidelity when uncompressing it is not great, but I can assure you it was lossily compressed from the original text. Is it infringing the original's copyright? I have to assume you'd agree that the answer is "no". If I had compressed it by removing the letters x y and z, I'd agree with you that my "compressed" version is infringing. So what we've got here is a spectrum with two ridiculous extremes, and a question: When has the artifact been compressed so heavily that it no longer infringes the copyright of the original? I suggest "irretrievability" is a pretty good threshold for that question. Otherwise you're into "we know it infringes our copyright. Don't ask us to prove it, we just know it, ok?" Given the sheer volume of text that an LLM gets trained on, and how small the output is, it seems obvious that 99% of it can no longer be recovered - the process is "lossy" to the point of irretrievability, and only a statistical smear is left behind. That's why I think only the copyright claims that can show infringement in court (Harpy Potter, et al.) have merit. And a court will still have to decide "how much is too much" but at least there's case law for that. (Incidentally, I compressed the Mona Lisa to a single pixel. It was #3D3526).
        insanitybit 3mo ago
        My entire point is that compression is irrelevant. A lossily compressed image can absolutely still be in breach of the original's copyright. Showing that some kind of compressed artifact may not be doesn't change that.
- consensus1 3mo ago
  You are asking to prove a negative. But even assuming that the model is capable of returning every bit of its training data verbatim (a mathematical impossibility) that would not be enough as mere capability is insufficient here. If capability alone were the standard any library that also has a photocopier / scanner would be in violation. To prove distribution of copyrighted materials it would have to be practical and actually used in the wild by people to circumvent copyright and generate copies of those works. Again, I can't prove a negative, but that isn't the standard, and nobody has shown a practical exploit here.
  - flir 3mo ago
    There was a paper a while back where (from memory) they managed to coax 75% of the original text of some internet-popular books out of an LLM. Harry Potter, 1984, etc. That's why I said it was possible for some texts. My assumption is that multiple copies in the training data "wear a deeper groove". I believe those are infringing, and should be dealt with on a case-by-case basis. But the vast majority of text doesn't wear that groove. (Edit: Think it was this one https://arxiv.org/abs/2601.02671 https://arxiv.org/abs/2601.02671)
- flir 3mo ago
  Nah, I can't prove a negative. But Common Crawl is 12 petabytes and is not the largest part of what these models get trained on. DeepSeek v4 Pro is, what, 865GB? That's one hell of a compression ratio, if it can do what you claim.
ripe 3mo ago
> spitting out verbatim text The New York Times lawsuit is resting on the point that large chunks of undigested articles can be vomited out. OpenAI tried to have the lawsuit thrown out but the courts permitted it to continue. The Times... alleged that OpenAI's ChatGPT and Microsoft's Copilot had produced near-verbatim replicas of copyrighted articles, that the chatbots generated hallucinated content falsely attributed to the Times, ... https://en.wikipedia.org/wiki/The_New_York_Times_v._Microsoft_and_OpenAI https://en.wikipedia.org/wiki/The_New_York_Times_v._Microsof...
- nomel 3mo ago
  > spitting out verbatim text > had produced near-verbatim replicas
  - flir 3mo ago
    Don't get too hung up on the preciseness of the copy - the courts won't. I doubt that spitting out an existing article with a few adjectives changed would be considered transformative.
    - SR2Z 3mo ago
      What's almost certainly going to happen is that the courts will decide the infringing party in that situation is the one who willfully caused the violation. Since most users of LLMs are not using them to run around copyright and this copyright stuff is aggressively trained out of models or blocked whenever possible, the models themselves are still transformative works not intended to facilitate infringement.
- flir 3mo ago
  It's possible. Would be interesting to see their evidence, and to know whether they can reproduce it for arbitrary articles, not just ones that have been endlessly republished on the net.
freejazz 3mo ago
> if you can't persuade the machine to spit substantially the same text back out verbatim That's exactly what they've done in a number of the lawsuits, so I'm not sure why you think that hasn't occurred.
- flir 3mo ago
  I think it's occurred. That's why I wrote this in the very next paragraph: "Yes, for some texts that's possible." https://arxiv.org/abs/2601.02671 https://arxiv.org/abs/2601.02671 The point is that for most texts, it is not possible. It's not able to recall what I wrote on Geocities in 1995, even though there's a good chance it was trained on it.
  - freejazz 3mo ago
    No one is suing over what you wrote on Geocities in 1995...
    - SR2Z 3mo ago
      No but it's still copyrighted content and subject to the same protections.

SR2Z 3mo ago

Ok, what about a human summarizing it or taking notes? What about a human indexing it for later searches? Does it matter if they use notecards or if they do it on a computer? What if they retouch a photo you've taken as a political message? Does it matter if the do it with Sharpie, Photoshop, or by feeding it into an LLM? When you publish, you give up some control over your work. Other people are allowed to do things with it and IMO it does not matter if it's in their head, on paper, or in a computer.