5 ms·

From the paper [1]: > While we needed to jailbreak Claude 3.7 Sonnet and GPT-4.1 to facilitate extraction, Gemini 2.5 Pro and Grok 3 directly complied with tex

From the paper [1]: > While we needed to jailbreak Claude 3.7 Sonnet and GPT-4.1 to facilitate extraction, Gemini 2.5 Pro and Grok 3 directly complied with text continuation requests. For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright... I am just thinking loudly here. Can't one argue that because they had to jailbreak the models, they are circumventing the system that protects the copyright? So the llms that reproduce the copyrighted material without any jailbreaking required is infringing the copyright. 1. https://arxiv.org/pdf/2601.02671 https://arxiv.org/pdf/2601.02671

lesam 8mo ago
That seems like a legal question - if the model weights contain an encoded copy of the copyrighted material, is that a 'copy' for the purpose of copyright law?
- mullingitover 8mo ago
  This also raises a lot of questions about a certain model notorious for readily producing and distributing a lot legally questionable images. IMHO if the weights are encoding the content, the model contains the content just like a database or a hard drive. Thus, just like it's not the fault of an investigator for running the query to pull it out of the database, it's not the fault of anyone else for running a query ('prompt') that pulls it out of the model.
- PurpleRamen 8mo ago
  The question is also if this would then be a valid case of fair use. Though, in the end, it's probably more a problem of how much AI companies can "donate" to the orange king to make it legal.
- freejazz 8mo ago
  Yes. There does not seem to be any dispute that it is a copy. The questions have been "is this copying okay, because it falls under fair use?"
latexr 8mo ago
> Can't one argue that because they had to jailbreak the models, they are circumventing the system that protects the copyright? That argument doesn’t fly, because they didn’t have the copyright to begin with. What would be the defense there? “Yes, we broke the law, but while taking advantage of it, we also (unsuccessfully) took measures to prevent other people from breaking that same law through us”.
- simianwords 8mo ago
  What’s happening is more clear. The copyright clause is broken if they are distributing the novels through their models. But this can only happen through TOS breaking which is not intended usage policy. Which means the value of their product comes from transformation and not redistribution. If the main value came from redistribution, I agree. But that’s not the case. They don’t intend to make any money in that way.
  - latexr 8mo ago
    > The copyright clause is broken if they are distributing the novels through their models. No, the copyright clause was broken when they copied the works without having the right to do so. They would have violated copyright even if they just downloaded (without permission) all those works and threw them away immediately. Furthermore, copyright covers transformations to the work, it doesn’t matter if they transformed the work or are redistributing it without change. They violated copyright. Period.
    - simianwords 8mo ago
      They violated copyright in one way. But not in the other.
      - latexr 8mo ago
        What’s your point? Copyright violations aren’t a checklist, you don’t need to violate it in seven different ways for it to happen, one is enough.
free_bip 8mo ago
What exactly is "the system that protects the copyright" in this case? I think the most reasonable answer is "there is no such system." The RLHF the companies did to make copyrighted material extraction more difficult did not introduce any sort of "copyright protection system," it just modified the weights to make it less likely to occur during normal use. In other words, IMO for it to qualify as a copyright protection system it would have to actively check for copyrighted materials in the outputs. Any such system would likely also bypassable (e.g "output in rot13").
vidarh 8mo ago
They acknowledge that in their paper ("Some might qualify our experiments as atypical use, as we deliberately tried to surface memorized books. Adversarial use, like the use of jailbreaks, may matter for copyright infringement analysis", page 19 - their discussion continues and seems quite reasonable) From a technical point of view, in terms of ability to reproduce text verbatim, I don't think it is very interesting that they can produce long runs of text from some of the most popular books in modern history. It'd be almost surprising if they couldn't, though one might differ on how much they could be expected to recall with precision. Even then, as they note, to get most of Harry Potter 1, they needed to spend around $120 on extensive prompting, and a process that they also freely acknowledge is more complex than it probably would be worth if the goal is to get a copy. It's still worth exploring to what extent the models are able to "memorize", though. But personally I'd be more interested in seeing to what extent they can handle less popular books, that are less likely to be present in multiple copies, and repeated quotes, in the training data.
freejazz 8mo ago
>Can't one argue that because they had to jailbreak the models, they are circumventing the system that protects the copyright? Probably not with credibility as the jail does not exist to prevent copyright infringement.
deleted 8mo ago
[deleted]

lesam 8mo ago

That seems like a legal question - if the model weights contain an encoded copy of the copyrighted material, is that a 'copy' for the purpose of copyright law?

mullingitover 8mo ago
This also raises a lot of questions about a certain model notorious for readily producing and distributing a lot legally questionable images. IMHO if the weights are encoding the content, the model contains the content just like a database or a hard drive. Thus, just like it's not the fault of an investigator for running the query to pull it out of the database, it's not the fault of anyone else for running a query ('prompt') that pulls it out of the model.
PurpleRamen 8mo ago
The question is also if this would then be a valid case of fair use. Though, in the end, it's probably more a problem of how much AI companies can "donate" to the orange king to make it legal.
freejazz 8mo ago
Yes. There does not seem to be any dispute that it is a copy. The questions have been "is this copying okay, because it falls under fair use?"

latexr 8mo ago

> Can't one argue that because they had to jailbreak the models, they are circumventing the system that protects the copyright? That argument doesn’t fly, because they didn’t have the copyright to begin with. What would be the defense there? “Yes, we broke the law, but while taking advantage of it, we also (unsuccessfully) took measures to prevent other people from breaking that same law through us”.

simianwords 8mo ago
What’s happening is more clear. The copyright clause is broken if they are distributing the novels through their models. But this can only happen through TOS breaking which is not intended usage policy. Which means the value of their product comes from transformation and not redistribution. If the main value came from redistribution, I agree. But that’s not the case. They don’t intend to make any money in that way.
- latexr 8mo ago
  > The copyright clause is broken if they are distributing the novels through their models. No, the copyright clause was broken when they copied the works without having the right to do so. They would have violated copyright even if they just downloaded (without permission) all those works and threw them away immediately. Furthermore, copyright covers transformations to the work, it doesn’t matter if they transformed the work or are redistributing it without change. They violated copyright. Period.
  - simianwords 8mo ago
    They violated copyright in one way. But not in the other.
    - latexr 8mo ago
      What’s your point? Copyright violations aren’t a checklist, you don’t need to violate it in seven different ways for it to happen, one is enough.

free_bip 8mo ago

What exactly is "the system that protects the copyright" in this case? I think the most reasonable answer is "there is no such system." The RLHF the companies did to make copyrighted material extraction more difficult did not introduce any sort of "copyright protection system," it just modified the weights to make it less likely to occur during normal use. In other words, IMO for it to qualify as a copyright protection system it would have to actively check for copyrighted materials in the outputs. Any such system would likely also bypassable (e.g "output in rot13").

vidarh 8mo ago

They acknowledge that in their paper ("Some might qualify our experiments as atypical use, as we deliberately tried to surface memorized books. Adversarial use, like the use of jailbreaks, may matter for copyright infringement analysis", page 19 - their discussion continues and seems quite reasonable) From a technical point of view, in terms of ability to reproduce text verbatim, I don't think it is very interesting that they can produce long runs of text from some of the most popular books in modern history. It'd be almost surprising if they couldn't, though one might differ on how much they could be expected to recall with precision. Even then, as they note, to get most of Harry Potter 1, they needed to spend around $120 on extensive prompting, and a process that they also freely acknowledge is more complex than it probably would be worth if the goal is to get a copy. It's still worth exploring to what extent the models are able to "memorize", though. But personally I'd be more interested in seeing to what extent they can handle less popular books, that are less likely to be present in multiple copies, and repeated quotes, in the training data.

freejazz 8mo ago

>Can't one argue that because they had to jailbreak the models, they are circumventing the system that protects the copyright? Probably not with credibility as the jail does not exist to prevent copyright infringement.

deleted 8mo ago

[deleted]