4 ms·

Copyright law stipulates the conditions in which content can be reproduced, not conditions in which it can be consumed. Arguably the material has been learned

Copyright law stipulates the conditions in which content can be reproduced, not conditions in which it can be consumed. Arguably the material has been learned and not copied. Maybe in some cases learned with an uncanny ability or photographic memory, but learned. (People with photographic memories also cannot reproduce content in an unlimited fashion).

b112 2y ago
Learned! There's nothing special about an LLM, there's no learning, and they regurgitate verbatim text too. May as well say curl + images in a db are learned as well, so thus I can use Mickey Miuse as I please in my php web page.
- drdeca 2y ago
  While learned is probably not the best word to use as far as describing the legality goes, I also don’t think “copied” is the right word. Let’s say that the model “is influenced by” the copyrighted material. That seems hard to argue against. So, now that we aren’t using the word “learned”, why would we say that the way the models are influenced by the copyrighted works that appear in the “training set” (not to imply that “training” in the usual sense is happening) counts as a copyright violation? Or, perhaps the claim is that the outputs of the model are violating copyright? If the output is substantially similar to some particular copyrighted work that is in the “training set”, and could work as a substitute for that work, and if the output resembling the work is in part due to the influence that the work had on the model, I think in this case it would be a clear case of violation of copyright. However, if it doesn’t have substantial similarity to any particular copyrighted work that influenced it, only similarity to the style common to many of the works that influenced it (even if all by the same author), my impression is that this would not constitute copyright infringement because styles are not protected by copyright, only individual works. (Now, is this unjust, in the case of it copying the style of some particular author/artist? Idk, maybe? But my impression is that copyright doesn’t protect styles, and that it probably shouldn’t protect styles in general… so I guess maybe if we had a law making a special case forbidding the (deliberate?) copying of a person’s particular style via some kind of machine learning model? Idk.)
  - galaxyLogic 2y ago
    Very insightful. Consider there are many "recreational imitators" who mimic how specific (famous) people speak. They are not violating copyright, they are just imitating a way of speaking.
    - echoangle 2y ago
      I don’t think this is a good argument because the way of speaking isn’t a copyright issue. I don’t think you have copyright on your specific way of speaking, only on specific recordings of you yourself speaking.
      - galaxyLogic 2y ago
        I think I'm saying the same thing. Style is not something that can be copyrighted but I hadn't much thought about it before. I'm not a lawyer. I guess trademarks are something, and design-patents perhaps that let you have IP over style. :-)
  - tsimionescu 2y ago
    The argument is that the LLM itself is essentially like a complex lossy archive of its entire training set. It's like an mp3 of all of the songs on Spotify, in some sense (of course, using all text on the internet instead of all songs on Spotify). This is the sense in which it is considered to be a copy of all of this.
    - deleted 2y ago
      [deleted]
- visarga 2y ago
  > There's nothing special about an LLM, there's no learning The model is 100-500x smaller than its training set. That is something hinting at learning, as direct storage is impossible.
  - toast0 2y ago
    Video compression ratios vs raw video is amazing too, but there's no learning and there's no doubt that the compressed form is subject to copyright.

b112 2y ago

Learned! There's nothing special about an LLM, there's no learning, and they regurgitate verbatim text too. May as well say curl + images in a db are learned as well, so thus I can use Mickey Miuse as I please in my php web page.

drdeca 2y ago
While learned is probably not the best word to use as far as describing the legality goes, I also don’t think “copied” is the right word. Let’s say that the model “is influenced by” the copyrighted material. That seems hard to argue against. So, now that we aren’t using the word “learned”, why would we say that the way the models are influenced by the copyrighted works that appear in the “training set” (not to imply that “training” in the usual sense is happening) counts as a copyright violation? Or, perhaps the claim is that the outputs of the model are violating copyright? If the output is substantially similar to some particular copyrighted work that is in the “training set”, and could work as a substitute for that work, and if the output resembling the work is in part due to the influence that the work had on the model, I think in this case it would be a clear case of violation of copyright. However, if it doesn’t have substantial similarity to any particular copyrighted work that influenced it, only similarity to the style common to many of the works that influenced it (even if all by the same author), my impression is that this would not constitute copyright infringement because styles are not protected by copyright, only individual works. (Now, is this unjust, in the case of it copying the style of some particular author/artist? Idk, maybe? But my impression is that copyright doesn’t protect styles, and that it probably shouldn’t protect styles in general… so I guess maybe if we had a law making a special case forbidding the (deliberate?) copying of a person’s particular style via some kind of machine learning model? Idk.)
- galaxyLogic 2y ago
  Very insightful. Consider there are many "recreational imitators" who mimic how specific (famous) people speak. They are not violating copyright, they are just imitating a way of speaking.
  - echoangle 2y ago
    I don’t think this is a good argument because the way of speaking isn’t a copyright issue. I don’t think you have copyright on your specific way of speaking, only on specific recordings of you yourself speaking.
    - galaxyLogic 2y ago
      I think I'm saying the same thing. Style is not something that can be copyrighted but I hadn't much thought about it before. I'm not a lawyer. I guess trademarks are something, and design-patents perhaps that let you have IP over style. :-)
- tsimionescu 2y ago
  The argument is that the LLM itself is essentially like a complex lossy archive of its entire training set. It's like an mp3 of all of the songs on Spotify, in some sense (of course, using all text on the internet instead of all songs on Spotify). This is the sense in which it is considered to be a copy of all of this.
  - deleted 2y ago
    [deleted]
visarga 2y ago
> There's nothing special about an LLM, there's no learning The model is 100-500x smaller than its training set. That is something hinting at learning, as direct storage is impossible.
- toast0 2y ago
  Video compression ratios vs raw video is amazing too, but there's no learning and there's no doubt that the compressed form is subject to copyright.