3 ms·

Maybe if you attach #NoAI next to the content it would be filtered out. But from a copyright standpoint, unless it regurgitates training data which is rare and

Maybe if you attach #NoAI next to the content it would be filtered out. But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe. They are decomposing, recombining, and regenerating language, not copying it. They execute user commands, not imitating any author on purpose. Say you want to read Harry Potter without paying, would you rather borrow a book or ask chatGPT to regurgitate the original? It would never work well, and be slower and more expensive to use AI. It's not a tool for infringement, it will naturally hallucinate and degrade the original, it can't possibly store a perfect copy of everything it trains on. In fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out. There is no space to put all that copyrighted data in a model, it necessarily compresses the hell out of it. Plain old piracy is still 1000x 'better' than infringement by AI.

bilekas 2y ago
> But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe. From a copyright perspective, I do agree, you're not reproducing someone else's work. However, it's a 'new-ish' area of open source licensing, if your product is a product from others peoples product without at a bare minimum, citations (given the author said 'go wild with this data'), it's maybe legally rude but not a problem. But without permission, IE; no permissive license to USE someone's content, it's SOME kind of new infringement ? > n fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out. This is wild and I didn't know this, but again, the gravy was made from bones.. I'm not sure it matters here ?
- visarga 2y ago
  Copyright as it says in the title concerns with copying, and does not confer rights to restrict automated analysis and statistical modeling. I think the trend against copyright coincides with the growth of internet. We moved from long form, passive consumption - books, movies, TV and radio - to games, social networks, search engines - generally interactive formats where there is no compensation for copying. We create our own content. LLMs are squarely in the interactive camp. Copyright was fit to the passive consumption model, but now it is standing in a precarious position facing the future. It is ineffective against internet and AI.
  - bilekas 2y ago
    I agree, but would you also agree that this is exactly why there is a need now for the courts to step in and define these 'new-frontiers' of data reproduction ?
- prewett 2y ago
  > the gravy was made from bones Probably not very relevant to the point, but gravy is made from the meat juices and fat. Bones get made into bone broth.

bilekas 2y ago

> But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe. From a copyright perspective, I do agree, you're not reproducing someone else's work. However, it's a 'new-ish' area of open source licensing, if your product is a product from others peoples product without at a bare minimum, citations (given the author said 'go wild with this data'), it's maybe legally rude but not a problem. But without permission, IE; no permissive license to USE someone's content, it's SOME kind of new infringement ? > n fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out. This is wild and I didn't know this, but again, the gravy was made from bones.. I'm not sure it matters here ?

visarga 2y ago
Copyright as it says in the title concerns with copying, and does not confer rights to restrict automated analysis and statistical modeling. I think the trend against copyright coincides with the growth of internet. We moved from long form, passive consumption - books, movies, TV and radio - to games, social networks, search engines - generally interactive formats where there is no compensation for copying. We create our own content. LLMs are squarely in the interactive camp. Copyright was fit to the passive consumption model, but now it is standing in a precarious position facing the future. It is ineffective against internet and AI.
- bilekas 2y ago
  I agree, but would you also agree that this is exactly why there is a need now for the courts to step in and define these 'new-frontiers' of data reproduction ?
prewett 2y ago
> the gravy was made from bones Probably not very relevant to the point, but gravy is made from the meat juices and fat. Bones get made into bone broth.