4 ms·

> I don’t actually think intellectual property restrictions are good We already see huge amounts of LLM generated garbage on the web, as blogspam, regular webs

> I don’t actually think intellectual property restrictions are good We already see huge amounts of LLM generated garbage on the web, as blogspam, regular websites, etc. Amazon is getting flooded with LLM-written books. Chat bots/search engines regurgitate the actually valuable content If things continue, soon the motivation for generating new content will trend towards zero.

timtom123 2y ago
I don't think so. People will just follow personalities not aggregators. I can go buy a Neal Stephenson and know it is not spam. I can watch Cody'sLab and no it is not spam. By the web do you really mean google and FaceBook? Most of the channels/feeds I follow have not been flooded with spam. I am pleased that the large corporation content aggregators are struggling. It isn't like they produced content themselves anyway.
- voltaireodactyl 2y ago
  One day Neal Stephenson will be dead. How will we find the next one?
bilekas 2y ago
> If things continue, soon the motivation for generating new content will trend towards zero. I see it differently, I see that this generated nonsense will be filtered out, like SEO 'hackers' way back in the day. And those real articles, who actually have some insight will be valued a lot, the issue at hand here, is that the handcrafted unique data that was created, will be pooled without consent to a training set for further changes. It's the equivalent of the rich eating the poor in my eyes just on automated steroids. Edit : To avoid all doom and gloom, a method to guarantee your site/data is not added to a training set is required. This sounds good on paper to me, and is really needed, but just like peoples personal privacy, I don't think the sentiment is there to put the effort in.
- visarga 2y ago
  Maybe if you attach #NoAI next to the content it would be filtered out. But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe. They are decomposing, recombining, and regenerating language, not copying it. They execute user commands, not imitating any author on purpose. Say you want to read Harry Potter without paying, would you rather borrow a book or ask chatGPT to regurgitate the original? It would never work well, and be slower and more expensive to use AI. It's not a tool for infringement, it will naturally hallucinate and degrade the original, it can't possibly store a perfect copy of everything it trains on. In fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out. There is no space to put all that copyrighted data in a model, it necessarily compresses the hell out of it. Plain old piracy is still 1000x 'better' than infringement by AI.
  - bilekas 2y ago
    > But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe. From a copyright perspective, I do agree, you're not reproducing someone else's work. However, it's a 'new-ish' area of open source licensing, if your product is a product from others peoples product without at a bare minimum, citations (given the author said 'go wild with this data'), it's maybe legally rude but not a problem. But without permission, IE; no permissive license to USE someone's content, it's SOME kind of new infringement ? > n fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out. This is wild and I didn't know this, but again, the gravy was made from bones.. I'm not sure it matters here ?
    - visarga 2y ago
      Copyright as it says in the title concerns with copying, and does not confer rights to restrict automated analysis and statistical modeling. I think the trend against copyright coincides with the growth of internet. We moved from long form, passive consumption - books, movies, TV and radio - to games, social networks, search engines - generally interactive formats where there is no compensation for copying. We create our own content. LLMs are squarely in the interactive camp. Copyright was fit to the passive consumption model, but now it is standing in a precarious position facing the future. It is ineffective against internet and AI.
exe34 2y ago
> If things continue, soon the motivation for generating new content will trend towards zero. only the ones trying to make money out of it. a lot of people create content because it has to be let out - it's like art. people have stuff to say, and they love saying it.
visarga 2y ago
I did some searching and found the web distribution contains factual inaccuracies up to 15%, while LLMs are around 5-15%. Generally the LLM text has better formatting and language. So I would trust books > LLM > web scrape.

timtom123 2y ago

I don't think so. People will just follow personalities not aggregators. I can go buy a Neal Stephenson and know it is not spam. I can watch Cody'sLab and no it is not spam. By the web do you really mean google and FaceBook? Most of the channels/feeds I follow have not been flooded with spam. I am pleased that the large corporation content aggregators are struggling. It isn't like they produced content themselves anyway.

voltaireodactyl 2y ago
One day Neal Stephenson will be dead. How will we find the next one?

bilekas 2y ago

> If things continue, soon the motivation for generating new content will trend towards zero. I see it differently, I see that this generated nonsense will be filtered out, like SEO 'hackers' way back in the day. And those real articles, who actually have some insight will be valued a lot, the issue at hand here, is that the handcrafted unique data that was created, will be pooled without consent to a training set for further changes. It's the equivalent of the rich eating the poor in my eyes just on automated steroids. Edit : To avoid all doom and gloom, a method to guarantee your site/data is not added to a training set is required. This sounds good on paper to me, and is really needed, but just like peoples personal privacy, I don't think the sentiment is there to put the effort in.

visarga 2y ago
Maybe if you attach #NoAI next to the content it would be filtered out. But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe. They are decomposing, recombining, and regenerating language, not copying it. They execute user commands, not imitating any author on purpose. Say you want to read Harry Potter without paying, would you rather borrow a book or ask chatGPT to regurgitate the original? It would never work well, and be slower and more expensive to use AI. It's not a tool for infringement, it will naturally hallucinate and degrade the original, it can't possibly store a perfect copy of everything it trains on. In fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out. There is no space to put all that copyrighted data in a model, it necessarily compresses the hell out of it. Plain old piracy is still 1000x 'better' than infringement by AI.
- bilekas 2y ago
  > But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe. From a copyright perspective, I do agree, you're not reproducing someone else's work. However, it's a 'new-ish' area of open source licensing, if your product is a product from others peoples product without at a bare minimum, citations (given the author said 'go wild with this data'), it's maybe legally rude but not a problem. But without permission, IE; no permissive license to USE someone's content, it's SOME kind of new infringement ? > n fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out. This is wild and I didn't know this, but again, the gravy was made from bones.. I'm not sure it matters here ?
  - visarga 2y ago
    Copyright as it says in the title concerns with copying, and does not confer rights to restrict automated analysis and statistical modeling. I think the trend against copyright coincides with the growth of internet. We moved from long form, passive consumption - books, movies, TV and radio - to games, social networks, search engines - generally interactive formats where there is no compensation for copying. We create our own content. LLMs are squarely in the interactive camp. Copyright was fit to the passive consumption model, but now it is standing in a precarious position facing the future. It is ineffective against internet and AI.

exe34 2y ago

> If things continue, soon the motivation for generating new content will trend towards zero. only the ones trying to make money out of it. a lot of people create content because it has to be let out - it's like art. people have stuff to say, and they love saying it.

visarga 2y ago

I did some searching and found the web distribution contains factual inaccuracies up to 15%, while LLMs are around 5-15%. Generally the LLM text has better formatting and language. So I would trust books > LLM > web scrape.