5 ms·

> Everything on the open web after 2023 is polluted with lowquality AI slop that poisons the data sets. Not even close to everything. E.g. training on the NY

> Everything on the open web after 2023 is polluted with lowquality AI slop that poisons the data sets. Not even close to everything. E.g. training on the NY Times and Wikipedia has zero meaningful AI. Training on books from reputable publishers similarly has zero meaningful AI. Any LLM usage was to polish prose or assist with research or whatever, but shouldn't affect the factual quality in any significant way. The web hasn't been polluted with AI any more than e-mail has been polluted with spam. Which is to say it's there, but it's also entirely viable to separate. Nobody's worried that the group email chain with friends is being overrun with spam or with AI.

dgfitz 2y ago
[flagged]
mplewis 2y ago
There is no heuristic that reliably separates human-written text from GPT text. It is impossible to avoid model collapse in such an environment.
- ValentinA23 2y ago
  Unless you're OpenAI and provided like 98% of ai generated text that was posted to the internet.
- Der_Einzige 2y ago
  Anti-slop sampler and bigram/trigram stylometric analysis absolutely can give evidence that someone used a particular LLM with particular settings. But in general it’s hard to separate human from AI text.
- crazygringo 2y ago
  You're ignoring the information about the source. It's not about a heuristic on text of unknown provenance -- it's about publishers that exert a certain level of editorial control and quality verification. Or social reputation mechanisms that achieve the same. That's what is preventing your "model collapse". Reputations of provenance. Not pure-text heuristics.
intended 2y ago
Training on Ny Times and Wikipedia… The Wikipedia part at least, is incorrect. Currently Wikipedia mods/admins are dealing with AI generated articles being uploaded. As for NYT - I am assuming that lots of those stories are already available in some blog or the other. The e-mail and web forums are 100% polluted with spam, which takes constant effort to remove. For GenAI based content, it is far harder to identify and remove. This example assumes the effort required to keep the web functional can deal with AI created content. Speaking from experience, our filters (human and otherwise) cannot. They fail to do so even now. PS: Even given your example of closed email chains - the information in that depends on sources people read. Like plastic pollution in the food chain, this is inescapable.
- crazygringo 2y ago
  > Currently Wikipedia mods/admins are dealing with AI generated articles being uploaded. And they've always dealt with spam and low-quality submissions before. The system is working. > As for NYT - I am assuming that lots of those stories are already available in some blog or the other. I don't know what relevance that has to what we're talking about. The point is, train on the NYT. Blogs don't change what's on the NYT. > The e-mail and web forums are 100% polluted with spam, which takes constant effort to remove. They've always been polluted with low-quality content. So yes, either don't train on them, or only train on highly upvoted solutions, etc. AI pollution isn't fundamentally any different from previous low-quality content and spam. It's not terribly difficult to determine which parts of the internet are known to be high-quality and train only on those. LLM's can't spam the NY Times.
  - intended 2y ago
    >The system is working. Given that this is an issue I've heard from wikipedia admins themselves, I am impressed by your confidence. >The point is, train on the NYT. Blogs don't change what's on the NYT. The counter point is that NYT content is already in the training data because its already replicated or copied into random blogs. >So yes, either don't train on them, or only train on highly upvoted solutions, etc. Highly upvoted messages on reddit are very regular bots copying older top comments. Mods already have issues with AI comments. ---- TLDR: Pollution is already happening. Verification does not scale, while generation scales.

dgfitz 2y ago

[flagged]

mplewis 2y ago

There is no heuristic that reliably separates human-written text from GPT text. It is impossible to avoid model collapse in such an environment.

ValentinA23 2y ago
Unless you're OpenAI and provided like 98% of ai generated text that was posted to the internet.
Der_Einzige 2y ago
Anti-slop sampler and bigram/trigram stylometric analysis absolutely can give evidence that someone used a particular LLM with particular settings. But in general it’s hard to separate human from AI text.
crazygringo 2y ago
You're ignoring the information about the source. It's not about a heuristic on text of unknown provenance -- it's about publishers that exert a certain level of editorial control and quality verification. Or social reputation mechanisms that achieve the same. That's what is preventing your "model collapse". Reputations of provenance. Not pure-text heuristics.

intended 2y ago

Training on Ny Times and Wikipedia… The Wikipedia part at least, is incorrect. Currently Wikipedia mods/admins are dealing with AI generated articles being uploaded. As for NYT - I am assuming that lots of those stories are already available in some blog or the other. The e-mail and web forums are 100% polluted with spam, which takes constant effort to remove. For GenAI based content, it is far harder to identify and remove. This example assumes the effort required to keep the web functional can deal with AI created content. Speaking from experience, our filters (human and otherwise) cannot. They fail to do so even now. PS: Even given your example of closed email chains - the information in that depends on sources people read. Like plastic pollution in the food chain, this is inescapable.

crazygringo 2y ago
> Currently Wikipedia mods/admins are dealing with AI generated articles being uploaded. And they've always dealt with spam and low-quality submissions before. The system is working. > As for NYT - I am assuming that lots of those stories are already available in some blog or the other. I don't know what relevance that has to what we're talking about. The point is, train on the NYT. Blogs don't change what's on the NYT. > The e-mail and web forums are 100% polluted with spam, which takes constant effort to remove. They've always been polluted with low-quality content. So yes, either don't train on them, or only train on highly upvoted solutions, etc. AI pollution isn't fundamentally any different from previous low-quality content and spam. It's not terribly difficult to determine which parts of the internet are known to be high-quality and train only on those. LLM's can't spam the NY Times.
- intended 2y ago
  >The system is working. Given that this is an issue I've heard from wikipedia admins themselves, I am impressed by your confidence. >The point is, train on the NYT. Blogs don't change what's on the NYT. The counter point is that NYT content is already in the training data because its already replicated or copied into random blogs. >So yes, either don't train on them, or only train on highly upvoted solutions, etc. Highly upvoted messages on reddit are very regular bots copying older top comments. Mods already have issues with AI comments. ---- TLDR: Pollution is already happening. Verification does not scale, while generation scales.