5 ms·

It's not about advantages, it's about the fact that if you pull in any wild dataset for a language model, there's an overwhelming chance that some of that wild

It's not about advantages, it's about the fact that if you pull in any wild dataset for a language model, there's an overwhelming chance that some of that wild data was produced by an AI. E.g. if you're training an LLM on Reddit, there's a good shot that some of the Reddit posts you're pulling in were generated by an AI.

amelius 4y ago
Then train on posts only if they were upvoted.
- gkbrk 4y ago
  You are assuming AI-generated content doesn't get upvoted. It's entirely possible that they even get more upvotes than real humans producing organic content.
  - amelius 4y ago
    Well, if it got upvoted (by humans) then isn't it reasonable to train on it? Of course, one challenge is to make sure AI doesn't interfere with moderation. Perhaps this whole situation will spur more work on moderation systems.
    - froggit 4y ago
      It seems like I keep seeing more about AI being used for moderation. I guess that'd be a form of AI being trained on the output of AI (or I guess the output of humans that an AI found acceptable). With AIs training each other it seems like, given enough time, it would only take a couple "poison" AIs in the loop to make things get really maladaptive. While this isn't what I'd call comparable to the huge models (written by real data scientists that might know what they're doing), I've seen a few nudges to data being fed into extremely basic self-reinforcing ML models (written by fake data scientists aka grad students in an intro AI class that didn't know what they were doing or why, but nonetheless fully understood the underlying math needed to write working models) start getting weird outputs on iterations much further down the line that became increasingly problematic, until eventually there really was no longer a relationship to the original data.
  - sebzim4500 4y ago
    If the AI content does better than the human generated content then why would training on it be a problem? Much of the success of chatgpt comes from RLHF, which can be viewed as training on its own data, but only the data which has been determined by a human to be especially good.
    - Ambolia 4y ago
      The AI content gets no input from the world outside of the internet except through humans.
- gameman144 4y ago
  That's one heuristic you could use, yes. If your goal is "engaging content", that's probably a reasonable one. If your goal is "human-generated content", it's not.

amelius 4y ago

Then train on posts only if they were upvoted.

gkbrk 4y ago
You are assuming AI-generated content doesn't get upvoted. It's entirely possible that they even get more upvotes than real humans producing organic content.
- amelius 4y ago
  Well, if it got upvoted (by humans) then isn't it reasonable to train on it? Of course, one challenge is to make sure AI doesn't interfere with moderation. Perhaps this whole situation will spur more work on moderation systems.
  - froggit 4y ago
    It seems like I keep seeing more about AI being used for moderation. I guess that'd be a form of AI being trained on the output of AI (or I guess the output of humans that an AI found acceptable). With AIs training each other it seems like, given enough time, it would only take a couple "poison" AIs in the loop to make things get really maladaptive. While this isn't what I'd call comparable to the huge models (written by real data scientists that might know what they're doing), I've seen a few nudges to data being fed into extremely basic self-reinforcing ML models (written by fake data scientists aka grad students in an intro AI class that didn't know what they were doing or why, but nonetheless fully understood the underlying math needed to write working models) start getting weird outputs on iterations much further down the line that became increasingly problematic, until eventually there really was no longer a relationship to the original data.
- sebzim4500 4y ago
  If the AI content does better than the human generated content then why would training on it be a problem? Much of the success of chatgpt comes from RLHF, which can be viewed as training on its own data, but only the data which has been determined by a human to be especially good.
  - Ambolia 4y ago
    The AI content gets no input from the world outside of the internet except through humans.
gameman144 4y ago
That's one heuristic you could use, yes. If your goal is "engaging content", that's probably a reasonable one. If your goal is "human-generated content", it's not.