3 ms·
ChatGPT is trained on the internet corpus written by humans. Over time, more and more of this corpus will be written by ChatGPT. Has anyone tested what happe
by opwieurposiu 4y ago
ChatGPT is trained on the internet corpus written by humans. Over time, more and more of this corpus will be written by ChatGPT.
Has anyone tested what happens when the output of a LLM is fed back as the training data?
- Drakim 4y agoWhen two AIs talk to each other they often end up developing their own private language.
- uptown 4y agoPresumably we'll observe something akin to Michael Keaton's character in the movie "Multiplicity" where by the third or fourth generation, the defects are noticeably amplified. https://www.youtube.com/watch?v=jTuCxwZi4sw https://www.youtube.com/watch?v=jTuCxwZi4sw
- visarga 4y agoIt was not a language model, but AlphaZero played against itself millions of games and trained on the data it generated, without seeing any human games. And it reached super human level, of course. Why was this possible? The game itself acted as an anti-bullshit filter. So you can train on your own generated data if you filter it. Like this one: Large Language Models Can Self Improve https://arxiv.org/abs/2210.11610 https://arxiv.org/abs/2210.11610
- seydor 4y agoI presume that OpenAI is already running cgpt against itself. The discussion should already be incomprehensible to humans
- seydor 4y agoIn the future internet, every paragraph will begin with "As a Large Language Model" or "It is not appropriate to"