4 ms·
Q: What happens when output from LLMs start feeding back into training new LLMs? A: A self degrading BS feedback loop leading to eventual collapse of reliabili
by jqpabc123 3y ago
Q: What happens when output from LLMs start feeding back into training new LLMs?
A: A self degrading BS feedback loop leading to eventual collapse of reliability.
It will take as much time and effort to verify output from a LLM as it does to do the research/legwork yourself.
Maybe more because you'll now have to filter out and reject all the plausible sounding BS created by LLMs.
- senttoschool 3y ago>Q: What happens when output from LLMs start feeding back into training new LLMs? Humans do this all the time. For example, fake news spreading and most people believing in it.
- staticman2 3y agoWhat's the point of constantly comparing this technology to humans? If I said my calculator gives false numbers you wouldn't reply "humans are sometimes bad at math".
- _ea1k 3y agoNo, but if my LLM were bad at math, I might say "humans are sometimes bad at math". One major missing element in current ML approaches is tool usage and how to decide when to use a tool vs not using one. It is an area that is currently being explored and expanded quickly.
- dekhn 3y agobecause the whole point of these systems is to take over menial tasks that humans are slow/bad/find boring. Same reason as calculators: they do something humans could do, but faster and more accurately, allowing us to focus our intelligence on higher order problems.
- raspyberr 3y agoBut some humans are skeptical and/or critical thinkers and will get this new information verified or disproven through other methods. Also, fake news spreading seems mainly an issue of media/social media. There is no information spreading mechanism set up yet for LLMs. Furthermore, to LLMs all information looks the same. To humans more emotionally charged information is more prominent.
- squarefoot 3y agoOne difference being that we have previous education, culture and critical thinking allowing us to not fall into bullshit even in case it is being spread by more sources than the correct information. I'm not sure an AI being trained on the load of garbage that comes from social media today could develop that level of resistance.
- senttoschool 3y agoYou overestimate the average human's ability to critically think for themselves.
- jqpabc123 3y agoTrue but it misses the point. LLMs will only help fake news spread faster because they're dumb --- they can't tell the difference between real and fake. They are indiscriminate multipliers of both real and fake data.
- kypro 3y agoThis idea that LLMs will get progressively dumber because of a training feedback loop is silly. If this ever did become become a genuine concern companies training LLMs would just use preexisting datasets and selectively add additional data that they can verify the quality of. Not everything is Reddit. I doubt the NYT is going to start publishing BS. Scientific journals like Nature also aren't going to start publishing BS because of LLMs. There will be plenty of great resources to train future iterations of LLMs, even if the days of just scraping the entire internet for a dataset has passed.
- kneebonian 3y ago> I doubt the NYT is going to start publishing BS. Scientific journals like Nature also aren't going to start publishing BS because of LLMs. Will they though? Leaving aside the NYT which I contend already publishes quite a bit of bull hockey. Here's how it goes. Researcher: Provides outline of paper, GPT turn this into a scientific paper for me. GPT: Turns it into a paper, but includes several not quite accurate statements, or hallucinates some information and facts, as well as fabricating just one or two citations. Researcher: Gives it a once over, not a through review because that would defeat the point of using an LLM to generate the content in the first place. Paper gets submitted. Reviewer: Receives paper, throws it into GPT, "summarize this for me", GPT spits out a summary that looks mostly good, the reviewer skims over the actual paper, there are no glaring problems it gets published. End result. We now have a paper with fabricated information stated as fact published in a reliable place, that then is cited and referenced, resulting in a degradation of quality of the information. This repeats and repeats and suddenly whole conferences are built on entirely hallucinated information.
- jqpabc123 3y agoIf this ever did become become a genuine concern companies training LLMs would just use preexisting datasets So they will be cutoff from the internet and never consume new data from it? Is this what they're doing now? I don't think so. Musk is threatening to sue for Twitter posts being used without permission for training purposes.
- 3y ago
- _ea1k 3y agoI remember when people used to tell me this about wikipedia. Its just a bunch of random people on the internet editing each other without knowledge. It'll just create a mess and take longer than just looking it up in a highly curated encyclopedia.
- chinchilla2020 3y agoThe different is that wikipedia has true 'inputs' from the real world. Actual knowledge is being ingested to wikipedia from physical reality. Chatbots can only consume what is digitally available. There is an inherent bias in what things are discussed on the internet (you would think our world is a cat-video obsessed world of opinionated political debators). Now most of the content we see on the internet will soon be chatbot-generated content. Chatbots will regurgitate and reconsume their content many times. Wikipedia doesn't have this issue. It is not a closed-loop internet phenomena.
- _ea1k 3y agoChatbots aren't really closed loop either, especially with reinforcement learning. Though it will quickly turn out to be the case that some fields get a lot more attention than others. Which also isn't dissimilar from wikipedia.
- hooverd 3y agoEh, Wikipedia is actually a bunch of highly curated fiefdoms.
- nabla9 3y agoThis is called "model collapse". It leads to worse performance.