5 ms·
This idea that LLMs will get progressively dumber because of a training feedback loop is silly. If this ever did become become a genuine concern companies trai
by kypro 3y ago
This idea that LLMs will get progressively dumber because of a training feedback loop is silly.
If this ever did become become a genuine concern companies training LLMs would just use preexisting datasets and selectively add additional data that they can verify the quality of.
Not everything is Reddit. I doubt the NYT is going to start publishing BS. Scientific journals like Nature also aren't going to start publishing BS because of LLMs.
There will be plenty of great resources to train future iterations of LLMs, even if the days of just scraping the entire internet for a dataset has passed.
- kneebonian 3y ago> I doubt the NYT is going to start publishing BS. Scientific journals like Nature also aren't going to start publishing BS because of LLMs. Will they though? Leaving aside the NYT which I contend already publishes quite a bit of bull hockey. Here's how it goes. Researcher: Provides outline of paper, GPT turn this into a scientific paper for me. GPT: Turns it into a paper, but includes several not quite accurate statements, or hallucinates some information and facts, as well as fabricating just one or two citations. Researcher: Gives it a once over, not a through review because that would defeat the point of using an LLM to generate the content in the first place. Paper gets submitted. Reviewer: Receives paper, throws it into GPT, "summarize this for me", GPT spits out a summary that looks mostly good, the reviewer skims over the actual paper, there are no glaring problems it gets published. End result. We now have a paper with fabricated information stated as fact published in a reliable place, that then is cited and referenced, resulting in a degradation of quality of the information. This repeats and repeats and suddenly whole conferences are built on entirely hallucinated information.
- jqpabc123 3y agoIf this ever did become become a genuine concern companies training LLMs would just use preexisting datasets So they will be cutoff from the internet and never consume new data from it? Is this what they're doing now? I don't think so. Musk is threatening to sue for Twitter posts being used without permission for training purposes.
- nh23423fefe 3y agoSeems like an incorrect model for what LLMs are. Do you imagine you can't learn english if you only read false statements?
- tedunangst 3y agoOf what use is a text generator that only makes false statements?
- lisasays 3y agoThis idea that LLMs will get progressively dumber because of a training feedback loop is silly. It's also a straw man. The concern isn't so much that that theses systems will move backwards, but rather that they will start saturating, and gradually degrade in terms of ROI (measured in terms of both cost/energy and the complexity of building of dealing with systems to deal with all poisoned data). The commenter above was stretching a bit when they said "collapse", but I think they got the basic gist of the issue right. Which is basically: diminishing returns.
- sanderjd 3y agoYes, companies training LLMs would not let this degradation of quality happen. But that's not the question this point is answering. It's an answer to the thought experiment of why LLMs won't be able to just become smarter and smarter by talking to themselves. The theory is that there is some point past which that will make them dumber rather than smarter, so it won't be done. We don't know for sure yet, but I personally find this theory compelling.
- worrycue 3y ago> This idea that LLMs will get progressively dumber because of a training feedback loop is silly. What's interesting is that it can get dumber due to consuming its own or other LLM's output. We humans read our own output and talk to other humans. Never heard of anyone getting dumber from being social.