3 ms·
The Reddit thread is reasonably inflamed, but the theory about changing reality downstream by changing the sources that chatbots ingest is a chilling one.
by jddj 1y ago
The Reddit thread is reasonably inflamed, but the theory about changing reality downstream by changing the sources that chatbots ingest is a chilling one.
- assword 1y agoLLM’s mark the true beginning of a post truth world. That’s why the serial liars are so excited.
- tim333 1y agoLLMs so far seem fairly good on being factual.
- AlexandrB 1y agoI wonder what portion of the reddit thread is posts by gen AI.
- eqvinox 1y agoChilling, but not very realistic. It's not like chatbots forget all the previously trained-in data when you change the source. In fact, it'd be a pretty hard problem to solve to get there (but actually desirable! - i.e. the ability to remove individual trained-in things.)
- llm_nerd 1y agoNew models are generally trained entirely from scratch, and we're constantly tossing old models into the dumpster and replacing them with the new hotness. Meaning it isn't like GPT-5 is just GPT-4 with some fine tuning, but instead they start an entirely fresh training process on their archive of source material. Now the archive of source material is a growing corpus, but a canonical source like the government's published version of the constitution will indeed get subbed out in an updated crawl, and that would be used as the basis of the training of the next model. Now the site has reverted to the correct text, but the hypothesis of corrupting models is a valid and real concern.