4 ms·
Out of curiosity, what happens to an LLM if, over time, it's increasingly trained on its own output and/or that of other LLMs? And has anyone figured out a goo
by CoastalCoder 3y ago
Out of curiosity, what happens to an LLM if, over time, it's increasingly trained on its own output and/or that of other LLMs?
And has anyone figured out a good way to minimize that when sourcing training data from e.g. Reddit?
- hospadar 3y agoI certainly have no idea but interesting to think about! One could argue that humans are trained on their own output so presumably interesting things could arise, especially if it's not a totally closed loop (LLM#1 -> some human curation -> LLM#2).
- fladrif 3y ago> humans are trained on their own output And it seems we're emulating that same cycle. Humans are trained not only on our own output, but also take changes from our environment. So in this sense LLM's environment would be human input.
- cyanydeez 3y agothe problem with the current tech is that the solid part of knowledge isn't separate from the fluid parts. You need to fundamentally understand that human knowledge in many generations was purposely fixed, by books, scholars, etc. Until that occurs, it's really just a miasma of bullshit.
- pjmorris 3y agoI read (probably on HN) where someone suggested that pre-LLM data will become the 'low background steel' (pre-Nuclear-age steel with lower levels of internal radiation) of machine learning. It seems like provenance/traceability of data sources will become important, both for the builders and users of LLMs.
- verytrivial 3y agoThere's a no-longer-very-speculative bit of Sci-Fi waiting here about whole societies "forking" based upon the curation of their training data. Arguably social-media + bots + paid state actors means we're already there.
- mrbungie 3y agoIsn't "societies forking based upon curation of their training data" practically what culture is? Now, I agree that having scalable generative AI and high-throughput-high-virality mass media (social networks) brings new unspeakeable horrors like feedback loops to the mix. Interesting times indeed.
- cyanydeez 3y agocan you point to an actual paid state actor. the term seems like mostly bullshit.
- deleted 3y ago[deleted]
- xwdv 3y agoContext drift.
- jmount 3y agohttps://news.ycombinator.com/item?id=36319076 https://news.ycombinator.com/item?id=36319076
- spacemanspiff01 3y agoI believe that performance degrades, as the models end up training on data that is more homogenized and less diverse when compared to original data. At least according to this paper (I think, FYI I am not a expert) https://arxiv.org/abs/2305.17493 https://arxiv.org/abs/2305.17493
- jerpint 3y agoIt can reinforce its own biases for one
- MengerSponge 3y agoHave you seen Multiplicity? "Hey Steve, come on up, I'm spittin on bugs"
- gnicholas 3y agoHm, sounds like dog-fooding, but where the 'food' is dog shit. Dog-shitting? Dog-shit-fooding?
- gpderetta 3y agoDog breakfasting
- lordnacho 3y agodogging
- TheSpiceIsLife 3y agoDogenshittification
- jstarfish 3y agoThe Poopboros.
- Daishiman 3y agoAI Centipede
- asow92 3y agoHopefully some of this is democratized away by humans voting on quality of output directly and/or indirectly?
- lozaning 3y agoHabsburg AI – a system that is so heavily trained on the outputs of other generative AI's that it becomes an inbred mutant, likely with exaggerated, grotesque features from https://twitter.com/jathansadowski/status/1625245803211272194 https://twitter.com/jathansadowski/status/162524580321127219...
- samstave 3y agoOr even, the Zuckerberg of AI in a meta-sorta-way'' This was pretty clever: >CaligulAI <-- @Rogntuudju
- CoastalCoder 3y ago> CaligulAI I believe it's spelled "Caligulae" because it's first declension. /Latin-joke
- samstave 3y agoLOVE it You know what would be funny ; ."Latin Joke Explainer" -- As explained in Latin Man-Splain-Terms."* EDIT: (You mastered my joke, and I appreciate it.) EDIT AGAIN: UI think we actually just coined a term ; 'Caligul::AI' -- Malicious AI for its own pleasure.
- Mistletoe 3y agoYou’ve heard it at a concert when the microphone picks up sound from the speakers and a positive feedback loop ensues. A lot of screeching.
- veave 3y agoLately I have been trying a new LLM called Falcon that was supposedly created in the UAE but it was mostly trained on conversations with ChatGPT... the result is that if you ask Falcon who created it, it will happily say OpenAI. I found that really amusing.
- whimsicalism 3y agoThe instruct trained variation, you mean.
- Aerbil313 3y agoI read a research paper which says LLMs don’t actually improve by training on the outputs of other LLMs. They only gain the ability to answer the specific questions included in the LLM output training dataset. Their reasoning capabilities etc. don’t improve.
- Aerbil313 3y agoI’m probably wrong. See the Orca model which came out recently.
- cj 3y agoPerhaps the source training data will shift to transcripts or subtitles of podcasts, cable television, TV shows, and other sources that haven't (yet) been polluted. Combined with pre-LLM data sets.
- avereveard 3y agoPossibly such data will be used in a different way. If it's embedded in a social network, engagement (votes, retweets,wwillhstecer) will act as a sort of crowd sourced reinforcement learning source instead of being direct part of the training set.
- Balooga 3y ago"Model Collapse" https://venturebeat.com/ai/the-ai-feedback-loop-researchers-warn-of-model-collapse-as-ai-trains-on-ai-generated-content/ https://venturebeat.com/ai/the-ai-feedback-loop-researchers-...
- the8472 3y agoTraining is becoming multi-modal anyway and will need grounding in physical reality so human-generated text will most likely become a smaller fraction of the data.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]