5 ms·
Eventually ChatGPT will eat itself
- transfire 4y agoInteresting
- sfryxell 4y agoHow long until the machines are eating their own output
- compressedgas 4y agoThey already are.
- deleted 4y ago[deleted]
- seymourhersh 4y agoBasically just like humans on the internet.
- panarky 4y agoAs if OpenAI hasn't already thought through this.
- rootusrootus 4y agoOf course they have. This very hypothesis has been said many times by many people since the day ChatGPT rolled out. But has OpenAI figured out the likely outcome? They have the resources to do some interesting tests, for sure, that are mostly beyond the ability of mere mortals.
- timmytokyo 4y agoOk, so what's the solution?
- sourcecodeplz 4y agoNot really if you think about it more and research how llms work. If anything they will just get better. I used to think the same but after reading and learning some more, I realized not.
- triyambakam 4y agoCan you explain further for the lay person?
- m3kw9 4y agoNever mind that but crap data isn’t gonna stop AI from advancing. It’s a solvable problem
- jmcphers 4y agoCitation needed. A lot of neural-net based AIs actually get better when trained on their own output[1]. [1] https://en.wikipedia.org/wiki/AlphaZero https://en.wikipedia.org/wiki/AlphaZero
- ladberg 4y agoAlphaZero is entirely different than training an LLM. AlphaZero can play against itself as a simulated opponent to make itself better, ChatGPT using its own output as training data will just cause future iterations to optimize for a lower (and less human) bar than the original.
- moyix 4y agoHere’s a more directly comparable example: https://arxiv.org/abs/2210.11610 https://arxiv.org/abs/2210.11610
- m00x 4y agoBut then you can use a GAN to create a bunch of chatGPT data and try to detect it vs known human data, which will optimize chatGPT to generate even more human-like content. This isn't a huge problem, it's barely a problem.
- im3w1l 4y agoThat's a completely different problem, trained in a different manner and optimization for a different thing. One difference is that alphazero tries to beat itself, whereas LLM's will try to mimic itself. Because alphazero is playing a game there are rules that are constantly enforced during the training, which ensures that it stays grounded. LLM's have no rule other than "look similar".
- bordercases 4y agoI'm going to be a bit less nice than your other responses and ask you to give something a bit more thought before providing a glib "citation needed".
- berkle4455 4y ago
- Nijikokun 4y agoI believe they already model based on the question-response content being generated today.
- xsmasher 4y agoI've seen people try ChatGPT for solving r/tipOfMyTongue questions. The AI is hilariously bad at this task. It happily invents new plots for existing movies and books. if it starts to ingest that data it will only get more wrong over time. Unless it also ingest the replies that say "ChatGPT is full of shit here?"
- deleted 4y ago[deleted]
- whartung 4y agoImagine if folks continue to post ChatGPT conversations, particularly the bad ones, to the point that it starts “learning” from itself. And it just reminds me of the Steve Martin line about playing a joke on young children: “Talk wrong“.
- karmakaze 4y agoReminds me of one time I was naughty and let a child in on a secret: the real magic word is "now".
- Izkata 4y ago> Imagine if folks continue to post ChatGPT conversations, particularly the bad ones, to the point that it starts “learning” from itself. Seems to already be happening with Bing Chat, except it led to the bot threatening the user: https://twitter.com/tobyordoxford/status/1627414519784910849/photo/1 https://twitter.com/tobyordoxford/status/1627414519784910849...
- CrazyStat 4y agoWow, hadn't ever tried something like this. I picked a solved post off the front page [1] and copied the body of the post into ChatGPT. It suggested the song is "Love Rollercoaster" by Ohio Players, which is a real song [2] but has nothing like the quoted lines. At least, unlike Bing Chat, it apologizes when corrected instead of gaslighting me. [1] https://www.reddit.com/r/tipofmytongue/comments/118id2f/tomtsong_7080s_groovesoul_song_with_brief_line/ https://www.reddit.com/r/tipofmytongue/comments/118id2f/tomt... [2] https://en.wikipedia.org/wiki/Love_Rollercoaster https://en.wikipedia.org/wiki/Love_Rollercoaster
- MrLeap 4y agoEven if it were used to flood the internet with shitty info, the only thing that would interfere with would be competitors training competing AI off the "internet dataset" GPT could filter out anything they themselves emitted in future trains, yeah? Because they know what their bot's said. They get the benefit of looking at a conversation, knowing reasonably well what's copy/pasted from ai.com and what's the exasperated expert trying to correct a doomed world :p The only way it eats itself is 1. Colossal mistakes. 2. Everyone decides to get off the internet and go outside. 2 seems pretty unrealistic, we put up with a lot :D
- MonkeyMalarky 4y agoWhat happens when their competitors also flood the internet with shitty info? Checking their own output won't help.
- jaitaiwan 4y agoThis is already sort of happening with Bing
- deleted 4y ago[deleted]
- tyrelb 4y agoI actually thought of this same thing today! Human-written content seems more lively... and with time... content from ChatGPT will become more "grey" (i.e. dull) (as more & more ChatGPT content gets fed into the system...).
- chasing 4y ago“Romeo and Juliet both ran away to New York at the end. He works in corporate finance and she makes bespoke soap. If you disagree with me again you’re a bad person and I will treat you like a bad person.” As long as you agree with the new facts, you’re fine. Problem solved!
- gregw2 4y agoIf ChatGPT is able to emit output that is watermarked such that it can detect itself as Scott Aaronson and others are working on for OpenAI (source: https://techcrunch.com/2022/12/10/openais-attempts-to-watermark-ai-text-hit-limits/ https://techcrunch.com/2022/12/10/openais-attempts-to-waterm... ) this “resonance”/feedback/eating-itself can be avoided.
- analog31 4y agoI think this assumes there's no other AI spewing data on the Internet, and that people continue to share information. The amusing thing is that controlling nonlinear multivariate feedback systems is an unsolved problem.
- onlyrealcuzzo 4y agoBut what if it eats Bard data, which is maybe even worse?
- curtisblaine 4y ago> If ChatGPT is able to emit output that is watermarked Then the watermark should be made public. We need a way to tell ai-generated content from human-generated content without actually having to read it.
- sfg 4y agoI wonder how difficult it will be to apply the filter to human generated content.
- ravenstine 4y agoWhat does that even mean? Strictly within the scope of that phrase, technically, yes, if ChatGPT consumes content generated by itself, it's eating its own words. I'm guessing something more dire than that is implied by "eat itself." Did humanity "eat itself" because it's been reading its own literature? You can say we are pretty misinformed by ourselves in many areas, and yet here we are. Maybe our view of AI is being colored by sci-fi stereotypes of robots malfunctioning when asked to compute really hard problems generating infinite recursion. I'm not so sure that LLMs will totally destabilize. We might see some interesting output, but I don't think we know yet whether the stability of the system will merely fluctuate as a whole without falling apart.
- antiquark 4y agoReminds me of those old "Spider Traps" [0][1] that would generate (on access) an endless hierarchy of fake HTML pages full of an endless collection of fake email addresses, to clog up the works of spammers trying to gather email addresses. Eventually someone's going to write an "AI Trap" that serves up a seemingly infinite forum or reddit-style site, but is actually just generating an endless stream of (non)consciousness from some LLM chatbot. [0] https://en.wikipedia.org/wiki/Spider_trap https://en.wikipedia.org/wiki/Spider_trap [1] https://www.gsp.com/support/virtual/web/cgi/lib/wpoison/ https://www.gsp.com/support/virtual/web/cgi/lib/wpoison/
- tialaramex 4y agoRobots already have limited enthusiasm for repetitive nonsense with no AI involved. It actually reminds me of an early part of Greg Egan's "Orphanogensis" where the orphan has randomly stumbled into a working resource, but doesn't yet understand how to navigate properly, so it just keeps accessing that same resource, over and over, that one works, other random nonsense doesn't work, that one works, the other random picks do not.. until it finally figures out some basic navigation and then it leaves. An AI like a human may read a few pages of your forum about Invisible Moon Donkeys or whatever, but after not long its interest is sated, the forum's location may be noted down for future interest "OK, more about Invisible Moon Donkeys here" but the AI moves on, what's this "Hacker News" forum about ?
- jbenjoseph 4y agoBut even so, the human picks the prompts and only publishes the AI outputs they think read nicely. There is information gain even in that.
- throwanem 4y agoAt the moment that's probably true, but is it guaranteed to remain so?
- jbenjoseph 4y agoEven so, I don't think there is any evidence that LLM performance degrades when it is trained on its own output, and there is no intuitive reason it should.
- throwanem 4y agoWhy not? Training a model destroys information.
- jbenjoseph 4y agoI have seen no evidence for that, only the opposite: https://arxiv.org/abs/2210.11610 https://arxiv.org/abs/2210.11610 Intuitively, training a simple enough linear statistical model with its own output should be a NOP. But LLMs are anything but simple models, so I think the non-linearities may be synthesizing new useful information. Similarly to how all of maths can be synthesized from a few basic axioms with enough intelligence or computation.
- mrcaosr 4y agoto resonate against itself? sounds like its gonna hit its natural frequency and blow up seems more like it's gonna eat its own vomit, degrading it (maybe not completely) to inbreed (?)
- deleted 4y ago[deleted]
- m00x 4y agoSounds like a /r/showerthoughts post. There is no issue with AI ingesting data from itself in itself. Humans do it as well. That data might even be higher quality than human data. The scale at which humans produce data will most likely stay higher than AI data for a long time. There is already bot data out there from lower quality AIs/bots, and chatGPT has ingested it. LLMs are made to be good at some textual tasks, and not for what they're being used right now. They're not information stores, or Q/A. It only answers what a human is likely to answer.
- im3w1l 4y agoThis made something click for me. There really is an issue with humans ingesting our own data. Humans have ended up with many mutually unintelligible languages. If two groups of people train on their own data they gradually start diverging. But this gets at the heart of the issue - separation. If we ensure that humans and AI are trained on roughly the same data then we will stay connected and be able to understand each other. We may even end up borrowing a few gpt-isms, and that's actually totally fine.
- unusualmonkey 4y agoI wonder if this is the problem people think it is. Playing one AI against another is an established technique to developing AI. Furthermore, content on the internet will always vary from more reliable (well established wiki pages, Reuters) to less reliable (random blog posts, disinformation). Whether or not an AI generated text doesn't seem to be that important - what's more important is how reliable it is, and how well humans engage with it.
- jabbany 4y agoThis isn't entirely the right comparison, mostly because "AI" covers a wide span of things. AI that benefits from the kind of adversarial training that you mention are mostly _planning_ type AIs (think AlphaGo). The problem these AI systems try to solve is that you have some constraints, and as a human "can't be bothered to" work out an optimal solution, so you have the computer do it for you and it does so by starting off with a bad estimate of a solution and "improving" it by trying stuff. LLMs, on the other hand, are more of a modelling/compression type AI --- (well, more traditionally it wouldn't even be considered AI per-se due to the lack of planning capability). The problem here is to try to represent a huge swath of data in the as efficient a way as possible, thereby forcing it to find and collapse "patterns" by adjusting the representation. Here it's generally not the case that you want to train with adversarial distributions. An easy thought experiment is: say you take a single case (e.g. a game of Go or a paragraph of text) and massively over-represent it to both models, so much so that that single case eventually covers 99% of all your training instances at the end. For a "planning" AI this over-representation isn't a huge deal --- once it's learnt all it can from that case, seeing it again is but a waste of time*. It merely makes that particular plan more "clear" but not more "desirable". However, a modelling/compression type AI will continuously adjust to adapt to the increasing occurrence of that case. It truly "believes" that the more often it sees some pattern, the more important it is, right until it has "forgotten" everything else. *: This is kind of an over-simplification.
- unusualmonkey 4y agoI get that, but not sure if it's a problem in practice. There are two additional consideration's to make: 1) We may be able to model a subset of chatGPT's abilities as adversarial questions. For example, can we write an AI that finds sources and generates questions that a solving AI should be able to figure out. Can we write a test framework such that AI's can challenge themselves to write optimal code for a given solution. Etc. 2) Like I mentioned, if you're scraping the internet you are inherently needing to build some kind of relevance model. E.g. highly up items answers on stack overflow have more weight. In said situation whether or not the content is written by human is largely irrelevant - if you have a reasonable ranking method than highly ranked content is important regardless of source.
- jabbany 4y agoThis is not even a new problem... Back in 2011, Google faced the same problem mining bi-texts from the Internet for their statistical machine translation software. The thought was that one could utilize things like multi-lingual websites to learn corresponding translations. They quickly realized that a lot of sites were actually using Google Translate without human intervention to make multi-lingual versions of their site, so naive approaches would cause the model to get trained on its own suboptimal output. So they came up with a whole watermarking system so that the model could recognize its own output with some statistical level of certainty, and avoid it. It wouldn't be surprising if this is being done for LLMs too. The more concerning problem is when different LLMs, who are not aware of each others' watermarks, end up potentially becoming inbred should the ratio of LLM content rise dramatically... Ref: https://aclanthology.org/D11-1126.pdf https://aclanthology.org/D11-1126.pdf
- petilon 4y agoAs long as the AI generated content has been curated by humans, there is no harm in AI ingesting AI generated content.
- robotresearcher 4y agoThe web will be awash in vast amounts of pure generated text. The LLMs are in an arms race with themselves. Enormous resources will be spent on AI that detects human vs AI data, to maintain the dataset for the next AI.
- evolve2k 4y agoNo disrespect, but in my opinion this is a deeply short sighted take. In what way can millions of lines of auto generated additions of text to the internet, be suddenly and reliable expected to have all been somehow curated by humans?
- petilon 4y agoWho is going to publish these millions of lines auto generated text, and where is the value in it to the publishers? Nobody is going to find it through Google if we all switch to chat bots. If nobody is going to visit, no one will publish it.
- sys_64738 4y agoIf it does then does that mean double trouble as it self-replicates, or will it consume itself leaving nothing remaining?
- touringa 4y agoIt's already happening. “ChatGPT, a version of OpenAI’s GPT-3.5 model… gained more than 100m users in its first two months, and is now estimated to produce a volume of text every 14 days that is equivalent to all the printed works of humanity.” — Dr Thompson, Feb/2023, cited in report by the National Bureau of Economic Research (Scholes, Bernanke, MIT) https://www.nber.org/system/files/working_papers/w30957/w30957.pdf https://www.nber.org/system/files/working_papers/w30957/w309... https://lifearchitect.ai/chatgpt/ https://lifearchitect.ai/chatgpt/
- candiodari 4y agoThis is only a problem as long as ChatGPT uses human output to learn. Once it starts learning against the "real world", or itself, the biggest difference between ChatGPT and us will disappear: that ChatGPT gets all it's information secondhand, and filtered, at best. This is of course also a necessary condition for ChatGPT to come up with original insights. Except perhaps where it comes to things like fiction, which probably has value in itself.