9 ms·
Eventually, OpenAI (and friends) are going to be training their models on almost exclusively AI generated content, which is more often than not slightly incorre
by ezekg 3y ago
Eventually, OpenAI (and friends) are going to be training their models on almost exclusively AI generated content, which is more often than not slightly incorrect when it comes to Q&A, and the quality of AI responses trained on that content will quickly deteriorate. Right now, most internet content is written by humans. But in 5 years? Not so much. I think this is one of the big problems that the AI space needs to solve quickly. Garbage in, garbage out, as the old saying goes.
- bevekspldnw 3y agoThe end state of training on web text has always been an ouroboros - primarily because of adtech incentives to produce low quality content at scale to capture micro pennies. The irony of the whole thing is brutal.
- ezekg 3y ago> The end state of training on web text has always been an ouroboros And when other mediums have been saturated with AI? Books, music, radio, podcasts, movies -- what then? Do we need a (curated?) unadulterated stockpile of human content to avoid the enshittification of everything?
- weregiraffe 3y ago>Do we need a (curated?) unadulterated stockpile of human content to avoid the enshittification of everything? Either that, or a human level AI.
- ethanbond 3y agoWell no, we need billions of human-level AIs who are experiencing a world as rich and various as the world that the billions of humans inhabit.
- ben_w 3y agoOnce we've got the first, making a billion is easy. That said… are content creators collectively (all media, film and books as well as web) a thin tail or a fat tail? I could easily believe most of the actual culture comes from 10k-100k people today, even if there's, IDK, ten million YouTubers or something (I have a YouTube channel, something like 14 k views over 14 years, this isn't "culturally relevant" scale, and even if it had been most of those views are for algorithmically generated music from 2010 that's a literal Markov chain).
- xipho 3y agoMaybe we could print out knowledge on dead trees and store them somewhere, perhaps make it publically available? (stolen joke, not mine).
- berniedurfee 3y agoYahoo.com will rise from the ashes.
- bevekspldnw 3y agoI mean, you’re not wrong. I’ve been building some unrelated web search tech and have considered just indexing all the sites I can about and making my own “non shit” search engine. Which really isn’t too hard if you want to do say, 10-50 sites. You can fit that on one 4TB nvme drive on a local workstation. I’m trying to work on monetization for my product now. The “personal Google” idea is really just an accidental byproduct of solving a much harder task. Not sure if people would pay for that alone.
- nicolas_17 3y agohttps://en.wikipedia.org/wiki/Low_background_steel https://en.wikipedia.org/wiki/Low_background_steel but for web content.
- oceanplexian 3y agoContent you’re allowed and capable of scraping on the Internet is such a small amount of data, not sure why people are acting otherwise. Common crawl alone is only a few hundred TB, I have more content than that on a NAS sitting in my office that I built for a few grand (Granted I’m a bit of a data hoarder). The fears that we have “used all the data” are incredibly unfounded.
- Eisenstein 3y ago> Facebook alone probably has more data than the entire dataset GPT4 was trained on and it’s all behind closed doors. Meta is happily training their own models with this data, so it isn't going to waste.
- bevekspldnw 3y agoNot Llama, they’ve been really clear about that. Especially with DMA cross-joining provisions and various privacy requirements it’s really hard for them, same for Google. However, Microsoft has been flying under the radar. If they gave all Hotmail and O365 data to OpenAI I’d not be surprised in the slightest.
- troq13 3y agoThe company that made a honeypot VPN to access competitor's traffic? They are definitively keeping their hands off their internal data, yes.
- deleted 3y ago[deleted]
- Dr_Birdbrain 3y agoI bet they are training their internal models on the data. Bet the real reason they are not training open source models on that data is because of fears of knowledge distillation, somebody else could distill LLaMa into other models. Once the data is in one AI, it can be in any AIs. This problem is of course exacerbated by open source models, but even closed models are not immune, as the Alpaca paper showed.
- sarah_eu 3y agoWell it will be multimodal, training and inferring on feeds of distributed sensing networks; radio, optical, acoustic, accelerometer, vibration, anything that's in your phone and much besides. I think the time of the text-only transformer has already passed.
- pants2 3y agoOpenAI will just litter microphones around public spaces to record conversations and train on them.
- sarah_eu 3y agoHas been happening for at least 10 years.
- khalladay 3y agoGot a source for that?
- deleted 3y ago[deleted]
- oceanplexian 3y agoWant a real conspiracy? What do you think the NSA is storing in that datacenter in Utah? Power point presentations? All that data is going to be trained into large models. Every phone call you ever had and every email you ever wrote. They are likely pumping enormous money into it as we speak, probably with the help of OpenAI, Microsoft and friends.
- berniedurfee 3y agoIt would be absolutely fascinating to talk to the LLMs of the various government spy agencies around the world.
- jayd16 3y agoIt's true that there will no longer be any virgin forest to scrape but it's also true that content humans want will still be most popular and promoted and curated and edited etc etc. Even if it's impossible to train on organic content it'll still be possible to get good content
- RogerL 3y agoIs it (I am not a worker in this space, so genuine question)? My thoughts - I teach myself all the time. Self reflection with a loss function can lead to better results. Why can't the LLMs do the same (I grasp that they may not be programmed that way currently)? Top engines already do it with chess, go, etc. They exceed human abilities without human gameplay. To me that seems like the obvious and perhaps only route to general intelligence. We as humans can recognize botnets. Why wouldn't the LLM? Sort of in a hierarchal boost - learn the language, learn about bots and botnets (by reading things like this discussion), learn to identify them, learn that their content doesn't help the loss function much, etc. I mean sure, if the main input is "as a language model I cannot..." and that is treated as 'gospel' that would lead to a poor LLM, but i don't think that is the future. LLMs are interacting with humans - how many times do they have to re-ask a question - that should be part of the learning/loss function. How often do they copy the text into their clipboard (weak evidence that the reply was good)? do you see that text in the wild, showing it was used? If so, in what context "Witness this horrible output of chatGPT: <blah>" should result in lower scores and suppression of that kind of thing. I dream of the day where I have a local LLM (ie individualized, I don't care where the hardware is) as a filter on my internet. Never see a botnet again, or a stack overflow q/a that is just "this has already been answered" (just show me where it was answered), rewrite things to fix grammar, etc. We already have that with automatic translation of languages in your browser, but now we have the tools for something more intelligent than that. That sort of thing. Of course there will be an arms race, but in one sense who cares. If a bot is entirely indistinguishable from a person, is that a difference that matters? I can think of scenerios where the answer is an emphatic YES, but overall it seems like a net improvement.
- bogwog 3y ago> Eventually, OpenAI (and friends) are going to be training their models on almost exclusively AI generated content What makes you think this is true? Yes, it's likely that the internet will have more AI generated content than real content eventually (if it hasn't happened already), but why do you think AI companies won't realize this and adjust their training methods?
- loloquwowndueo 3y agoMany AI content detectors have been retired because they are unreliable - AI can’t consistently identify AI-generated content. How would they adjust then?
- mlboss 3y agoThe only way out of this is robots that can go out in the world and collect data. Write in natural language what they observed which can then be used to train better LLMs.
- Salgat 3y agoAs long as humans continue to filter out the bad content generated by AI, it should be fine.
- eightysixfour 3y agoIt is already solved. Look at how Microsoft trained Phi - they used existing models to generate synthetic data from textbooks. That allowed them to create a new dataset grounded in “fact” at a far higher quality than common crawl or others. It looks less like an ouroboros and more like a bootstrapping problem.
- mattc0m 3y agoAI training on AI-generated content is a future problem. Using textbooks is a good idea, until our textbooks are being written by AI. This problem can't really be avoided once we begin using AI to write, understand, explain, and disseminate information for us. It'll be writing more than blogs and SEO pages. How long before we start readily using AI to write academic journals and scientific papers? It's really only a matter of time, if it's not already happening.
- eightysixfour 3y agoYou need to separate “content” and “knowledge.” GenAI can create massive amounts of content, but the knowledge you give it to create that content is what matters and why RAG is the most important pattern right now. From “known good” sources of knowledge, we can generate an infinite amount of content. We can add more “known good” knowledge to the model by generating content about that knowledge and training on it. I agree there will be many issues keeping up with what “known good” is, but that’s always been an issue.
- ezekg 3y ago> We can add more “known good” knowledge to the model by generating content about that knowledge and training on it. That's my entire point -- AI only generates content right now, but it will also be the source of content for training purposes soon. We need a "known good" human knowledge-base, otherwise generative AI will degenerate as AI generated content proliferates. Crawling the web, like in the case of the OP, isn't going to work for much longer. And books, video, and music are next.
- TaylorAlexander 3y agoI really really hope that five years from now we are not still using AI systems that behave the way today's do, based on probabilistic amalgamations of the whole of the internet. I hope we have designed systems that can reason about what they are learning and build reasonable mental models about what information is valuable and what can be discarded.
- SrslyJosh 3y agoEveryone saying "ouroboros": The phrase you're looking for is "human centipede". =)
- mkl 3y agoNo, the point is it's a circle, with the output feeding back into the input.
- atleastoptimal 3y agoThey've obviously been thinking about this for a while and are well aware of the pitfalls of training on AI based content. This is why they're making such aggressive moves into video, audio, other better and more robust ground forms of truth. Do you really think that they aren't aware of this issue? It's funny whenever people bring this up, they think AI companies are some mindless juggernauts who will simply train without caring about data quality at all and end up with worse models that they'll still for some reason release. Don't people realize that attention to data quality is the core differentiating feature that lead companies like OpenAI to their market dominance in the first place?
- FridgeSeal 3y agoI for one, welcome the junk-data-ouroboros-meta-model-collapse. I think it’ll force us out of this local maxima of “moar data moar good” mindset, and give us collectively, a chance to evaluate the effect these things have our society. Some proverbial breathing room.