This is one of those things which is plausible sounding but probably untrue. We are generating exponentially more data with every passing year, only a small fra

This is one of those things which is plausible sounding but probably untrue. We are generating exponentially more data with every passing year, only a small fraction of which is going to be AI-generated. Consider: - you could put a microphone in a busy public place and do speech to text on everything you heard - you could train on audio data from films etc. Yes some (tiny if the WGA and others have their way) fraction of the script will be AI generated but a lot won't be, actors will improv etc. - There will still be a massive and ever-expanding amount of human-generated text to train on. For example we have the fast-growing niches of AI doomsterism and AI mindless optimism. - You could hook it up to gemini to find out what the hipsters are thinking - public domain books are a massive training corpus and every year a new vintage gets added. It's going to be quite some years before the AI-generated garbage tier gets through the copyright period and out into public domain. - In certain cases AI can actually help to train AI models. Obviously we saw this in the reinforcement learning explosion around AlphaZero etc but we're starting to see it in language models with things like voicebox[1] being used to train other models, and Facebook's paper shows significant model improvements from training this way. [1] https://about.fb.com/news/2023/06/introducing-voicebox-ai-for-speech-generation/ https://about.fb.com/news/2023/06/introducing-voicebox-ai-fo...