> If feeding AI generated stuff to train an AI is bad then what does that tell us about humans consuming AI generated stuff? It's not unqualified bad. It is ba

> If feeding AI generated stuff to train an AI is bad then what does that tell us about humans consuming AI generated stuff? It's not unqualified bad. It is bad if you do it for many times in a loop, and without any external inputs. But you can put fresh material right in the prompt and it won't suffer from retraining on synthetic data. If you train a small model from scratch on purely synthetic data, like the Microsoft Phi models, it comes out competent and 5x more efficient than models trained on web text. So there's the flip side. You can do it in bad way, and you can do it in a good way.