Karpathy was asked on a recent podcast, "how do humans get around model collapse?" Here's what he said: “These analogies are surprisingly good. Humans collapse

Karpathy was asked on a recent podcast, "how do humans get around model collapse?" Here's what he said: “These analogies are surprisingly good. Humans collapse during the course of their lives. Children haven't overfit yet. They will say stuff that will shock you. Because they're not yet collapsed. But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.”