3 ms·
This is why I don't think model collapse actually matters: people have been deliberately training LLMs on synthetic data for over a year at this point. As far
by simonw 2y ago
This is why I don't think model collapse actually matters: people have been deliberately training LLMs on synthetic data for over a year at this point.
As far as I can tell model collapse happens when you deliberately train LLMs on low quality LLM-generated data so that you can write a paper about it.