3 ms·
"Model collapse" is often overstated, as this paper demonstrates: https://arxiv.org/pdf/2404.01413 https://arxiv.org/pdf/2404.01413 The original model collapse
by ainch 1mo ago
"Model collapse" is often overstated, as this paper demonstrates: https://arxiv.org/pdf/2404.01413 https://arxiv.org/pdf/2404.01413
The original model collapse paper assumes you train networks on 100% synthetic data produced by the previous generation. But if you maintain some portion of real data then the problem is mitigated.
- lazide 1mo agoNot when you also including poisoning attacks. I remember the original paper showing issues with even a couple percent of certain kinds of synthetic data too, not 100%.