4 ms·
Ok, yes, but that's not what model collapse is. I was expecting an interesting finding on LLM training stalling due to AI pollution of the internet (the trainin
by dweinus 1y ago
Ok, yes, but that's not what model collapse is. I was expecting an interesting finding on LLM training stalling due to AI pollution of the internet (the training data). Instead this a rant about the pollution of the internet. All good points and it will lead to model collapse, but this is a clickbait headline.
- deleted 1y ago[deleted]
- daveguy 1y agoBut, the internet is already polluted and being constantly polluted by LLMs. It feels like LLMs are the PFAS of the internet. So, I'm not sure what your argument is, but I would welcome clarification. Maybe I am misunderstanding your definition of model collapse. The article references this definition of model collapse[0]. I'm curious when you say "that's not what model collapse is" -- what do you mean by model collapse that is different from the definition of the article and reference? [0] https://www.nature.com/articles/s41586-024-07566-y https://www.nature.com/articles/s41586-024-07566-y
- dweinus 1y agoMy point is that their rant is not about that definition, even if they reference it. Again, they are not wrong in their concerns at all! It's just that having a PFAS'd internet is not a "sign of model collapse". A sign of model collapse would be new LLMs having degraded performance.
- daveguy 1y ago> A sign of model collapse would be new LLMs having degraded performance. That's a good point. Stagnant performance would be a different thing -- more indicative of the technology itself.