5 ms·

One thing that isn't captured by this article's analogy, and that is a flaw in the study: new LLMs can train on the results of multiple different models, not ju

One thing that isn't captured by this article's analogy, and that is a flaw in the study: new LLMs can train on the results of multiple different models, not just their direct predecessor. If you had the same image processed by a large variety of different compression algorithms, you might find you are able to fairly accurately infer the original pixels. The entropy is drastically reduced. If there were many different models being trained and used widely it would, at minimum help mitigate this issue. Also, having multimodal models will likely change the balance. If models can train directly on "real world" data that can help fill in the entropy gaps.

SketchySeaBeast 3y ago
It seems like that would require a semi-deliberate "breeding" program or a guaranteed wide diversity of models. At the moment there doesn't seem to be a large enough pool of high quality models. The internet is going to grow to be full of the content of a small number of proficient LLMs. Given that this content isn't being flagged as generated it guarantees the few models will train on their own output, or the output of other models who trained on their own output. Incestuous learning is pretty much guaranteed unless generated content starts being flagged or there is an explosion of entirely novel models.
- dwallin 3y ago
  Yeah, it's definitely not a guarantee but there are already viable paths out of the mess. I just wanted to push back against the notion that it's somehow inevitable or a forgone conclusion that it will happen. Personally, I'm hoping we will see a Cambrian explosion of new LLM models and approaches. We've seen some beginnings of this in image generation so it's not entirely implausible. Another thing the study doesn't capture: What is the effect of combined human + AI content? It's plausible that an explosion (due to lowered barriers) of new human guided/augmented ai content could counteract the effect.

SketchySeaBeast 3y ago

It seems like that would require a semi-deliberate "breeding" program or a guaranteed wide diversity of models. At the moment there doesn't seem to be a large enough pool of high quality models. The internet is going to grow to be full of the content of a small number of proficient LLMs. Given that this content isn't being flagged as generated it guarantees the few models will train on their own output, or the output of other models who trained on their own output. Incestuous learning is pretty much guaranteed unless generated content starts being flagged or there is an explosion of entirely novel models.

dwallin 3y ago
Yeah, it's definitely not a guarantee but there are already viable paths out of the mess. I just wanted to push back against the notion that it's somehow inevitable or a forgone conclusion that it will happen. Personally, I'm hoping we will see a Cambrian explosion of new LLM models and approaches. We've seen some beginnings of this in image generation so it's not entirely implausible. Another thing the study doesn't capture: What is the effect of combined human + AI content? It's plausible that an explosion (due to lowered barriers) of new human guided/augmented ai content could counteract the effect.