3 ms·
The dangers of "model collapse" are wildly overstated. Sure, if you feed the unfiltered output of an LLM into a new LLM, inbreeding will eventually collapse the
by threeducks 2y ago
The dangers of "model collapse" are wildly overstated. Sure, if you feed the unfiltered output of an LLM into a new LLM, inbreeding will eventually collapse the model, but if you filter the data with some grounding in reality, the results will get better and better. The best example is probably AlphaGo Zero, which was grounded with the rules of the game and did not even receive any supervised data. For programming, grounding can happen by simply executing the code. But even if you do not have grounding, you can still just throw a lot of test time compute at the output to filter out garbage, which is probably good enough.
- wruza 2y agoSame for stable diffusion (being the same tech). I trained models purely on model outputs and didn’t see any horrific handwavy things everyone’s talked about. You can’t get more of what’s already there, obviously. But as a way to bake some prompt / control pic in, it works great. I actually find/feel this easier to achieve the same result, because I can vary the output with dynamic prompting and seeds, models, etc, while keeping the desired concept intact, with a little picking. This helps to better distill it into a model. Natural imagery is usually too diverse and requires lots of effort to dissect into the parts you want.