3 ms·
> there is going to be a "pre-GPT" internet training set from 2022 Well, yeah, there are several here, and I think all the major image generators are using som
by dragonwriter 3y ago
> there is going to be a "pre-GPT" internet training set from 2022
Well, yeah, there are several here, and I think all the major image generators are using some combination of them as their starting points: https://laion.ai/ https://laion.ai/
> As AI increases as an overall % of all online posts and activity it will death spiral on model quality.
Nope, it will just mean that it will be more expensive to source additional training data on top of the massive trove of existing "clean" (from intentional poisoning) data (much of which isn't perfectly captioned and human work on improving captioning can improve its utility in model training, as can more advanced models with more advanced text encoders, etc.)
If poisoning was widespread, it wouldn't impact "big model" quality much -- they aren't grabbing new random data on the internet for continuous training. It might drive up the expense of community fine tuning, which often does depend on sourcing representative imagery for target styles or concepts from, among other places, the internet.