3 ms·

there is going to be a "pre-GPT" internet training set from 2022 and earlier that will basically be the last 'complete' not poisoned data set. As AI increases

there is going to be a "pre-GPT" internet training set from 2022 and earlier that will basically be the last 'complete' not poisoned data set. As AI increases as an overall % of all online posts and activity it will death spiral on model quality.

pupileater 3y ago
Haha, like https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel
- tomrod 3y ago
  Not too far off! AI retraining on its own output is not a great thing.
  - GaggiX 3y ago
    ChatGPT trains on its own output by exploring its own sampling space using reinforcement learning, Dalle 3 was trained on a dataset of synthetic captions, they are both SOTA, you can use AI to train AI models.
  - danielbln 3y ago
    IT can be, though. High quality synthetic data for training is a thing, see Phi-2 [0] (and many more examples, people having been using GPT-4 outputs for finetuning a LOT to great avail). [0] https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/ https://www.microsoft.com/en-us/research/blog/phi-2-the-surp...
  - dragonwriter 3y ago
    > AI retraining on its own output is not a great thing. For most forms of GenAI, including image generators, using existing models to generate synthetic data (and then doing human labelling, etc., as appropriate) for fine tuning is quite common and important. Why do you think its not a great thing?
sjfjsjdjwvwvc 3y ago
I disagree. There is still plenty of genuine data being produced. Also in the case of images, most of them have digital watermarks. I see no reason to believe there will be a death spiral on model quality - in the last year or so the opposite has been happening. Of course the jury is still out on the future.
- DaiPlusPlus 3y ago
  > Also in the case of images, most of them have digital watermarks. Do they now?
  - sjfjsjdjwvwvc 3y ago
    All the images you create with Stable Diffusion include the generation data in the PNG by default. Of course you can remove it.
    - dragonwriter 3y ago
      Moreover, anything created with StabilityAI's code in its default settings also includes an invisible watermark in the image data, not just the generation information in the metadata.
amelius 3y ago
Only if we're too lazy to pre-filter the content manually.
dragonwriter 3y ago
> there is going to be a "pre-GPT" internet training set from 2022 Well, yeah, there are several here, and I think all the major image generators are using some combination of them as their starting points: https://laion.ai/ https://laion.ai/ > As AI increases as an overall % of all online posts and activity it will death spiral on model quality. Nope, it will just mean that it will be more expensive to source additional training data on top of the massive trove of existing "clean" (from intentional poisoning) data (much of which isn't perfectly captioned and human work on improving captioning can improve its utility in model training, as can more advanced models with more advanced text encoders, etc.) If poisoning was widespread, it wouldn't impact "big model" quality much -- they aren't grabbing new random data on the internet for continuous training. It might drive up the expense of community fine tuning, which often does depend on sourcing representative imagery for target styles or concepts from, among other places, the internet.

pupileater 3y ago

Haha, like https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel

tomrod 3y ago
Not too far off! AI retraining on its own output is not a great thing.
- GaggiX 3y ago
  ChatGPT trains on its own output by exploring its own sampling space using reinforcement learning, Dalle 3 was trained on a dataset of synthetic captions, they are both SOTA, you can use AI to train AI models.
- danielbln 3y ago
  IT can be, though. High quality synthetic data for training is a thing, see Phi-2 [0] (and many more examples, people having been using GPT-4 outputs for finetuning a LOT to great avail). [0] https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/ https://www.microsoft.com/en-us/research/blog/phi-2-the-surp...
- dragonwriter 3y ago
  > AI retraining on its own output is not a great thing. For most forms of GenAI, including image generators, using existing models to generate synthetic data (and then doing human labelling, etc., as appropriate) for fine tuning is quite common and important. Why do you think its not a great thing?

sjfjsjdjwvwvc 3y ago

I disagree. There is still plenty of genuine data being produced. Also in the case of images, most of them have digital watermarks. I see no reason to believe there will be a death spiral on model quality - in the last year or so the opposite has been happening. Of course the jury is still out on the future.

DaiPlusPlus 3y ago
> Also in the case of images, most of them have digital watermarks. Do they now?
- sjfjsjdjwvwvc 3y ago
  All the images you create with Stable Diffusion include the generation data in the PNG by default. Of course you can remove it.
  - dragonwriter 3y ago
    Moreover, anything created with StabilityAI's code in its default settings also includes an invisible watermark in the image data, not just the generation information in the metadata.

amelius 3y ago

Only if we're too lazy to pre-filter the content manually.

dragonwriter 3y ago

> there is going to be a "pre-GPT" internet training set from 2022 Well, yeah, there are several here, and I think all the major image generators are using some combination of them as their starting points: https://laion.ai/ https://laion.ai/ > As AI increases as an overall % of all online posts and activity it will death spiral on model quality. Nope, it will just mean that it will be more expensive to source additional training data on top of the massive trove of existing "clean" (from intentional poisoning) data (much of which isn't perfectly captioned and human work on improving captioning can improve its utility in model training, as can more advanced models with more advanced text encoders, etc.) If poisoning was widespread, it wouldn't impact "big model" quality much -- they aren't grabbing new random data on the internet for continuous training. It might drive up the expense of community fine tuning, which often does depend on sourcing representative imagery for target styles or concepts from, among other places, the internet.