3 ms·

What the article doesn't illustrate is that it destroys fine detail in the image, even in the thumbnails of the reference paper: https://arxiv.org/pdf/2310.1382

What the article doesn't illustrate is that it destroys fine detail in the image, even in the thumbnails of the reference paper: https://arxiv.org/pdf/2310.13828.pdf https://arxiv.org/pdf/2310.13828.pdf Also... Maybe I am naive, but it seems rather trivial to work around with a quick prefilter? I don't know if tradition denoising would be enough, but worst case you could run img2img diffusion. reply

GaryNumanVevo 3y ago
The poisoned images aren't intended to be viewed, rather scraped and pass a basic human screen. You wouldn't be able to denoise as you'd have to denoise the entire dataset, the entire point is that these are virtually undetectable from typical training set examples, but they can push prompt frequencies around at will with a small number of poisoned examples.
- minimaxir 3y ago
  > You wouldn't be able to denoise as you'd have to denoise the entire dataset Doing that requires much less compute than training a large generative image model.
  - GaryNumanVevo 3y ago
    > the entire point is that these are virtually undetectable from typical training set examples I'll repeat this point for clarity. After going over the paper again, denoising shouldn't affect this attack, it's the ability of plausible images to not be detected by human or AI discriminators (yet)
  - brucethemoose2 3y ago
    I guess the idea is that the model trainers are ignorant of this and wouldn't know to preprocess/wouldn't bother? That's actually quite plausible.
    - BugsJustFindMe 3y ago
      > I guess the idea is that the model trainers are ignorant of this Maybe they're ignorant of it right up until you announce it, but then they're no longer ignorant of it.
      - brucethemoose2 3y ago
        Right, but they aren't necessarily paying attention to this. I am not trying to belittle foundational model trainers, but a lot goes on in ML land. Even groups can't track every development.

GaryNumanVevo 3y ago

The poisoned images aren't intended to be viewed, rather scraped and pass a basic human screen. You wouldn't be able to denoise as you'd have to denoise the entire dataset, the entire point is that these are virtually undetectable from typical training set examples, but they can push prompt frequencies around at will with a small number of poisoned examples.

minimaxir 3y ago
> You wouldn't be able to denoise as you'd have to denoise the entire dataset Doing that requires much less compute than training a large generative image model.
- GaryNumanVevo 3y ago
  > the entire point is that these are virtually undetectable from typical training set examples I'll repeat this point for clarity. After going over the paper again, denoising shouldn't affect this attack, it's the ability of plausible images to not be detected by human or AI discriminators (yet)
- brucethemoose2 3y ago
  I guess the idea is that the model trainers are ignorant of this and wouldn't know to preprocess/wouldn't bother? That's actually quite plausible.
  - BugsJustFindMe 3y ago
    > I guess the idea is that the model trainers are ignorant of this Maybe they're ignorant of it right up until you announce it, but then they're no longer ignorant of it.
    - brucethemoose2 3y ago
      Right, but they aren't necessarily paying attention to this. I am not trying to belittle foundational model trainers, but a lot goes on in ML land. Even groups can't track every development.