3 ms·
> But my understanding of what they’re actually doing is a data poisoning attack on the fine-tuning process (e.g. someone tries to finetune StableDiffusion on a
by dragonwriter 4y ago
> But my understanding of what they’re actually doing is a data poisoning attack on the fine-tuning process (e.g. someone tries to finetune StableDiffusion on a handful of a particular artist’s images).
No, its poisoning core model training as well, because the core models are trained with scraped art with artist and other metadata, which is why you can use “in the style of…” prompts for lots of artists with the core models. It also applies to fine tuning, etc.
- Imnimo 4y agoIt is absolutely true that the Glazed images will be scooped up by a fresh webscrape for training data for a new model. But there isn't any evidence that this would provide any actual defense. Their paper only studies the fine-tuning scenario. It seems to me that if you train your text-to-image system from scratch on Glazed images, Glaze has lost its upper-hand. You'd essentially be performing adversarial training but with a fixed adversary! At the very least, I'd want to see some actual experiments on training from scratch before telling artists that Glaze will protect them in that scenario. And I'm very skeptical that it would.
- dragonwriter 4y agoI hadn’t really thought about that. If it doesn’t work against people training the base models, and those are inevitably going to be trained on a wider and wider set of internet-available imagery, it seems like this is even more futile.
- sebzim4500 4y agoWouldn't having a small portion of adverserially modified images in your training set improve the robustness of the model? It's a known technique to do this kind of thing intentionally to train models that are more resistant to adverserial attacks. One reason people don't do this is the cost of running PGD is so high, but in this case your adversaries are doing it for you free of charge.