4 ms·
> altering an image’s pixels in a way that wreaks havoc to computer vision but leaves the image unaltered to a human’s eyes Can someone who understands the ori
by mysterypie 3y ago
> altering an image’s pixels in a way that wreaks havoc to computer vision but leaves the image unaltered to a human’s eyes
Can someone who understands the original paper give an ELI5 on how that's possible?
I understand how labeling an image of a cat as "dog" could poison a dataset.
I also understand how adding images of a toaster (which sort of looks like a handbag) to a dataset of handbags could poison a dataset.
But I don't understand what's happening in Figure 6 in the original paper. The pairs of pictures of the dogs, cars, etc. look absolutely identical. What exactly is happening there?
- taylorius 3y agoComputer vision systems are not as robust as human vision systems. There are subtle changes to pixel values that can utterly flummox a computer vision system, but seem undetectable to a human looking at the image.
- gooob 3y agowhat is the nature of these pixel changes?
- taylorius 3y agoIt varies. Specially crafted noise added to the pixel values. Looks like random noise, but obviously isn't. TBH I'm not an expert, but as I understand it, It is "trained" using the vision network, with a loss function that is some combination of being low amplitude, and reducing the strength of the correct image identification.
- mysterypie 3y agoHow do subtle pixel changes cause the image of a dog to classified as a cat rather than as gibberish?
- amenhotep 3y agoThis is a guess (I have read some of the paper but to me it does not seem to explain), but presumably you work backwards from the result you want. There is some vector that comes out of the classifier model that would represent cat, and another that would represent dog; you get the difference between these two models, and work back through the layers of matrix multiplication finding places where small perturbations in the input data force the classifier to make the exact errors you need.
- dragonwriter 3y agoThey claim that the poisoning is transferrable across different models, which would not be the case if that was true. OTOH, given this group's track record with their earlier poisoning effort (Glaze) vs. the adulatory press that they managed to arrange, I don't expect much from Nightshade despite the similarly adulatory press. Great media relations game, though.
- hackernewds 3y agoI've read about this https://www.nytimes.com/2023/02/13/technology/ai-art-generator-lensa-stable-diffusion.html https://www.nytimes.com/2023/02/13/technology/ai-art-generat...