5 ms·

Computer vision systems are not as robust as human vision systems. There are subtle changes to pixel values that can utterly flummox a computer vision system, b

Computer vision systems are not as robust as human vision systems. There are subtle changes to pixel values that can utterly flummox a computer vision system, but seem undetectable to a human looking at the image.

gooob 3y ago
what is the nature of these pixel changes?
- taylorius 3y ago
  It varies. Specially crafted noise added to the pixel values. Looks like random noise, but obviously isn't. TBH I'm not an expert, but as I understand it, It is "trained" using the vision network, with a loss function that is some combination of being low amplitude, and reducing the strength of the correct image identification.
mysterypie 3y ago
How do subtle pixel changes cause the image of a dog to classified as a cat rather than as gibberish?
- amenhotep 3y ago
  This is a guess (I have read some of the paper but to me it does not seem to explain), but presumably you work backwards from the result you want. There is some vector that comes out of the classifier model that would represent cat, and another that would represent dog; you get the difference between these two models, and work back through the layers of matrix multiplication finding places where small perturbations in the input data force the classifier to make the exact errors you need.
  - dragonwriter 3y ago
    They claim that the poisoning is transferrable across different models, which would not be the case if that was true. OTOH, given this group's track record with their earlier poisoning effort (Glaze) vs. the adulatory press that they managed to arrange, I don't expect much from Nightshade despite the similarly adulatory press. Great media relations game, though.
hackernewds 3y ago
I've read about this https://www.nytimes.com/2023/02/13/technology/ai-art-generator-lensa-stable-diffusion.html https://www.nytimes.com/2023/02/13/technology/ai-art-generat...

gooob 3y ago

what is the nature of these pixel changes?

taylorius 3y ago
It varies. Specially crafted noise added to the pixel values. Looks like random noise, but obviously isn't. TBH I'm not an expert, but as I understand it, It is "trained" using the vision network, with a loss function that is some combination of being low amplitude, and reducing the strength of the correct image identification.

mysterypie 3y ago

How do subtle pixel changes cause the image of a dog to classified as a cat rather than as gibberish?

amenhotep 3y ago
This is a guess (I have read some of the paper but to me it does not seem to explain), but presumably you work backwards from the result you want. There is some vector that comes out of the classifier model that would represent cat, and another that would represent dog; you get the difference between these two models, and work back through the layers of matrix multiplication finding places where small perturbations in the input data force the classifier to make the exact errors you need.
- dragonwriter 3y ago
  They claim that the poisoning is transferrable across different models, which would not be the case if that was true. OTOH, given this group's track record with their earlier poisoning effort (Glaze) vs. the adulatory press that they managed to arrange, I don't expect much from Nightshade despite the similarly adulatory press. Great media relations game, though.

hackernewds 3y ago

I've read about this https://www.nytimes.com/2023/02/13/technology/ai-art-generator-lensa-stable-diffusion.html https://www.nytimes.com/2023/02/13/technology/ai-art-generat...