3 ms·

I think she means you can add white noise to the front end making the output a little bit non-deterministic. Since it won't give the same confidence number twic

• antognini 10y ago
  Just adding white noise won't work. If you add white noise to an image the NN will make almost exactly the same prediction. The issue with adversarial images is that you add a very particular perturbation to the image, not just any perturbation at all.
  • darawk 10y ago
    I think the idea proposed is that if you add white noise on top of the adversarial perturbation, it will destroy that very particular perturbation.
    • esoterica 10y ago
      It won't destroy the perturbation. There is a low dimensional manifold along which the cost function will decrease/increase. The adversarial perturbation lies on that manifold, but random white noise (which is a random high dimensional vector) will have close to zero length with high probability when projected onto the manifold and hence won't affect the cost function.
    • antognini 10y ago
      Ohhhh, I see. I'm not sure whether that would work. I would have to actually do the experiment. It may not work because this bizarro region of parameter space may be somewhat robust to perturbations, so in a sense you may have to travel out same way you travelled in. But, then again, maybe not. Edit: I should add that these perturbations appear to be very robust to different architectures and datasets. So, the same adversarial perturbation will trick different NNs that were trained on different datasets. This suggests that it will probably be fairly robust to noise. But maybe not! I'm not aware of this experiment having been done.