3 ms·
My idea is that without added noise, knowing the model and the input allows crafting a pertubation that leads to an erroneous output of the model. With noise a
by madez 9y ago
My idea is that without added noise, knowing the model and the input allows crafting a pertubation that leads to an erroneous output of the model.
With noise added there is less correlation between the input and the output. At the extreme with 100% randomness added, there is no correlation anymore between any pertubations of the input and the output. However, there is unfortunately also no correlation anymore between the input and the output.
What happens if you add a bit of noise? The more noise, the smaller the correlation between the perturbations and the output. At what point is the probability of a successful attack sufficiently small?
To clarify, I mean adding the noise not in the training phase or to the images itself, but at the input stage into the model. That way even the repeated input of the same image would result in different inputs to the model.
I'm not sure this type of protection is efficient and effective, but it's an idea.
- eanzenberg 9y agoAdding noise is important for generalization and predictive power but it may just shift the attacking pixel from one to another, since the model itself is inherently compressed, inversely correlated with its complexity.