5 ms·

No, this paper by Papernot et al shows how to do blackbox attacks without knowledge of model internals. https://arxiv.org/abs/1602.02697 https://arxiv.org/abs/1

• naveen99 9y ago
  Thanks. But they still need to be able to use the black box a brute force number of times on shady inputs and get detailed outputs during their gradient descent. Not going to be allowed on a check reader or anything sensitive.
  • haraldurt 9y ago
    Not necessarily. Adversarial examples have been shown to, for instance, be transferable across different networks with different hyperparameters (e.g., number of layers) trained on disjoint subsets of a training set [0, section 4.2]. There are more references from the paper linked by the OP. [0] https://arxiv.org/abs/1312.6199 https://arxiv.org/abs/1312.6199
    • naveen99 9y ago
      Thanks. I wonder if adversarial training helps prevent overfitting too. Could you use adversarial training to beat alphago ?
      • yorwba 9y ago
        You could not, because AlphaGo is not a classifier (so it isn't well-defined what an adversarial example is) and the input space is discrete (Go board state) and you can't do ε-small perturbations (two different states differ by at least one stone).