3 ms·

Are there adversarial examples for Alpha Go Zero?

• robotresearcher 9y ago
  The NN played against itself.
• danielvf 9y ago
  What do you mean? AlphaGo Zero has played itself, and they have published 20 of those games.
  • Barrin92 9y ago
    the OP is referring to the use of a GAN, which is a different type of setup. GAN's were not used for alphaGo, as the article points out Deepmind uses reinforcement learning and MCTS.
    • nschucher 9y ago
      They're most likely referring to adversarial attacks where degenerate inputs are constructed that could cause AlphaGo Zero to perform sub-optimally or catastrophically fail (see OpenAI Research [0]). This is distinct from generative adversarial networks (GANs) or adversarial self-play (which I guess AlphaGo Zero is an example of). [0] https://blog.openai.com/adversarial-example-research/ https://blog.openai.com/adversarial-example-research/
• comex 9y ago
  It seems like it would be drastically harder to attack a Go-playing network compared to an image classifier, just because there’s so much less freedom in input. With an image, you can vary each pixel independently however you want. In a game of Go, on any given turn there are probably only a handful of moves that don’t amount to throwing away territory, which would probably be harmful enough to offset any gain by ‘confusing’ the AI. Not only that. the list of suitable moves depends on the AI’s choices in addition to its opponent’s, making the state space highly discontinuous. Who knows, though… maybe there’s some vulnerability so severe that you can spend half the game wasting your moves and win anyway, because the AI gets so confused that it stops playing with even basic competence. I guess that image classifiers giving 99.9% confidence to the wrong class also seems pretty ‘incompetent’… and it would be very cool to see. Another obstacle is that whereas an image classifier is basically a purely learned function, Monte Carlo tree search has a hard-coded element of applying potential moves to find new states to explore - always following the game’s rules correctly. If you go more than a few moves ahead, it has a relatively unpredictable choice of what states to explore; but as you get closer to the current move, I think it starts to look a bit more like brute force. So if you want the AI to make a really bad move - one that has severe consequences in the next few turns - you need to not just make the NN think it looks good, but that the game state still looks good even after you make any possible counterplay. Disclaimer: I don’t actually know much about either neural networks or Go.
• gwern 9y ago
  Possibly but no one has demonstrated them, and there are probably many fewer in Zero than in previous AlphaGos. Previously AlphaGos suffered from problems which Silver calls 'delusions' (https://www.reddit.com/r/MachineLearning/comments/76xjb5/ama_we_are_david_silver_and_julian_schrittwieser/dohp9vc/ https://www.reddit.com/r/MachineLearning/comments/76xjb5/ama...) ie persistent long-term misevaluations of specific board positions, which were however (entirely? how would we know if not?) fixed by Zero's self-play - arguably these are not 'adversarial examples' in the sense you're thinking of but then I'm not sure what would constitute an adversarial example for a Go agent, so maybe the AlphaGo 'delusions' were the equivalent. (On a side note, the more I think about Anthony et al 2017/AlphaGo Zero's tree/expert iteration, the more beautifully simple it appears to me.)