3 ms·
Last week, I wrote a blog post (https://blog.openai.com/robust-adversarial-inputs/ https://blog.openai.com/robust-adversarial-inputs/) about how it's possible t
by anishathalye 9y ago
Last week, I wrote a blog post (https://blog.openai.com/robust-adversarial-inputs/ https://blog.openai.com/robust-adversarial-inputs/) about how it's possible to synthesize really robust adversarial inputs for neural networks. The response was great, and I got several requests to write a tutorial on the subject because what was already out there wasn't all that accessible. This post, written in the form of an executable Jupyter notebook, is that tutorial!
Security/ML is a fairly new area of research, but I think it's going to be pretty important in the next few years. There's even a very timely Kaggle competition about this (https://www.kaggle.com/c/nips-2017-defense-against-adversarial-attack https://www.kaggle.com/c/nips-2017-defense-against-adversari...) run by Google Brain. I hope that this blog post will help make this really neat area of research slightly more approachable/accessible! Also, the attacks don't require that much compute power, so you should be able to run the code from the post on your laptop.
- zitterbewegung 9y agoThank you very much! I requested this and I think I might have possibly come off as rude or entitled but I am so happy you donated your time to create this tutorial!
- anishathalye 9y agoOh, I didn't take it that way at all. In fact, I thought it was a great idea! :)
- sjroot 9y agoThanks so much for taking the time to write this post. I am a graduate student in computer security interested in branching into adversarial ML, and I have to echo your opinion that the material available today is not too accessible.
- jcims 9y agoI'm a dinosaur with 20 years in infosec and interested in the same thing. The role of the defender is going to become very interesting very quickly.
- sp332 9y agoDoes it take a lot longer to synthesize the robust version than the naive version?
- anishathalye 9y agoIt does take longer to synthesize robust adversarial examples - how much longer depends on the distribution of transformations. The rotation example in this blog post was pretty quick to create, maybe a minute or two on a Titan X. The printed-out version in the OpenAI blog post took something like 20 minutes on a Titan X.