3 ms·

It makes me wonder if you could create a physical scene that, when photographed by a digital camera, exploited the camera.

by caf 9y ago

quxs 9y ago
Not a digital camera, but I had once written some poor vision processing code that would segfault when it recognized a certain pattern. It was interesting to think about, that I could break my program by showing it a pciture.
EvilTerran 9y ago
I'm reminded of David Langford's "basilisks": http://www.infinityplus.co.uk/stories/blit.htm http://www.infinityplus.co.uk/stories/blit.htm [edit] more in the series: http://ansible.uk/writing/c-b-faq.html http://ansible.uk/writing/c-b-faq.html http://www.lightspeedmagazine.com/fiction/different-kinds-of-darkness/ http://www.lightspeedmagazine.com/fiction/different-kinds-of...
number6 9y ago
Or a kind of message or language that when seen by a human or processed by a brain can crash or exploit it ... It's also the plot of a good book: https://en.wikipedia.org/wiki/Snow_Crash https://en.wikipedia.org/wiki/Snow_Crash
- captainmuon 9y ago
  You can use adversarial neural networks to make a special picture that looks like noise or junk, but a certain neural network will recognize e.g. a dog in it. You basically train the ANN to make something that the other NN classifies as dog, but humans don't. I often wonder if you could do that with humans. I don't think you can crash a human brain a la Snow Crash or Basilisk, but I think it is plausible to craft an abstract picture that creates fear, arousal, or some other "primitive" response.
  - digikata 9y ago
    > but I think it is plausible to craft an abstract picture that creates fear, arousal, or some other "primitive" response. Visit a modern art museum - I think one can find many examples.
gibberfish 9y ago
There's ongoing work in adversarial examples for neural networks. This could conceivably be used to exploit self-driving cars, making it see things that aren't there.

It makes me wonder if you could create a physical scene that, when photographed by a digital camera, exploited the camera.

by caf 9y ago

It makes me wonder if you could create a physical scene that, when photographed by a digital camera, exploited the camera.

quxs 9y ago

Not a digital camera, but I had once written some poor vision processing code that would segfault when it recognized a certain pattern. It was interesting to think about, that I could break my program by showing it a pciture.

EvilTerran 9y ago

I'm reminded of David Langford's "basilisks": http://www.infinityplus.co.uk/stories/blit.htm http://www.infinityplus.co.uk/stories/blit.htm [edit] more in the series: http://ansible.uk/writing/c-b-faq.html http://ansible.uk/writing/c-b-faq.html http://www.lightspeedmagazine.com/fiction/different-kinds-of-darkness/ http://www.lightspeedmagazine.com/fiction/different-kinds-of...

number6 9y ago

Or a kind of message or language that when seen by a human or processed by a brain can crash or exploit it ... It's also the plot of a good book: https://en.wikipedia.org/wiki/Snow_Crash https://en.wikipedia.org/wiki/Snow_Crash

captainmuon 9y ago
You can use adversarial neural networks to make a special picture that looks like noise or junk, but a certain neural network will recognize e.g. a dog in it. You basically train the ANN to make something that the other NN classifies as dog, but humans don't. I often wonder if you could do that with humans. I don't think you can crash a human brain a la Snow Crash or Basilisk, but I think it is plausible to craft an abstract picture that creates fear, arousal, or some other "primitive" response.
- digikata 9y ago
  > but I think it is plausible to craft an abstract picture that creates fear, arousal, or some other "primitive" response. Visit a modern art museum - I think one can find many examples.

gibberfish 9y ago

There's ongoing work in adversarial examples for neural networks. This could conceivably be used to exploit self-driving cars, making it see things that aren't there.