5 ms·

This book featured adversarial inputs to the human neural network long before deep learning was cool.

jacquesm 7y ago
Epileptic seizures can be triggered, that's the only actual adversarial attack on the human neural network that I'm aware of that actually works, and is more common than you might think: https://www.epilepsysociety.org.uk/photosensitive-epilepsy https://www.epilepsysociety.org.uk/photosensitive-epilepsy
- schoen 7y ago
  You could say that all perceptual illusions are adversarial inputs for the human senses. They're just not damaging ones. In a lot of adversarial input stuff for AI, you're trying to get the AI to misclassify something, not necessarily to damage the AI or even change its learning at all. In Snow Crash the perceptions were definitely damaging to the perceivers, but I don't think you have to go nearly that far to have adversarial inputs!
  - jacquesm 7y ago
    True, but it was just the Snow Crash like ones that I had in mind. There are also adversarial attacks possible on all the other senses once you understand how they work at a physical or pre-processing level. You can make something cold feel hot, trigger speech patterns to go off by pre-setting them with auditory patterns, cause people to hear sounds that aren't there and so on. Given how many of these pathways are buggy it is quite impressive how well the brain can make sense of its surroundings. Mimicry in the animal kingdom is another 'adversarial input' that has potentially damaging effects on the perceivers (hunger!) but it is very much a defensive mechanism so that probably should not count either.
    - wizardforhire 7y ago
      True nothing like snowcrash has yet to be created to any of our knowledges. Of course... if it was... how would we be able to respond to this thread! But this is whole other rabbit hole to go down that in light of some recents posts on the front page today doesn’t seem too fun.
  - hwbehrens 7y ago
    I once read a piece of text whose contents was so vividly horrific that it caused me to momentarily pass out, falling out of my chair. Granted, it passed through an intermediate representation in my brain, but I would still count it as an adversarial input to my vision that caused lasting damage. The damage in Snow Crash just happened to be more severe than "short-term distress followed by periodic unsettlement".
- wizardforhire 7y ago
  Idk, human neural networks seem to be susceptible to all sorts of adversarial attacks. Snow-crash is the first example I know of, of the attack being condensed to a bitmap and experienced solely through the optic nerve and subsequently and substantially condensed in the time domain. Typically the sort of results of an adversarial attack against a human neural network similar to one described in snowcrash is more indicative of long term and repeated exposure to adversarial stimulus that usually starts at birth and last through sometimes as late as adulthood. Snowcrash condenses this effect and that’s whats striking and interesting. To elaborate, the sort of adversarial attack’s I’m thinking of, are: indoctrination; gaslighting; cycles of abuse (a very nasty one that can go on for generations!); just about anything in the cia psy-ops manual, oh and I shouldn’t leave off entrapment (that’s a fun one).
- aasasd 7y ago
  We just don't yet know enough about the low-level workings of the brain. Apart from psychological manipulation, of course―as already noted in another comment.
- turingbike 7y ago
  I assume you have never explored your own literal blind spot [0] and how the brain just plasters over it with some hacky texture filling... [0] https://en.wikipedia.org/wiki/Blind_spot_(vision) https://en.wikipedia.org/wiki/Blind_spot_(vision)
  - jacquesm 7y ago
    Why would you assume that?

Epileptic seizures can be triggered, that's the only actual adversarial attack on the human neural network that I'm aware of that actually works, and is more common than you might think: https://www.epilepsysociety.org.uk/photosensitive-epilepsy https://www.epilepsysociety.org.uk/photosensitive-epilepsy

schoen 7y ago
You could say that all perceptual illusions are adversarial inputs for the human senses. They're just not damaging ones. In a lot of adversarial input stuff for AI, you're trying to get the AI to misclassify something, not necessarily to damage the AI or even change its learning at all. In Snow Crash the perceptions were definitely damaging to the perceivers, but I don't think you have to go nearly that far to have adversarial inputs!
- jacquesm 7y ago
  True, but it was just the Snow Crash like ones that I had in mind. There are also adversarial attacks possible on all the other senses once you understand how they work at a physical or pre-processing level. You can make something cold feel hot, trigger speech patterns to go off by pre-setting them with auditory patterns, cause people to hear sounds that aren't there and so on. Given how many of these pathways are buggy it is quite impressive how well the brain can make sense of its surroundings. Mimicry in the animal kingdom is another 'adversarial input' that has potentially damaging effects on the perceivers (hunger!) but it is very much a defensive mechanism so that probably should not count either.
  - wizardforhire 7y ago
    True nothing like snowcrash has yet to be created to any of our knowledges. Of course... if it was... how would we be able to respond to this thread! But this is whole other rabbit hole to go down that in light of some recents posts on the front page today doesn’t seem too fun.
- hwbehrens 7y ago
  I once read a piece of text whose contents was so vividly horrific that it caused me to momentarily pass out, falling out of my chair. Granted, it passed through an intermediate representation in my brain, but I would still count it as an adversarial input to my vision that caused lasting damage. The damage in Snow Crash just happened to be more severe than "short-term distress followed by periodic unsettlement".
wizardforhire 7y ago
Idk, human neural networks seem to be susceptible to all sorts of adversarial attacks. Snow-crash is the first example I know of, of the attack being condensed to a bitmap and experienced solely through the optic nerve and subsequently and substantially condensed in the time domain. Typically the sort of results of an adversarial attack against a human neural network similar to one described in snowcrash is more indicative of long term and repeated exposure to adversarial stimulus that usually starts at birth and last through sometimes as late as adulthood. Snowcrash condenses this effect and that’s whats striking and interesting. To elaborate, the sort of adversarial attack’s I’m thinking of, are: indoctrination; gaslighting; cycles of abuse (a very nasty one that can go on for generations!); just about anything in the cia psy-ops manual, oh and I shouldn’t leave off entrapment (that’s a fun one).
aasasd 7y ago
We just don't yet know enough about the low-level workings of the brain. Apart from psychological manipulation, of course―as already noted in another comment.
turingbike 7y ago
I assume you have never explored your own literal blind spot [0] and how the brain just plasters over it with some hacky texture filling... [0] https://en.wikipedia.org/wiki/Blind_spot_(vision) https://en.wikipedia.org/wiki/Blind_spot_(vision)
- jacquesm 7y ago
  Why would you assume that?