5 ms·

Reading minds with deep learning

82 points by whatrocks 8y ago

emilwallner 8y ago
While the results are really cool, they're not perfect, and seemingly even the best companies working with deep learning are still not getting it right every time. It makes me wonder if this technology will ever be useful for these tasks... I mean would you be willing to use BCIs even if they made small errors from time to time?
- samlynnevans 8y ago
  Yes that's the big question, and I mention that in the article. Let's say for a moment that we can't achieve 100% accuracy, well one parallel to this situation would be autocorrect on our phones; it often corrects you when you didn't want it to or misinterprets your meaning. Yet we still use it as there is net gain in terms of convenience. If a BCI could instantly put your thought on to paper, and left you solely with a few typos to mop up, I think we'd all be willing to compromise.
- cwkoss 8y ago
  There is a lot of variance in error rate in biological human nerves as well. Certainly a lot of people on the low-end of the bell curve could benefit. Interesting to consider whether a 'filter' could be made for people with parkinson's that would drop tremor noise while preserving intentional actions. Perhaps with robotic exosleeves tremor could be offset/counteracted - or eventually we could input the filtered signal back into the nerve.
  - samlynnevans 8y ago
    That would be a fantastic application, however it would differ slightly in that it would not be a BCI (as in a traditional brain computer interface), but a device within the brain that impeded certain signals being sent to motor neurons that it detected were Parkinsonian. This would involve directly manipulating the output of the brain to the body itself, and require some serious bio-mechanical hardware!
    - AstralStorm 8y ago
      This hardware already exists as an implantable electrode and is used in really hard cases. It is a competitive inhibitor and additionally stimulates dopamine release. As for putting something like that in spinal cord or motor centers, very risky surgery. Much more than cranial stimulator one. If this device fails in such a simple way as being jostled, you die.
- exo-pla-net 8y ago
  See the progression of voice recognition. The answer is, yes, a BCI is still useful and will still be used, even if it makes errors. It will of course grow more useful and become more embedded into our lives, as it gets more accurate.
- rajangdavis 8y ago
  I think being able to get 90%+ accuracy is pretty bad ass. I think with more data over time, there can hopefully be a good amount of progress that can be made. I don't even know what would make sense to improve the model (such as lowering false positive or false negative rates), but this research sounds pretty exciting!
janeybrad 8y ago
I think BCIs are super exciting. Humans have such high bandwidth in terms of input; tons of information is interpreted by our nervous system every second. Yet in terms of output, we are relatively poor; we can only type a few words a minute! This article suggests we could one day work 'like digital octopuses' but I'm naturally dubious, do you think it's possible as CTRL-labs propose that we could entirely rewire our nervous systems' output to let us operate computers as if we had multiple hands?
- samlynnevans 8y ago
  Great points. Thomas reardon (CTRL-labs founder) says the same in a talk you can see on YouTube (definitely check it out). I share your doubts, and wonder exactly how neuroplastic we are. Perhaps we are simply built to operate in a certain way, and solely tapping in to our nerves responsible for output won't necessarily allow us to output in hugely different ways. Even if possible, it may not be so simple,and instead becoming a digital octopus may be like having to learn the piano, requiring years of practice!
  - Cyphase 8y ago
    Here's that talk you mentioned: https://www.youtube.com/watch?v=5Z5aZK2C3ew https://www.youtube.com/watch?v=5Z5aZK2C3ew (2018 O'Reilly AI Keynote, Thomas Reardon, CEO, CTRL-labs)
    - samlynnevans 8y ago
      Cheers!
- carapace 8y ago
  > do you think it's possible as CTRL-labs propose that we could entirely rewire our nervous systems' output to let us operate computers as if we had multiple hands? Yes. You would not quite do it that way though. Your motor cortex is capable of handling that sort of thing[1] but your conscious awareness isn't[2] so you have to develop a kind of expanded awareness, like a dancer or martial artist, but in an abstract space.[3] (I suppose you could just hallucinate having eight arms, now that I think about it.) You don't have to wait, you can get results with crude hardware. The trick is to use hypnosis. See my other comment in this page[4]. We could have fantastic "magical" UI a minute ago, but people connect a brain to a computer and then focus on programming the computer. It's silly. (BTW, our output channels are primarily the face and the whole body[5], but there's evidence that sign language predates spoken language. I've heard that pre-verbal babies can learn to communicate with gestures.) [1] Higher dimensionality or more degrees of freedom [2] Generally, for most people. However it is possible to train your mind to carry out two or more simultaneous activities, such as writing a sentence with one hand while solving a math equation with the other. But this is sort of a stunt. Most folks would never conceivably need to do something like that. [3] Obligatory reference to "Cyberspace", and "Minority Report" UI. [4] https://news.ycombinator.com/item?id=18564989 https://news.ycombinator.com/item?id=18564989 [5] Gait, posture, overall tension/ease, etc.
twillmas 8y ago
Curious about differences in brain waves for desire versus intent. I might desire to say something awful to that person who just honked at me, but I probably shouldn’t - and right now wouldn’t. Would a BCI system be able to tell the difference?
- samlynnevans 8y ago
  Haha, that's a fantastic point! And in order to get anywhere closer to making such differentiations, surface EEG S (used for the results into his article) would surely not suffice. The technology and results we are collecting currently are just scratching the surface, though I'd suspect if a BCI was capable of detecting intent of such complexity as the situation you described, it would also probably have the capacity to know whether it was something you planned to act on or not.
carapace 8y ago
If you connect a computer to a brain which element is the more sophisticated processing device? The brain. If you want high-fidelity cheap BCI don't bother with fancy sensors, galvanic skin response is enough. The trick is to use the high-powered processor to do the heavy lifting. You use hypnosis and create a simple feedback loop to calibrate e.g. one channel per finger and one per thumb. That gives you 8 data lines and a clock/ack signal. An eight-bit parallel port from your mind to your computer. You can now come out of trance and "type" by thinking at the machine in a certain way (no imaginary keyboard) and your verbal thoughts appear on the screen. Other output modalities are possible. It's fucking trivial, if you'll pardon my language. It's ridiculous to connect a computer to a brain and then concentrate on the computer, rather than the brain, to develop control and communication systems. It's a bit like trying to move your bicycle using a skateboard. But the greater problem with focusing on the CPU rather than the brain is that it reinforces a world that doesn't demand much personal growth to use powerful tools. BCI where the computer does the work relieves the user of learning how to operate his or her own brain. BCI-by-hypnosis requires the user to learn enough self-operation to be able to go into a light trance and learn a simple motor pattern, and the meta-lesson "side effect" is more valuable and more important than the ability to interact with computers with greater facility. (BTW, I doubt this would ever be a mass-market product, because you're not just selling a peripheral at that point...) - - - - edit: Folks, I'm gonna ask that you please not downvote me on this one without saying something about why. I've never been able to get anyone to listen or take me seriously on this particular subject, and frankly that sucks. I know this works but no one cares and it's frustrating. So, again, please, if you think this is worth a downvote can you please take a minute and tell me why?
- trevyn 8y ago
  That’s cool! Have you actually done this? What clock rate can you achieve?
  - carapace 8y ago
    I forgot to mention in my main reply, I did set up a binary Boolean signal system as part of my exploration of hypnosis years ago. At first it was finger twitching, but then later the system was arm twitches, right for affirmative, left for negatory, non-response meant I[1] was framing the query in a way that precluded an unambiguous y/n answer. I made use of this "comm channel" for years, occasionally friends would remark on my odd arm twitches, eventually they "wore off" in the sense that I didn't need them anymore. Anyhow, this is what I meant about "trivial". These days you could wear e.g. a Fitbit on each arm and "Ta-da!" you've got a BCI. You wouldn't even have to stop by RadioShack (RIP) or wire up IMUs to an Arduino. [1] At a minimum you have to postulate three "entities": the perceive, the conscious mind, and the unconscious mind, for the model to make sense. In the above "I" the conscious mind is posing queries to "me" the unconscious mind, and I'm not mentioning the Perceiver at all.

82 points by whatrocks 8y ago

While the results are really cool, they're not perfect, and seemingly even the best companies working with deep learning are still not getting it right every time. It makes me wonder if this technology will ever be useful for these tasks... I mean would you be willing to use BCIs even if they made small errors from time to time?

samlynnevans 8y ago
Yes that's the big question, and I mention that in the article. Let's say for a moment that we can't achieve 100% accuracy, well one parallel to this situation would be autocorrect on our phones; it often corrects you when you didn't want it to or misinterprets your meaning. Yet we still use it as there is net gain in terms of convenience. If a BCI could instantly put your thought on to paper, and left you solely with a few typos to mop up, I think we'd all be willing to compromise.
cwkoss 8y ago
There is a lot of variance in error rate in biological human nerves as well. Certainly a lot of people on the low-end of the bell curve could benefit. Interesting to consider whether a 'filter' could be made for people with parkinson's that would drop tremor noise while preserving intentional actions. Perhaps with robotic exosleeves tremor could be offset/counteracted - or eventually we could input the filtered signal back into the nerve.
- samlynnevans 8y ago
  That would be a fantastic application, however it would differ slightly in that it would not be a BCI (as in a traditional brain computer interface), but a device within the brain that impeded certain signals being sent to motor neurons that it detected were Parkinsonian. This would involve directly manipulating the output of the brain to the body itself, and require some serious bio-mechanical hardware!
  - AstralStorm 8y ago
    This hardware already exists as an implantable electrode and is used in really hard cases. It is a competitive inhibitor and additionally stimulates dopamine release. As for putting something like that in spinal cord or motor centers, very risky surgery. Much more than cranial stimulator one. If this device fails in such a simple way as being jostled, you die.
exo-pla-net 8y ago
See the progression of voice recognition. The answer is, yes, a BCI is still useful and will still be used, even if it makes errors. It will of course grow more useful and become more embedded into our lives, as it gets more accurate.
rajangdavis 8y ago
I think being able to get 90%+ accuracy is pretty bad ass. I think with more data over time, there can hopefully be a good amount of progress that can be made. I don't even know what would make sense to improve the model (such as lowering false positive or false negative rates), but this research sounds pretty exciting!

janeybrad 8y ago

I think BCIs are super exciting. Humans have such high bandwidth in terms of input; tons of information is interpreted by our nervous system every second. Yet in terms of output, we are relatively poor; we can only type a few words a minute! This article suggests we could one day work 'like digital octopuses' but I'm naturally dubious, do you think it's possible as CTRL-labs propose that we could entirely rewire our nervous systems' output to let us operate computers as if we had multiple hands?

samlynnevans 8y ago
Great points. Thomas reardon (CTRL-labs founder) says the same in a talk you can see on YouTube (definitely check it out). I share your doubts, and wonder exactly how neuroplastic we are. Perhaps we are simply built to operate in a certain way, and solely tapping in to our nerves responsible for output won't necessarily allow us to output in hugely different ways. Even if possible, it may not be so simple,and instead becoming a digital octopus may be like having to learn the piano, requiring years of practice!
- Cyphase 8y ago
  Here's that talk you mentioned: https://www.youtube.com/watch?v=5Z5aZK2C3ew https://www.youtube.com/watch?v=5Z5aZK2C3ew (2018 O'Reilly AI Keynote, Thomas Reardon, CEO, CTRL-labs)
  - samlynnevans 8y ago
    Cheers!
carapace 8y ago
> do you think it's possible as CTRL-labs propose that we could entirely rewire our nervous systems' output to let us operate computers as if we had multiple hands? Yes. You would not quite do it that way though. Your motor cortex is capable of handling that sort of thing[1] but your conscious awareness isn't[2] so you have to develop a kind of expanded awareness, like a dancer or martial artist, but in an abstract space.[3] (I suppose you could just hallucinate having eight arms, now that I think about it.) You don't have to wait, you can get results with crude hardware. The trick is to use hypnosis. See my other comment in this page[4]. We could have fantastic "magical" UI a minute ago, but people connect a brain to a computer and then focus on programming the computer. It's silly. (BTW, our output channels are primarily the face and the whole body[5], but there's evidence that sign language predates spoken language. I've heard that pre-verbal babies can learn to communicate with gestures.) [1] Higher dimensionality or more degrees of freedom [2] Generally, for most people. However it is possible to train your mind to carry out two or more simultaneous activities, such as writing a sentence with one hand while solving a math equation with the other. But this is sort of a stunt. Most folks would never conceivably need to do something like that. [3] Obligatory reference to "Cyberspace", and "Minority Report" UI. [4] https://news.ycombinator.com/item?id=18564989 https://news.ycombinator.com/item?id=18564989 [5] Gait, posture, overall tension/ease, etc.

twillmas 8y ago

Curious about differences in brain waves for desire versus intent. I might desire to say something awful to that person who just honked at me, but I probably shouldn’t - and right now wouldn’t. Would a BCI system be able to tell the difference?

samlynnevans 8y ago
Haha, that's a fantastic point! And in order to get anywhere closer to making such differentiations, surface EEG S (used for the results into his article) would surely not suffice. The technology and results we are collecting currently are just scratching the surface, though I'd suspect if a BCI was capable of detecting intent of such complexity as the situation you described, it would also probably have the capacity to know whether it was something you planned to act on or not.

carapace 8y ago

If you connect a computer to a brain which element is the more sophisticated processing device? The brain. If you want high-fidelity cheap BCI don't bother with fancy sensors, galvanic skin response is enough. The trick is to use the high-powered processor to do the heavy lifting. You use hypnosis and create a simple feedback loop to calibrate e.g. one channel per finger and one per thumb. That gives you 8 data lines and a clock/ack signal. An eight-bit parallel port from your mind to your computer. You can now come out of trance and "type" by thinking at the machine in a certain way (no imaginary keyboard) and your verbal thoughts appear on the screen. Other output modalities are possible. It's fucking trivial, if you'll pardon my language. It's ridiculous to connect a computer to a brain and then concentrate on the computer, rather than the brain, to develop control and communication systems. It's a bit like trying to move your bicycle using a skateboard. But the greater problem with focusing on the CPU rather than the brain is that it reinforces a world that doesn't demand much personal growth to use powerful tools. BCI where the computer does the work relieves the user of learning how to operate his or her own brain. BCI-by-hypnosis requires the user to learn enough self-operation to be able to go into a light trance and learn a simple motor pattern, and the meta-lesson "side effect" is more valuable and more important than the ability to interact with computers with greater facility. (BTW, I doubt this would ever be a mass-market product, because you're not just selling a peripheral at that point...) - - - - edit: Folks, I'm gonna ask that you please not downvote me on this one without saying something about why. I've never been able to get anyone to listen or take me seriously on this particular subject, and frankly that sucks. I know this works but no one cares and it's frustrating. So, again, please, if you think this is worth a downvote can you please take a minute and tell me why?

trevyn 8y ago
That’s cool! Have you actually done this? What clock rate can you achieve?
- carapace 8y ago
  I forgot to mention in my main reply, I did set up a binary Boolean signal system as part of my exploration of hypnosis years ago. At first it was finger twitching, but then later the system was arm twitches, right for affirmative, left for negatory, non-response meant I[1] was framing the query in a way that precluded an unambiguous y/n answer. I made use of this "comm channel" for years, occasionally friends would remark on my odd arm twitches, eventually they "wore off" in the sense that I didn't need them anymore. Anyhow, this is what I meant about "trivial". These days you could wear e.g. a Fitbit on each arm and "Ta-da!" you've got a BCI. You wouldn't even have to stop by RadioShack (RIP) or wire up IMUs to an Arduino. [1] At a minimum you have to postulate three "entities": the perceive, the conscious mind, and the unconscious mind, for the model to make sense. In the above "I" the conscious mind is posing queries to "me" the unconscious mind, and I'm not mentioning the Perceiver at all.