6 ms·

Brain-Machine Interface Isn't Sci-Fi Anymore

85 points by kartD 9y ago

hprotagonist 9y ago
>...whether he has a hand is irrelevant This is probably a reach. At a bare minimum, decoding EMG won't work if you never had a hand. If you had an amputation, the means by which your hand was removed will greatly impact the efficacy of this sytem.
- philipov 9y ago
  Could you explain in more detail what the effects of different methods would be?
  - hprotagonist 9y ago
    amputations too high up, or accidental amputations that mangle rather than cleanly remove, are more likely to make EMG decoding harder. If the axons are too damaged during the amputation they'll wither and die, so the signal's lost; if the arm is removed too high up, it becomes more challenging to isolate existing neurons that went all the way to the fingers. In both cases, muscle movement itself will swamp the motor neuron signal in ways that make demuxing very challenging.
    - dTal 9y ago
      >neurons that went all the way to the fingers. nitpick: there are no such motor neurons. The fingers are controlled by muscles in the palm and forearm. But you probably knew this :)
- lurker456 9y ago
  if the nerve endings between brain and shoulder are there, it could still work. The device would need to give consistent feedback for the brain to be trained. it's similar to a stroke victim learning to walk again.
hacker_9 9y ago
"He resumes typing. Only this time he is typing on…nothing. Just the flat tabletop. Yet the result is the same: The words he taps out appear on the monitor. The text on the screen is being generated not by his fingertips, but rather by the signals his brain is sending to his fingers. The armband is intercepting those signals, interpreting them correctly, and relaying the output to the computer" This is a pretty neat idea, but I feel not being able to see the keyboard is kind of going to ruin this one. Even with touch typing you need some feedback as to where the 'F' and 'J' are, and even then you can't reach the whole keyboard. I suspect you can work around that with some sort of 'finger swipe' language though. But what about the mouse? Being able to put your hand on something is lot more comfortable than, for example, using a touchpad on a laptop. The problem with these sort of devices (reminds me of the wii remote) is all the stress is moved to the fingers and wrists, which doesn't pay off well in the long run.
- lukeschlather 9y ago
  Your first concern seems like a non-issue. Assuming your fingers are always making the same movements, it's effectively like the F and J are magically wherever your index fingers happen to be. "Can't reach the whole keyboard" is a bit of a problem, but the obvious way it would work is that the signal which the computer interprets as "position my hand over the numpad" becomes a input mode shift. There's probably more there. Without knowing anything about how the sensor tech works, I suspect that there's a difference between the signals when my hands are on home row and when my right hand is on the numpad. The mouse seems like it might be even easier in the long run. I'm assuming the training is harder, but waving hands in the air sounds way better than resting hands on a padded surface. RSI in general sounds like the X factor. Input via odd muscle twitching sounds like it could raise all sorts of weird muscle issues long-term.
  - jalk 9y ago
    I believe that "Resting" is actually key here - the Minority Report UI will most likely tire your arms pretty quickly.
icoder 9y ago
Since he has a hand, how do they discern between the electrical signals picked up from the motor neurons, and the electrical signals created by the muscles (which are a lot stronger)? To compare: some years ago I saw a talk from neuroscience researcher about a locked in patients. Only with great effort they were able to discern a single on/off signal from the brain, and only after a lot of practice. I asked why commercial products seemed much further than that, and he explained that these basically pickup subtle nearby (ie from the eye) muscles movements (that locked in patients no longer have), and users only 'believe' they are controlling the system directly with their brain.
- hacker_9 9y ago
  Motor neurons move the muscles, so does it matter which signal they pick up? With some machine learning in there, it should be able to discern between actual actions and random twitches. Edit: for down voters who didn't read the article, they already are using machine learning: "Picking up the signals is only the first step. Perhaps the most difficult part is then transforming them into signals that the device understands. This requires a combination of coding, machine learning, and neuroscience."
  - wlesieutre 9y ago
    Because one very important application for brain/computer interfaces is prosthetics. If muscles in the hand are interpreting the signals from motor neurons, and the computer reacts to muscle movements, then it's not useful for that. The amputees have motor neurons, but they don't have hands so you need to get the nerve signals more directly. I don't know anything more than GP with regard to this project, but that's the question they're asking.
- caycep 9y ago
  i suspect they do EMG. Much MUCH easier to pick up muscle, than motor neuron signal.
deleted 9y ago
[deleted]
munin 9y ago
I guess Silicon Valley hates it because the research was done by a bunch of academics and government scientists, but they've been working on and successfully demoing a brain interface that can do control and feedback with arms: https://hub.jhu.edu/2016/08/25/prosthetic-arm-sensory-touch/ https://hub.jhu.edu/2016/08/25/prosthetic-arm-sensory-touch/
- Confiks 9y ago
  You may guess right ahead without putting any effort into explaining yourself, but I'm pretty sure the team behind TFA hasn't ventured into your stray path considering the number of publications made, cited here: https://ctrl-labs.com/people https://ctrl-labs.com/people
Confiks 9y ago
It seems that the article is really careful to never show "those weird armbands sitting between his wrist and his elbows". Would anyone have an idea what kind of sensors it would require to measure signals traveling through motor nerves? Aside from the question of how to interpret these signals. The site of the CTRL-Labs team shows an impressive list [1] of publications. Only one [2] seems slightly related to to prototype of the article, where they test the role of the motor cortex with different movements in mice. [1] https://ctrl-labs.com/people https://ctrl-labs.com/people [2] http://www.cell.com/neuron/fulltext/S0896-6273(17)30594-9 http://www.cell.com/neuron/fulltext/S0896-6273(17)30594-9

85 points by kartD 9y ago

>...whether he has a hand is irrelevant This is probably a reach. At a bare minimum, decoding EMG won't work if you never had a hand. If you had an amputation, the means by which your hand was removed will greatly impact the efficacy of this sytem.

philipov 9y ago
Could you explain in more detail what the effects of different methods would be?
- hprotagonist 9y ago
  amputations too high up, or accidental amputations that mangle rather than cleanly remove, are more likely to make EMG decoding harder. If the axons are too damaged during the amputation they'll wither and die, so the signal's lost; if the arm is removed too high up, it becomes more challenging to isolate existing neurons that went all the way to the fingers. In both cases, muscle movement itself will swamp the motor neuron signal in ways that make demuxing very challenging.
  - dTal 9y ago
    >neurons that went all the way to the fingers. nitpick: there are no such motor neurons. The fingers are controlled by muscles in the palm and forearm. But you probably knew this :)
lurker456 9y ago
if the nerve endings between brain and shoulder are there, it could still work. The device would need to give consistent feedback for the brain to be trained. it's similar to a stroke victim learning to walk again.

hacker_9 9y ago

"He resumes typing. Only this time he is typing on…nothing. Just the flat tabletop. Yet the result is the same: The words he taps out appear on the monitor. The text on the screen is being generated not by his fingertips, but rather by the signals his brain is sending to his fingers. The armband is intercepting those signals, interpreting them correctly, and relaying the output to the computer" This is a pretty neat idea, but I feel not being able to see the keyboard is kind of going to ruin this one. Even with touch typing you need some feedback as to where the 'F' and 'J' are, and even then you can't reach the whole keyboard. I suspect you can work around that with some sort of 'finger swipe' language though. But what about the mouse? Being able to put your hand on something is lot more comfortable than, for example, using a touchpad on a laptop. The problem with these sort of devices (reminds me of the wii remote) is all the stress is moved to the fingers and wrists, which doesn't pay off well in the long run.

lukeschlather 9y ago
Your first concern seems like a non-issue. Assuming your fingers are always making the same movements, it's effectively like the F and J are magically wherever your index fingers happen to be. "Can't reach the whole keyboard" is a bit of a problem, but the obvious way it would work is that the signal which the computer interprets as "position my hand over the numpad" becomes a input mode shift. There's probably more there. Without knowing anything about how the sensor tech works, I suspect that there's a difference between the signals when my hands are on home row and when my right hand is on the numpad. The mouse seems like it might be even easier in the long run. I'm assuming the training is harder, but waving hands in the air sounds way better than resting hands on a padded surface. RSI in general sounds like the X factor. Input via odd muscle twitching sounds like it could raise all sorts of weird muscle issues long-term.
- jalk 9y ago
  I believe that "Resting" is actually key here - the Minority Report UI will most likely tire your arms pretty quickly.

icoder 9y ago

Since he has a hand, how do they discern between the electrical signals picked up from the motor neurons, and the electrical signals created by the muscles (which are a lot stronger)? To compare: some years ago I saw a talk from neuroscience researcher about a locked in patients. Only with great effort they were able to discern a single on/off signal from the brain, and only after a lot of practice. I asked why commercial products seemed much further than that, and he explained that these basically pickup subtle nearby (ie from the eye) muscles movements (that locked in patients no longer have), and users only 'believe' they are controlling the system directly with their brain.

hacker_9 9y ago
Motor neurons move the muscles, so does it matter which signal they pick up? With some machine learning in there, it should be able to discern between actual actions and random twitches. Edit: for down voters who didn't read the article, they already are using machine learning: "Picking up the signals is only the first step. Perhaps the most difficult part is then transforming them into signals that the device understands. This requires a combination of coding, machine learning, and neuroscience."
- wlesieutre 9y ago
  Because one very important application for brain/computer interfaces is prosthetics. If muscles in the hand are interpreting the signals from motor neurons, and the computer reacts to muscle movements, then it's not useful for that. The amputees have motor neurons, but they don't have hands so you need to get the nerve signals more directly. I don't know anything more than GP with regard to this project, but that's the question they're asking.
caycep 9y ago
i suspect they do EMG. Much MUCH easier to pick up muscle, than motor neuron signal.

deleted 9y ago

[deleted]

munin 9y ago

I guess Silicon Valley hates it because the research was done by a bunch of academics and government scientists, but they've been working on and successfully demoing a brain interface that can do control and feedback with arms: https://hub.jhu.edu/2016/08/25/prosthetic-arm-sensory-touch/ https://hub.jhu.edu/2016/08/25/prosthetic-arm-sensory-touch/

Confiks 9y ago
You may guess right ahead without putting any effort into explaining yourself, but I'm pretty sure the team behind TFA hasn't ventured into your stray path considering the number of publications made, cited here: https://ctrl-labs.com/people https://ctrl-labs.com/people

Confiks 9y ago

It seems that the article is really careful to never show "those weird armbands sitting between his wrist and his elbows". Would anyone have an idea what kind of sensors it would require to measure signals traveling through motor nerves? Aside from the question of how to interpret these signals. The site of the CTRL-Labs team shows an impressive list [1] of publications. Only one [2] seems slightly related to to prototype of the article, where they test the role of the motor cortex with different movements in mice. [1] https://ctrl-labs.com/people https://ctrl-labs.com/people [2] http://www.cell.com/neuron/fulltext/S0896-6273(17)30594-9 http://www.cell.com/neuron/fulltext/S0896-6273(17)30594-9