7 ms·

DARPA grant to work on sensing and stimulating the brain noninvasively [video]

83 points by grawprog 5y ago

ckemere 5y ago
The title is misleading. DARPA has funded teams to work on minimally invasive brain interfaces, but AFAIK they have yet to be achieved.
- AussieWog93 5y ago
  And as someone who studied this field at a PhD level, I would be shocked if they managed to achieve a level of control on par with even a standard Xbox controller any time within the next two decades. The whole field is propelled by deliberate misinformation and hype used to farm grants.
  - graderjs 5y ago
    Really, is it really so hard? I had the impression from seeing those videos of people operating robotic limbs through biofeedback or typing through biofeedback or the write-ups on that AI related work to determine words from brainwave patterns that a lot of this stuff was achieving impressive capabilities. but maybe the sort of sensory bandwidth like required to operate say a helicopter is just a completely other order of magnitude to think that you could do it through the same channel. What's the main blockers or bottlenecks in this space?
    - dexwiz 5y ago
      I would imagine some of those videos are the hype and misinformation that the GP is talking about
      - AussieWog93 5y ago
        Yes, a lot of the videos shown (such as the pig walking cycle in the Neuralink presentation) deliberately misrepresent the challenges, limitations and lack of progress with the technology in an attempt to get more funding. I guess you could argue that people writing the grants should perform more due diligence, but at the same time I believe we should be able to trust scientists to act ethically and honestly in the same way we trust doctors.
        graderjs 5y ago
        At the same time I reckon we should throw money at stuff even half- (not baked, but, half-) completed...because the pay-off is high (if it works) even if the current state of art right now is weak. I get your frustration at the perceived support for a research direction you view as wrong, but I think this is normal, especially in emerging tech. There are often competing approaches, and funding favorites. But I didn't know this--what you said about the difficulty of signal collection, and rejection--and really value that alternative perspective to the one I've seen in the press about using AI, which you think is overblown. Thank you! :)
        AussieWog93 5y ago
        >At the same time I reckon we should throw money at stuff even half- (not baked, but, half-) completed...because the pay-off is high (if it works) even if the current state of art right now is weak. I don't disagree with that line of reasoning in general - moonshot projects have led to amazing discoveries in the past and should definitely be funded. In this particular case, though, there are both very, very strong arguments to suggest that we reached the peak of what the current paradigm can achieve back in the late 00's - as well as very, very strong (financial and cultural) pressure to ensure all new research projects in the field adhere to the current paradigm. >I get your frustration at the perceived support for a research direction you view as wrong, but I think this is normal, especially in emerging tech. It's so, so much deeper than that. The researchers in the area know full well what they're doing is complete bullshit, but toe the line anyway because that's where the funding comes from. It's not just a wrong research direction but a soft and invisible form of academic fraud. This is not normal (or at least shouldn't be normal), and we should not be feeding the beast. We could possibly have much, much stronger BCI tech today if the incumbent culture weren't actively impeding innovation.
        graderjs 5y ago
        Wow, man. Thanks for sharing more how you feel about it. I thought it was just equal competing approaches, different camps. But you tell a different story. I really appreciate that. That's not just fraud, I think...but it's like wasted scientific capacity. You know? They're researchers, they could be working to build something, but instead they're taking money on an approach that can't work. That's sick. Feels bad.... So what do you think's the solution? Like if you could take 10 million dollars to do your own lab with BCI tech what direction would you go in? ;p ;) xx
    - grp000 5y ago
      I thought a lot of those videos worked off of nerve impulses at the terminated end of the limb.
      - AussieWog93 5y ago
        The guy with the robotic arm that looks like Bryan Cranston (can't remember his name off the top of my head) definitely uses nerve endings at the end of the limb. Others, such as the patient in Krishna Shenoy's 2017 paper, though, use invasive BCIs.
    - AussieWog93 5y ago
      >I had the impression from seeing those videos of people operating robotic limbs through biofeedback or typing through biofeedback or the write-ups on that AI related work to determine words from brainwave patterns that a lot of this stuff was achieving impressive capabilities. Sticking to BCI specifically and not other forms for biofeedback, most of those demos you see come from people with invasive, penetrative electrode arrays such as the Utah array. In some people, these sensors can enable a high level of control and performance (not Xbox controller level, but still Atari controller level which would be incredibly useful for people with Motor Neurone Disease). Unfortunately the body rejects them after a few months and they're back to square one. To be honest with you, I don't think that the problem's intractable. Perhaps a clever materials engineer could develop a coating for the electrode arrays that the body accepts long-term and allows the user to train for years rather than months. Perhaps we could develop a new form of sensor altogether. I don't know specifically what the answer is, but it won't come from the current paradigm of "turn garbage data into non-garbage output through the power of Machine Learning somehow" that we've been stuck in for the better part of two decades. And unfortunately, projects adhering to that paradigm is where all of the funding (and people) goes to.
      - cinquemb 5y ago
        > "turn garbage data into non-garbage output through the power of Machine Learning somehow" Yeah, i was pretty turned off by this in the lab i've worked at/contracted for. A lot of support for this kind of stuff wrt EEF neurofeedback while ignoring the physics of what could be done with the raw data (if the PI actually understands or had a E&M class at least). Ppl in charge of getting the grants don't really care about the fact we do not gain any insight into whats going on when we use black box systems… oh well. I'd be happy to play around with biosemi 128 electrode cap+array + dsp + beamformers[0] and openemu[1] for two decades, deff not gonna happen in academia though. [0] https://github.com/cinquemb/LCMVBeamformer https://github.com/cinquemb/LCMVBeamformer [1] https://github.com/cinquemb/OpenEmu https://github.com/cinquemb/OpenEmu
      - kodah 5y ago
        I did cognitive neurotherapy after a series of TBIs. The therapy seemed to work on me, though, what I've heard is that it's hit and miss. Of course, there's always the chance that I'd have recovered that way anyway, but given the state my academics were in I doubt it. For reference, the machine I used employed a bike game that supported movement across the screen. I eventually could make the bike go faster as well as up and down, nearly on command. It focused on alpha, beta, and theta waves measured by diodes stuck to your scalp. Not sure if that narrows that down, but I'm curious to know your thoughts.
  - pyinstallwoes 5y ago
    Yeah, but have you worked at the DARPA level?
mrtnmcc 5y ago
I'd written an EEG signal processing and inference backend for a drone control system that did this in 2010. It picks up on Mu rhythms from the motor cortex. It flew around campus at UIUC.. https://ieeexplore.ieee.org/abstract/document/5509671/ https://ieeexplore.ieee.org/abstract/document/5509671/ Would be skeptical they could do this in a battlefield situation, we always found you had to be very still to not create surface electric potentials which drown out EEG.
- akomtu 5y ago
  Do you know if there's a detectable em signal when the subject alternates two images or colors in imagination? Say, black-red or black-white. The two colors would supposedly correspond to different energy levels in the brain and that would generate the em wave. The idea was inspired by an old trick: even without wifi hardware, a processor can emit a em wave by alternating between two clockrates.
  - mrtnmcc 5y ago
    From my experience, visual imagery / imagination is not discernable in EEG, it simply doesn't produce large scale modulations like the motor cortex does. However with MRI or fMRI I believe it is possible, and have seen papers even reconstructing visualized numbers.
    - mrtnmcc 5y ago
      Following up after more thought, we had seen 'tongue motor imagery' (imagination of moving tongue) as clearly discernible in EEG. The motor cortex also has eye muscle mappings nearby. http://mrtnmcc.com/wp-content/uploads/2013/12/motor_imagery.png http://mrtnmcc.com/wp-content/uploads/2013/12/motor_imagery.... So it is conceivable that motor control of the eye or imagination of it could be detected. In the case of imagining dark or light perhaps this could affect pupil/iris dilation muscles.
      - akomtu 5y ago
        What range of em spectrum is analysed and what's the sensitivity? I suspect that motor commands produce a blunt low frequency signal, while emotions and thoughts produce something barely discernible.
dang 5y ago
Url changed from https://idstch.com/technology/biosciences/darpa-n3-developing-nonsurgical-brain-machine-interfaces-for-soldiers-to-use-his-thoughts-alone-to-control-multiple-unmanned-vehicles-or-a-bomb-disposal-robot-on-battlefield/ https://idstch.com/technology/biosciences/darpa-n3-developin..., which points to this (and does nothing else). Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Readers: note that the some of the comments are complaining about the earlier submission, or rather its title ("DARPA Developed Nonsurgical Brain Interfaces to Control Drones Using Thoughts")—unsurprisingly, since that's literally all that was there—other than ads, of course.
sn41 5y ago
Won't this mean that you cannot think of anything else while operating the equipment?
- omgitsabird 5y ago
  Do you need to not think of anything else to move your arms?
  - SavantIdiot 5y ago
    Not until you said that. Thanks.
  - AussieWog93 5y ago
    Ex-BCI research student here. The nerves that control your arm are wired directly into your central nervous system and can active your muscles in direct response to action potentials fanned out from a single neuron. These noninvasive technologies pick up aggregate signals from hundreds of thousands of neurons all firing pseudo-randomly, filtered spatially by the dura (thick layer of membrane that protects the brain), skull, flesh, skin and hair. A more "HN" analogy would be like comparing a directly wired Ethernet connection (nerves connected to the CNS) to a side channel attack on an air-gapped system (noninvasive EEG). While randomly running background processes (normally) won't affect the data transmitted to the network, they will absolutely foil most side channel attacks.
    - zaptrem 5y ago
      This is a great analogy.
    - westurner 5y ago
      What about with realtime NIRS with an (inverse?) scattering matrix? From https://www.openwater.cc/technology https://www.openwater.cc/technology : > Below are examples of the image quality we have achieved with our breakthrough scanning systems that use just red and near-infrared light and ultrasound pings. https://en.wikipedia.org/wiki/Near-infrared_spectroscopy https://en.wikipedia.org/wiki/Near-infrared_spectroscopy Another question: is it possible to do ah molecular identification similar to idk quantum crystallography with photons of any wavelength, such as NIRS? Could that count things in samples? https://twitter.com/westurner/status/1239012387367387138 https://twitter.com/westurner/status/1239012387367387138 : > ... quantum crystallography: https://en.wikipedia.org/wiki/Quantum_crystallography https://en.wikipedia.org/wiki/Quantum_crystallography There's probably some limit to infrared crystallography that anyone who knows anything about particles and lattices would know about ?
pingeroo 5y ago
Is this real?
- justinclift 5y ago
  Not yet. The whole article is about future stuff being researched, but without any actual results yet. Whoever submitted this went with a click bait title that doesn't reflect the article.
  - graderjs 5y ago
    Unfortunately there seems to have been a bit of that on the front page lately. Like the article yesterday about how Australia had decided to implement social credit which unfortunately again was something just like a half year old parliamentary committee proposal that wasn't anywhere near becoming law.
    - mattkrause 5y ago
      This is perhaps a little different, because the drone angle actually did come from DARPA. We wrote a proposal for the same program, and it specifically called out "flying a drone [in VR]" as a way that they wanted to see your system's capabilities demonstrated. The real aims of the program were to "read out" signals from the brain that would let you control an external device. Depending on the phase and level of invasiveness, these signals were supposed to contain three, five, or ten degrees of freedom. It was also supposed to have a fairly low decoding latency (~100 ms). At the end of the program, a human participant was supposed to demonstrate the system by performing a DOD-relevant task with it. These requirements map nicely onto moving something through space: 3DOF corresponds to x/y/z; add in yaw and pitch to get to five, and a second object gets you ten.
      - graderjs 5y ago
        That's awesome! Thanks for sharing :) I think it was just the original title that seemed off -- suggesting they're already built it ;p :) xx
black_13 5y ago
And who needs this?
twarge 5y ago
Here's a USB-powered unshielded MEG sensor that you can just carry out into the woods https://arxiv.org/abs/2001.03534 https://arxiv.org/abs/2001.03534 So that's new.
lawrenceyan 5y ago
Are there any ongoing clinical trials? This definitely looks like groundbreaking research. What does the safety profile look like in comparison to using implanted devices to treat epilepsy, etc.
- lawrenceyan 5y ago
  Did some searching myself, found this pretty useful: https://neuro.psychiatryonline.org/doi/10.1176/appi.neuropsych.17110262 https://neuro.psychiatryonline.org/doi/10.1176/appi.neuropsy...

83 points by grawprog 5y ago

The title is misleading. DARPA has funded teams to work on minimally invasive brain interfaces, but AFAIK they have yet to be achieved.

AussieWog93 5y ago
And as someone who studied this field at a PhD level, I would be shocked if they managed to achieve a level of control on par with even a standard Xbox controller any time within the next two decades. The whole field is propelled by deliberate misinformation and hype used to farm grants.
- graderjs 5y ago
  Really, is it really so hard? I had the impression from seeing those videos of people operating robotic limbs through biofeedback or typing through biofeedback or the write-ups on that AI related work to determine words from brainwave patterns that a lot of this stuff was achieving impressive capabilities. but maybe the sort of sensory bandwidth like required to operate say a helicopter is just a completely other order of magnitude to think that you could do it through the same channel. What's the main blockers or bottlenecks in this space?
  - dexwiz 5y ago
    I would imagine some of those videos are the hype and misinformation that the GP is talking about
    - AussieWog93 5y ago
      Yes, a lot of the videos shown (such as the pig walking cycle in the Neuralink presentation) deliberately misrepresent the challenges, limitations and lack of progress with the technology in an attempt to get more funding. I guess you could argue that people writing the grants should perform more due diligence, but at the same time I believe we should be able to trust scientists to act ethically and honestly in the same way we trust doctors.
      - graderjs 5y ago
        At the same time I reckon we should throw money at stuff even half- (not baked, but, half-) completed...because the pay-off is high (if it works) even if the current state of art right now is weak. I get your frustration at the perceived support for a research direction you view as wrong, but I think this is normal, especially in emerging tech. There are often competing approaches, and funding favorites. But I didn't know this--what you said about the difficulty of signal collection, and rejection--and really value that alternative perspective to the one I've seen in the press about using AI, which you think is overblown. Thank you! :)
        AussieWog93 5y ago
        >At the same time I reckon we should throw money at stuff even half- (not baked, but, half-) completed...because the pay-off is high (if it works) even if the current state of art right now is weak. I don't disagree with that line of reasoning in general - moonshot projects have led to amazing discoveries in the past and should definitely be funded. In this particular case, though, there are both very, very strong arguments to suggest that we reached the peak of what the current paradigm can achieve back in the late 00's - as well as very, very strong (financial and cultural) pressure to ensure all new research projects in the field adhere to the current paradigm. >I get your frustration at the perceived support for a research direction you view as wrong, but I think this is normal, especially in emerging tech. It's so, so much deeper than that. The researchers in the area know full well what they're doing is complete bullshit, but toe the line anyway because that's where the funding comes from. It's not just a wrong research direction but a soft and invisible form of academic fraud. This is not normal (or at least shouldn't be normal), and we should not be feeding the beast. We could possibly have much, much stronger BCI tech today if the incumbent culture weren't actively impeding innovation.
        graderjs 5y ago
        Wow, man. Thanks for sharing more how you feel about it. I thought it was just equal competing approaches, different camps. But you tell a different story. I really appreciate that. That's not just fraud, I think...but it's like wasted scientific capacity. You know? They're researchers, they could be working to build something, but instead they're taking money on an approach that can't work. That's sick. Feels bad.... So what do you think's the solution? Like if you could take 10 million dollars to do your own lab with BCI tech what direction would you go in? ;p ;) xx
  - grp000 5y ago
    I thought a lot of those videos worked off of nerve impulses at the terminated end of the limb.
    - AussieWog93 5y ago
      The guy with the robotic arm that looks like Bryan Cranston (can't remember his name off the top of my head) definitely uses nerve endings at the end of the limb. Others, such as the patient in Krishna Shenoy's 2017 paper, though, use invasive BCIs.
  - AussieWog93 5y ago
    >I had the impression from seeing those videos of people operating robotic limbs through biofeedback or typing through biofeedback or the write-ups on that AI related work to determine words from brainwave patterns that a lot of this stuff was achieving impressive capabilities. Sticking to BCI specifically and not other forms for biofeedback, most of those demos you see come from people with invasive, penetrative electrode arrays such as the Utah array. In some people, these sensors can enable a high level of control and performance (not Xbox controller level, but still Atari controller level which would be incredibly useful for people with Motor Neurone Disease). Unfortunately the body rejects them after a few months and they're back to square one. To be honest with you, I don't think that the problem's intractable. Perhaps a clever materials engineer could develop a coating for the electrode arrays that the body accepts long-term and allows the user to train for years rather than months. Perhaps we could develop a new form of sensor altogether. I don't know specifically what the answer is, but it won't come from the current paradigm of "turn garbage data into non-garbage output through the power of Machine Learning somehow" that we've been stuck in for the better part of two decades. And unfortunately, projects adhering to that paradigm is where all of the funding (and people) goes to.
    - cinquemb 5y ago
      > "turn garbage data into non-garbage output through the power of Machine Learning somehow" Yeah, i was pretty turned off by this in the lab i've worked at/contracted for. A lot of support for this kind of stuff wrt EEF neurofeedback while ignoring the physics of what could be done with the raw data (if the PI actually understands or had a E&M class at least). Ppl in charge of getting the grants don't really care about the fact we do not gain any insight into whats going on when we use black box systems… oh well. I'd be happy to play around with biosemi 128 electrode cap+array + dsp + beamformers[0] and openemu[1] for two decades, deff not gonna happen in academia though. [0] https://github.com/cinquemb/LCMVBeamformer https://github.com/cinquemb/LCMVBeamformer [1] https://github.com/cinquemb/OpenEmu https://github.com/cinquemb/OpenEmu
    - kodah 5y ago
      I did cognitive neurotherapy after a series of TBIs. The therapy seemed to work on me, though, what I've heard is that it's hit and miss. Of course, there's always the chance that I'd have recovered that way anyway, but given the state my academics were in I doubt it. For reference, the machine I used employed a bike game that supported movement across the screen. I eventually could make the bike go faster as well as up and down, nearly on command. It focused on alpha, beta, and theta waves measured by diodes stuck to your scalp. Not sure if that narrows that down, but I'm curious to know your thoughts.
- pyinstallwoes 5y ago
  Yeah, but have you worked at the DARPA level?

mrtnmcc 5y ago

I'd written an EEG signal processing and inference backend for a drone control system that did this in 2010. It picks up on Mu rhythms from the motor cortex. It flew around campus at UIUC.. https://ieeexplore.ieee.org/abstract/document/5509671/ https://ieeexplore.ieee.org/abstract/document/5509671/ Would be skeptical they could do this in a battlefield situation, we always found you had to be very still to not create surface electric potentials which drown out EEG.

akomtu 5y ago
Do you know if there's a detectable em signal when the subject alternates two images or colors in imagination? Say, black-red or black-white. The two colors would supposedly correspond to different energy levels in the brain and that would generate the em wave. The idea was inspired by an old trick: even without wifi hardware, a processor can emit a em wave by alternating between two clockrates.
- mrtnmcc 5y ago
  From my experience, visual imagery / imagination is not discernable in EEG, it simply doesn't produce large scale modulations like the motor cortex does. However with MRI or fMRI I believe it is possible, and have seen papers even reconstructing visualized numbers.
  - mrtnmcc 5y ago
    Following up after more thought, we had seen 'tongue motor imagery' (imagination of moving tongue) as clearly discernible in EEG. The motor cortex also has eye muscle mappings nearby. http://mrtnmcc.com/wp-content/uploads/2013/12/motor_imagery.png http://mrtnmcc.com/wp-content/uploads/2013/12/motor_imagery.... So it is conceivable that motor control of the eye or imagination of it could be detected. In the case of imagining dark or light perhaps this could affect pupil/iris dilation muscles.
    - akomtu 5y ago
      What range of em spectrum is analysed and what's the sensitivity? I suspect that motor commands produce a blunt low frequency signal, while emotions and thoughts produce something barely discernible.

dang 5y ago

Url changed from https://idstch.com/technology/biosciences/darpa-n3-developing-nonsurgical-brain-machine-interfaces-for-soldiers-to-use-his-thoughts-alone-to-control-multiple-unmanned-vehicles-or-a-bomb-disposal-robot-on-battlefield/ https://idstch.com/technology/biosciences/darpa-n3-developin..., which points to this (and does nothing else). Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Readers: note that the some of the comments are complaining about the earlier submission, or rather its title ("DARPA Developed Nonsurgical Brain Interfaces to Control Drones Using Thoughts")—unsurprisingly, since that's literally all that was there—other than ads, of course.

sn41 5y ago

Won't this mean that you cannot think of anything else while operating the equipment?

omgitsabird 5y ago
Do you need to not think of anything else to move your arms?
- SavantIdiot 5y ago
  Not until you said that. Thanks.
- AussieWog93 5y ago
  Ex-BCI research student here. The nerves that control your arm are wired directly into your central nervous system and can active your muscles in direct response to action potentials fanned out from a single neuron. These noninvasive technologies pick up aggregate signals from hundreds of thousands of neurons all firing pseudo-randomly, filtered spatially by the dura (thick layer of membrane that protects the brain), skull, flesh, skin and hair. A more "HN" analogy would be like comparing a directly wired Ethernet connection (nerves connected to the CNS) to a side channel attack on an air-gapped system (noninvasive EEG). While randomly running background processes (normally) won't affect the data transmitted to the network, they will absolutely foil most side channel attacks.
  - zaptrem 5y ago
    This is a great analogy.
  - westurner 5y ago
    What about with realtime NIRS with an (inverse?) scattering matrix? From https://www.openwater.cc/technology https://www.openwater.cc/technology : > Below are examples of the image quality we have achieved with our breakthrough scanning systems that use just red and near-infrared light and ultrasound pings. https://en.wikipedia.org/wiki/Near-infrared_spectroscopy https://en.wikipedia.org/wiki/Near-infrared_spectroscopy Another question: is it possible to do ah molecular identification similar to idk quantum crystallography with photons of any wavelength, such as NIRS? Could that count things in samples? https://twitter.com/westurner/status/1239012387367387138 https://twitter.com/westurner/status/1239012387367387138 : > ... quantum crystallography: https://en.wikipedia.org/wiki/Quantum_crystallography https://en.wikipedia.org/wiki/Quantum_crystallography There's probably some limit to infrared crystallography that anyone who knows anything about particles and lattices would know about ?

pingeroo 5y ago

Is this real?

justinclift 5y ago
Not yet. The whole article is about future stuff being researched, but without any actual results yet. Whoever submitted this went with a click bait title that doesn't reflect the article.
- graderjs 5y ago
  Unfortunately there seems to have been a bit of that on the front page lately. Like the article yesterday about how Australia had decided to implement social credit which unfortunately again was something just like a half year old parliamentary committee proposal that wasn't anywhere near becoming law.
  - mattkrause 5y ago
    This is perhaps a little different, because the drone angle actually did come from DARPA. We wrote a proposal for the same program, and it specifically called out "flying a drone [in VR]" as a way that they wanted to see your system's capabilities demonstrated. The real aims of the program were to "read out" signals from the brain that would let you control an external device. Depending on the phase and level of invasiveness, these signals were supposed to contain three, five, or ten degrees of freedom. It was also supposed to have a fairly low decoding latency (~100 ms). At the end of the program, a human participant was supposed to demonstrate the system by performing a DOD-relevant task with it. These requirements map nicely onto moving something through space: 3DOF corresponds to x/y/z; add in yaw and pitch to get to five, and a second object gets you ten.
    - graderjs 5y ago
      That's awesome! Thanks for sharing :) I think it was just the original title that seemed off -- suggesting they're already built it ;p :) xx

black_13 5y ago

And who needs this?

twarge 5y ago

Here's a USB-powered unshielded MEG sensor that you can just carry out into the woods https://arxiv.org/abs/2001.03534 https://arxiv.org/abs/2001.03534 So that's new.

lawrenceyan 5y ago

Are there any ongoing clinical trials? This definitely looks like groundbreaking research. What does the safety profile look like in comparison to using implanted devices to treat epilepsy, etc.

lawrenceyan 5y ago
Did some searching myself, found this pretty useful: https://neuro.psychiatryonline.org/doi/10.1176/appi.neuropsych.17110262 https://neuro.psychiatryonline.org/doi/10.1176/appi.neuropsy...