5 ms·

Could someone explain to a dummy like me: what are the actual technological bottlenecks for brain-machine interfaces? Like... setting aside all the things that

Could someone explain to a dummy like me: what are the actual technological bottlenecks for brain-machine interfaces? Like... setting aside all the things that, given enough time/effort, we'll reach some useful maximum. I'm told that with ML (eg: image classification) we'll eventually train the systems enough that they'll do a pretty amazing job. So is it converting analog brain signals to digital? Is the rate of data transmission even relevant here? What would happen if we had enough "brain data" from a single person to saturate a 10 gigabit network? Do we have the software to do anything meaningful with that?

judge2020 5y ago
> What would happen if we had enough "brain data" from a single person to saturate a 10g network? Upgrade to USB 4 :)
- 0xFACEFEED 5y ago
  Derp. Corrected to 10 gigabit.
  - mattkrause 5y ago
    This is not too far off what experimental systems already do. My lab has equipment that collects data from 288 electrodes in the brain, each at 16 bits/sample x 30,000 samples/sec. This works out to 140 mbps, not counting overhead or other types of data we collect, and it’s not abnormally large. If we had a research question that required it, the vendor sells versions with up to 512 channels—-and you can gang them together for even more. However, this probably isn’t the direction BCI is headed. A lot of the signal is redundant. Some of this is because nearby neurons tend to do very similar things, and some of it is because signals propagate pretty well through the brain so electrodes pick up the same signal in different places. As a result, most non-research applications don’t need all of that data and it’s increasingly possible to do some preprocessing right at the brain: folks at Imperial have made all kinds of cool ASICs that extract spikes.
pama 5y ago
Our brain was not built to directly transmit signals nor to accept input wires to read / write signals. I am sure one or the other can eventually be overcome safely. But we are not there yet.
- ryandvm 5y ago
  The corpus callosum is pretty much exactly a signal bus between the two hemispheres. Granted, it's a 200-million wire bus, but it basically transmits entire "thoughts" between the hemispheres. I think the dream brain interface would be a tap on the CC.
tbenst 5y ago
The big challenge for non-invasive BCI is that most interesting information content in the brain (e.g. speech) is encoded in high frequency firing activity, and the skull acts as a low-pass filter.
- anonymouse008 5y ago
  This is brilliance in a sentence - best of luck in your pursuits.
- Johnythree 5y ago
  There are no high frequency signals in the brain. It's all relatively slow electro-chemical reactions. There are however a huge number of slow signals running in parallel. In total these would add up to a huge bandwidth if constrained on a single channel. To put it another way, the scull is almost entirely resistive. And you can't make a low-pass-filter with just resistors.
  - tbenst 5y ago
    I’m using high frequency in a domain specific way here: neuroscientists consider gamma (30-100Hz) to be high (temporal)frequency for action potential activity. But I wasn’t just referring to temporal filtering—the skull acts as a low-pass spatial filter, too [1]. [1] https://pubmed.ncbi.nlm.nih.gov/8813415/ https://pubmed.ncbi.nlm.nih.gov/8813415/
didntknowya 5y ago
the main problem is not software, AI, etc. It's that we can't get the data reliably- i.e. current sensors are not sensitive enough to pick up brainwaves/activity (esp with interference from muscle etc). The gold standard is electrodes implanted in a brain but is a bit invasive as an interface.
ebb_earl_co 5y ago
Like all of WaitButWhy's articles, I think this one [0] helps a person understand from the fundamentals. I linked to part 3 which is the section for Brain-Machine Interfaces in particular, but it behooves you to read from the beginning [0] https://waitbutwhy.com/2017/04/neuralink.html#part3 https://waitbutwhy.com/2017/04/neuralink.html#part3
devindotcom 5y ago
Right now the state of the art options are either surgically implanted electrode arrays (invasive and very limited in what they can detect), electroencephalogram type helmets (non-invasive but only get very vague signals), or fMRI type imaging (more precise but still only gross detail, and requires an enormous complex machine). There's no obvious way forward with any of these that produces what you or I might consider a true brain machine interface. We don't have the tech AND we don't understand the brain enough. Fortunately you don't need a brain-reading device to produce something useful, just like you don't need a teraflop computer to go to the moon. I've written recently about an EEG helmet that can be used by profoundly disabled folks to navigate a UI, type, and so on, and that doesn't require a precise signal at all. So I think what you'll find is while the Musks of the world are chasing a sci-fi dream of what they think the technology ought to be, most of the utility will come out of using what it's actually capable of in a smart and compassionate way.
- echelon 5y ago
  > Fortunately you don't need a brain-reading device to produce something useful If we're ever to achieve any measure of "immortality", BCI is probably the only way. We could clone monoclonal, brainless humans with universal HLA haplotypes for spare parts assuming we could get past the religious/ick factor. Beyond just organ harvesting, a full body (ie head) transplant could rejuvenate the immune system and potentially reverse many of the effects of aging assuming we could get past central tolerance. (Maybe not an issue with immunosuppressants or monoclonal lines without B/T cells, but that sucks and I think it can be optimized.) This would be the best next step in increasing human lifespan dramatically. It wouldn't save you from physical accident-induced death or irreversible brain atrophy, though. Completely sci-fi conjecture: If we could build clusters of machines capable of running the same distributed work that the human brain does in real time, and if we could get enough signal out of the brain through much more advanced (and invasive) instruments, we might be able to model a subset of a person's memories and run them at real time. If instead of just copying signal, perhaps we could leave a human hooked up and "amplify" their capacity for thinking by supplementing their brain with a computer. We might be able to copy memories into the new computerized system while the person is still alive and thinking. If the way consciousness works is amenable to such a process, we might be able to perform a one way "move" operation. Essentially digitizing a person as a destructive operation, and killing the body when the process is complete. Of course we're not anywhere close to anything like this. It's all hypothetical and not at all close to the science. Cloning humans though: entirely within our capacity. We'd see incredible medical benefits in doing it, including substantial lifespan increases if we routinely replace older parts sourced from clones. People are just too religious. (Monoclonal brainless humans would lack a consciousness. They're not much different from plants in that respect. Entirely ethical to use for parts and experiments.)
dr_dshiv 5y ago
So many challenges. One is that we can only receive information from certain parts of the brain—those where the neurons are perpendicular to the scalp. As you know, the brain is really convoluted, so this makes for a very messy signal. Another issue is that the raw noise makes it extremely difficult to detect faint signals. An approach for this is training models that have expectations based on the stimuli themselves; e.g., to find neural resonance with different frequencies in the stimulus.
uuidgen 5y ago
The brain is more like a CPU implemented in FPGA than a CPU synthesised into silicon. Signals are only somewhat local - you get increased activity in this or that region and can correlate it with what happens in the brain - like power analysis attacks on crypto chips - the algorithm has to recover the data from the limited, noisy signal we are able to pickup. You can't just put electrode in every single neuron to read its state like you can't put an electrode into every single piece of conductive metal on a chip - it would be to tight to fit and everything would stop working.
- TheOtherHobbes 5y ago
  Which is a problem, because a no compromise BCI requires access to a large percentage of neurons with both read and write. More realistically it requires access to every single element inside those neurons. Otherwise you're just poking at things in concrete gloves. Not only is there no technology for this, there isn't even a foundation which could be developed to create the technology.
mattkrause 5y ago
All of it and none of it. Extracting a control signal with few degrees of freedom from an EEG signal is doable right now. A reasonably motivated undergrad with a few hundred bucks of OpenBCI gear should be able to get “MindPong” up and running pretty quickly. But…it’ll be a little janky: the signal quality won’t be great, especially if you’re out in the real world, trying to do normal stuff at the same time. The content of that signal is also limited, in part due to the skull and scalp that sit between your electrode in the brain. This also make it difficult but, surprisingly, not impossible to perturb the brain with electric current (tDCS, tACS), magnets (TMS), or other techniques. What about putting something inside the brain itself? We know information is likely much more accessible without the skull/scalp filtering, but we don’t completely understand how information is represented in the brain, how to modify those representations, or even really how to get the raw data out: most neural implants have a pretty short lifetime before they’re ruined by the immune system, mechanical strain, etc. We’re not totally ignorant: there’s been some amazing progress decoding motor and speech intentions and, after a 20+ year hiatus, cool new electrode technology (Pandromics, Neuralink, etc) but there’s a lot to be discovered and invented. To sum up, we have some fairly crude stuff working now, but it’s looking for a killer app, especially in humans. Building something more like a movie requires work on many different fronts, ranging from materials science to build the electrodes to neuro/ML to understand what those electrodes see.
- sterlind 5y ago
  I feel like the "killer app" is accessibility. There's so many disabilities (quadriplegia, SMA, amputations, stroke, ALS etc.) where having a BCI that lets you reliably control a wheelchair would be a godsend. I always see futurology/inspiration porn articles about impressive demos of this stuff, when is it hitting the market? It doesn't have to be the next iPhone, it just has to let people control their mobility or communicate.
  - mattkrause 5y ago
    “Reliably” is the tricky bit. Accessibility gear needs to be robust, but a lot of the BCIs, especially the noninvasive ones, tend to be a bit janky. As a result, you can zip around in VR once the experienced lab tech sets you up, but maybe don’t have the DOF to control a real wheelchair or a system that disabled people can set up themselves. Some of this is probably just need some good systems engineering, but there are some legit technical challenges too. I think this will start to change soon: there’s a lot of money flowing into neurotech and hopefully, some of it will end up with people who are more serious than hype-y. (If nothing else, I’d like a job :-))
aheilbut 5y ago
We already have some excellent, high-throughput brain-machine interfaces: they’re called screens, keyboards, microphones, and cameras.
- suby 5y ago
  This isn't a helpful comment. There are many valid use cases for wanting a direct brain to computer interface, disability and injury chief among them.
morphle 5y ago
see the lectures by Mary Lou Jepson at https://www.openwater.cc/ https://www.openwater.cc/ She is building the state of the art brain scanner and neuron read/write with 50 chips

judge2020 5y ago

> What would happen if we had enough "brain data" from a single person to saturate a 10g network? Upgrade to USB 4 :)

0xFACEFEED 5y ago
Derp. Corrected to 10 gigabit.
- mattkrause 5y ago
  This is not too far off what experimental systems already do. My lab has equipment that collects data from 288 electrodes in the brain, each at 16 bits/sample x 30,000 samples/sec. This works out to 140 mbps, not counting overhead or other types of data we collect, and it’s not abnormally large. If we had a research question that required it, the vendor sells versions with up to 512 channels—-and you can gang them together for even more. However, this probably isn’t the direction BCI is headed. A lot of the signal is redundant. Some of this is because nearby neurons tend to do very similar things, and some of it is because signals propagate pretty well through the brain so electrodes pick up the same signal in different places. As a result, most non-research applications don’t need all of that data and it’s increasingly possible to do some preprocessing right at the brain: folks at Imperial have made all kinds of cool ASICs that extract spikes.

pama 5y ago

Our brain was not built to directly transmit signals nor to accept input wires to read / write signals. I am sure one or the other can eventually be overcome safely. But we are not there yet.

ryandvm 5y ago
The corpus callosum is pretty much exactly a signal bus between the two hemispheres. Granted, it's a 200-million wire bus, but it basically transmits entire "thoughts" between the hemispheres. I think the dream brain interface would be a tap on the CC.

tbenst 5y ago

The big challenge for non-invasive BCI is that most interesting information content in the brain (e.g. speech) is encoded in high frequency firing activity, and the skull acts as a low-pass filter.

anonymouse008 5y ago
This is brilliance in a sentence - best of luck in your pursuits.
Johnythree 5y ago
There are no high frequency signals in the brain. It's all relatively slow electro-chemical reactions. There are however a huge number of slow signals running in parallel. In total these would add up to a huge bandwidth if constrained on a single channel. To put it another way, the scull is almost entirely resistive. And you can't make a low-pass-filter with just resistors.
- tbenst 5y ago
  I’m using high frequency in a domain specific way here: neuroscientists consider gamma (30-100Hz) to be high (temporal)frequency for action potential activity. But I wasn’t just referring to temporal filtering—the skull acts as a low-pass spatial filter, too [1]. [1] https://pubmed.ncbi.nlm.nih.gov/8813415/ https://pubmed.ncbi.nlm.nih.gov/8813415/

didntknowya 5y ago

the main problem is not software, AI, etc. It's that we can't get the data reliably- i.e. current sensors are not sensitive enough to pick up brainwaves/activity (esp with interference from muscle etc). The gold standard is electrodes implanted in a brain but is a bit invasive as an interface.

ebb_earl_co 5y ago

Like all of WaitButWhy's articles, I think this one [0] helps a person understand from the fundamentals. I linked to part 3 which is the section for Brain-Machine Interfaces in particular, but it behooves you to read from the beginning [0] https://waitbutwhy.com/2017/04/neuralink.html#part3 https://waitbutwhy.com/2017/04/neuralink.html#part3

devindotcom 5y ago

Right now the state of the art options are either surgically implanted electrode arrays (invasive and very limited in what they can detect), electroencephalogram type helmets (non-invasive but only get very vague signals), or fMRI type imaging (more precise but still only gross detail, and requires an enormous complex machine). There's no obvious way forward with any of these that produces what you or I might consider a true brain machine interface. We don't have the tech AND we don't understand the brain enough. Fortunately you don't need a brain-reading device to produce something useful, just like you don't need a teraflop computer to go to the moon. I've written recently about an EEG helmet that can be used by profoundly disabled folks to navigate a UI, type, and so on, and that doesn't require a precise signal at all. So I think what you'll find is while the Musks of the world are chasing a sci-fi dream of what they think the technology ought to be, most of the utility will come out of using what it's actually capable of in a smart and compassionate way.

echelon 5y ago
> Fortunately you don't need a brain-reading device to produce something useful If we're ever to achieve any measure of "immortality", BCI is probably the only way. We could clone monoclonal, brainless humans with universal HLA haplotypes for spare parts assuming we could get past the religious/ick factor. Beyond just organ harvesting, a full body (ie head) transplant could rejuvenate the immune system and potentially reverse many of the effects of aging assuming we could get past central tolerance. (Maybe not an issue with immunosuppressants or monoclonal lines without B/T cells, but that sucks and I think it can be optimized.) This would be the best next step in increasing human lifespan dramatically. It wouldn't save you from physical accident-induced death or irreversible brain atrophy, though. Completely sci-fi conjecture: If we could build clusters of machines capable of running the same distributed work that the human brain does in real time, and if we could get enough signal out of the brain through much more advanced (and invasive) instruments, we might be able to model a subset of a person's memories and run them at real time. If instead of just copying signal, perhaps we could leave a human hooked up and "amplify" their capacity for thinking by supplementing their brain with a computer. We might be able to copy memories into the new computerized system while the person is still alive and thinking. If the way consciousness works is amenable to such a process, we might be able to perform a one way "move" operation. Essentially digitizing a person as a destructive operation, and killing the body when the process is complete. Of course we're not anywhere close to anything like this. It's all hypothetical and not at all close to the science. Cloning humans though: entirely within our capacity. We'd see incredible medical benefits in doing it, including substantial lifespan increases if we routinely replace older parts sourced from clones. People are just too religious. (Monoclonal brainless humans would lack a consciousness. They're not much different from plants in that respect. Entirely ethical to use for parts and experiments.)

dr_dshiv 5y ago

So many challenges. One is that we can only receive information from certain parts of the brain—those where the neurons are perpendicular to the scalp. As you know, the brain is really convoluted, so this makes for a very messy signal. Another issue is that the raw noise makes it extremely difficult to detect faint signals. An approach for this is training models that have expectations based on the stimuli themselves; e.g., to find neural resonance with different frequencies in the stimulus.

uuidgen 5y ago

The brain is more like a CPU implemented in FPGA than a CPU synthesised into silicon. Signals are only somewhat local - you get increased activity in this or that region and can correlate it with what happens in the brain - like power analysis attacks on crypto chips - the algorithm has to recover the data from the limited, noisy signal we are able to pickup. You can't just put electrode in every single neuron to read its state like you can't put an electrode into every single piece of conductive metal on a chip - it would be to tight to fit and everything would stop working.

TheOtherHobbes 5y ago
Which is a problem, because a no compromise BCI requires access to a large percentage of neurons with both read and write. More realistically it requires access to every single element inside those neurons. Otherwise you're just poking at things in concrete gloves. Not only is there no technology for this, there isn't even a foundation which could be developed to create the technology.

mattkrause 5y ago

All of it and none of it. Extracting a control signal with few degrees of freedom from an EEG signal is doable right now. A reasonably motivated undergrad with a few hundred bucks of OpenBCI gear should be able to get “MindPong” up and running pretty quickly. But…it’ll be a little janky: the signal quality won’t be great, especially if you’re out in the real world, trying to do normal stuff at the same time. The content of that signal is also limited, in part due to the skull and scalp that sit between your electrode in the brain. This also make it difficult but, surprisingly, not impossible to perturb the brain with electric current (tDCS, tACS), magnets (TMS), or other techniques. What about putting something inside the brain itself? We know information is likely much more accessible without the skull/scalp filtering, but we don’t completely understand how information is represented in the brain, how to modify those representations, or even really how to get the raw data out: most neural implants have a pretty short lifetime before they’re ruined by the immune system, mechanical strain, etc. We’re not totally ignorant: there’s been some amazing progress decoding motor and speech intentions and, after a 20+ year hiatus, cool new electrode technology (Pandromics, Neuralink, etc) but there’s a lot to be discovered and invented. To sum up, we have some fairly crude stuff working now, but it’s looking for a killer app, especially in humans. Building something more like a movie requires work on many different fronts, ranging from materials science to build the electrodes to neuro/ML to understand what those electrodes see.

sterlind 5y ago
I feel like the "killer app" is accessibility. There's so many disabilities (quadriplegia, SMA, amputations, stroke, ALS etc.) where having a BCI that lets you reliably control a wheelchair would be a godsend. I always see futurology/inspiration porn articles about impressive demos of this stuff, when is it hitting the market? It doesn't have to be the next iPhone, it just has to let people control their mobility or communicate.
- mattkrause 5y ago
  “Reliably” is the tricky bit. Accessibility gear needs to be robust, but a lot of the BCIs, especially the noninvasive ones, tend to be a bit janky. As a result, you can zip around in VR once the experienced lab tech sets you up, but maybe don’t have the DOF to control a real wheelchair or a system that disabled people can set up themselves. Some of this is probably just need some good systems engineering, but there are some legit technical challenges too. I think this will start to change soon: there’s a lot of money flowing into neurotech and hopefully, some of it will end up with people who are more serious than hype-y. (If nothing else, I’d like a job :-))

aheilbut 5y ago

We already have some excellent, high-throughput brain-machine interfaces: they’re called screens, keyboards, microphones, and cameras.

suby 5y ago
This isn't a helpful comment. There are many valid use cases for wanting a direct brain to computer interface, disability and injury chief among them.

morphle 5y ago

see the lectures by Mary Lou Jepson at https://www.openwater.cc/ https://www.openwater.cc/ She is building the state of the art brain scanner and neuron read/write with 50 chips