6 ms·

I don't understand how you can gather such a targeted signal by sticking wires in just a few places. Like, you can't just tape a metal coat hanger to an iPhone

I don't understand how you can gather such a targeted signal by sticking wires in just a few places. Like, you can't just tape a metal coat hanger to an iPhone and start getting text messages through it. But somehow you can put some wires just as few mm into the surface of a brain and extract fine motor signals??

swordsmith 5y ago
The location of the electrodes aren't arbitrary. The region of implantation is likely the hand-motor cortex (which has thickness on the order of mm), the area associated with arm movements (check out somatotopy), so the signals acquired are actually targeted for the demo task. This relationship has been known in neuroscience for almost a century and has been validated in decades of brain-machine interface experiments. They are also recording from up to 1000 channels, which is probably overkill for mind pong, tbh. But you'll need that many implanted electrodes to study long term electrode biocompatibility.
- mattkrause 5y ago
  Even outside of primary motor/somatosensory areas, 2D cursor control is a pretty standard BMI paradigm. Not exactly “Hello, World” but maybe “ToDo List App”.
  - caddemon 5y ago
    Yeah their implant hardware is really impressive but "mind pong" is a pretty common starter project for at home EEG hacking, let alone legit BCI research. The internet is littered with examples of mind pong for the Muse like this one: https://medium.com/@nayvelt.lina/playing-ping-pong-with-my-brain-49025f044b9f https://medium.com/@nayvelt.lina/playing-ping-pong-with-my-b... Granted it moves slower than actual Atari Pong, and takes a few minutes to get the hang of controlling the paddle. But IME it isn't much harder to control the up/down of the paddle with the EEG than it is to play a game like flappy bird, and that's using a couple random scalp electrodes from a consumer device. So yeah thanks for bringing this up, because I feel some of the comments here are acting like this is much closer to "mind reading" than it actually is. Not that it isn't cool, but the overhyping kinda kills it for me lol. Is this how robotics people feel when Boston Dynamics releases a new video?
  - swordsmith 5y ago
    I'd argue it is the "Hello, World" in any BMI lab. Assuming existing implants and electronics, and non-naive monkey, advanced undergrads should get it in a semester.
    - mattkrause 5y ago
      That's why I went with To Do list :-) It's not totally trivial, but a reasonably skilled person should be able to get a not-too-janky version working with a bit of effort. (Also, where are you that undergrads get to interact with monkeys?!)
- jonnycomputer 5y ago
  I think the source of wonder here is that neural networks are very un-CPU like, in the level of correlation between the units. I'm not sure you could probe a CPU in a similar manner and be able to recover meaningful information. A binary 101X10 are entirely different numbers depending on what X is, for example, and you woulnd't necessarily expect X to correlate with the place values elsehwere (e.g. 1st and 5th bit). Similar arguments, apply, to probing different components, such as registers or instruction memory.
  - swordsmith 5y ago
    Funny you should compare understanding biological neural networks to a microprocessor. Check out this paper (https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268 https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...), which systematically addresses how applying traditional neuroscience methods to analyzing a microprocessor would get you.
    - jonnycomputer 5y ago
      I've read that paper years ago. I'm afraid I find it unimpressive. It is not so hard, really, to fail, when you fail to try. I have a computer science degree; I've built CPUs (from all the sequential and combinational circuits on up); I also work in neuroscience, and do some of the kinds of analysis this paper tries to criticize. For example, a very basic approach is design experimental tasks that have contrasting conditions, or events to be predicted. They don't do that. Even something as simple as pressing the controller left vs right vs not moving at all in donkeykong would have been more interesting. In any case, brains and traditional computer circuits work on different principles as I noted in my OP. Furthermore, there is quite a lot more redundancy in brains--and partial-redundancy is a bit of how it works; for example, if you have a population of neurons with different tuning curves, then the output in response to a stimulus can be integrated, and a likelihood distribution obtained for the actual value of a stimulus; that's just fundamentally different than how, say, a 5-stage pipelined RISC cpu works.
ampdepolymerase 5y ago
The most interesting thing here is not the mind reading, it is the miniaturization of computing and sparse data transfer. Neuralink off-loaded the computation to the sensor, sending back aggregate data snapshots. Usually you will need massive ASIC/FPGA computation banks with a lot of messy to handle the data transfer and readout but they took a shortcut bye sending only counts of spikes that exceeds a certain threshold. There is nothing stopping you from using a cloth hangar. You just need to make sure it is impedance matched and properly sanitized for subdermal use. The signal will be much worse as a cloth hangar had gauge/diameter that is several magnitudes bigger than that of a neuron. Nothing here is particularly "novel" that it cannot be trivially replicated if you have the budget of one of the FAANGs. If you have 100million in capital, you can easily find poor neuroscience and EE PhDs who will be willing to work for you for <80K a year just for the opportunity to build something like this. The reason it is difficult and out of reach for smaller companies is because small amplifiers and electronic packages that can be subdermally implanted requires significant manufacturing capital as they fall under semiconductors and biomedical devices. Once the industry settle on a design expect the price to drop dramatically as the core chips and parts become commoditized.
- toomuchtodo 5y ago
  Is there anyway to get this signal non invasively (magnetic field detection or photonically)? Or are implanted electrodes table stakes for a neural interface?
  - lwansbrough 5y ago
    Not really, that we know of. To appreciate the complexity you could read this post[0] from Wait But Why? Your brain has approximately 40,000 neurons in the space of a cubic millimeter. Each saying something completely different. It’s incomprehensible noise unless you start dividing it up like Neuralink has. They get a few thousand probes in the size of a penny, which is a far greater level of resolution than you could probably ever expect from a wireless solution. [0] https://waitbutwhy.com/2017/04/neuralink.html https://waitbutwhy.com/2017/04/neuralink.html
mhh__ 5y ago
MI5 were lifting crypto keys based on keypress acoustics in the 50s, I wouldn't be so certain. If you put the phone up an expensive enough tool you can at very least find side channels
JanSolo 5y ago
My understanding is that neurons are like a higher-order connected mesh. A bit like many fully-connected layers in a Machine Learning model. You can't just read fine motor signals from any neuron, but given enough neurons, since they're deeply connected, you can infer a close approximation of the intended motor signals given a known output. That's why they need a lot of sampling wires in the brain; to makes sure they have enough data to make that inference. Luckily, machine-learning is very good at figuring out patterns in this sort of data. My guess is that they train an ML model using the inputs and outputs when the joystick is connected. Then they can use the trained model to just generate the output directly from the inputs when the joystick is disconnected. This will work fine for one particular monkeybrain/interface combo; however, you cannot use the same model on other monkeys. You have to train a new one each time.
- judge2020 5y ago
  > Like, you can't just tape a metal coat hanger to an iPhone and start getting text messages through it. You could if we didn't encrypt connections. On that note, does anyone know how to upgrade libssl across my entire nervous system?
  - croddin 5y ago
    > On that note, does anyone know how to upgrade libssl across my entire nervous system? ‘Inception’ shows someone who tries to do something like that, but there seems to be a vulnerability.
  - meowface 5y ago
    Or have Bluetooth disabled by default for my brain. The game examples were super fascinating, but since I was already essentially familiar with the fundamental principles and technology, it didn't quite impact me as much as the researcher pairing his iPhone with a Bluetooth transceiver in the monkey's brain. That sent weird shivers down my spine.
- babesh 5y ago
  This reminds me of dropout where the neural network won’t be dependent on a specific connection. Instead, it will be statistical which will actually be a good fit for this approach. https://en.wikipedia.org/wiki/Dilution_(neural_networks) https://en.wikipedia.org/wiki/Dilution_(neural_networks)
NavinF 5y ago
>Like, you can't just tape a metal coat hanger to an iPhone and start getting text messages through it. You can get a 128 key out of most electronic devices in a reasonable amount of time: https://en.wikipedia.org/wiki/Electromagnetic_attack#Known_attacks https://en.wikipedia.org/wiki/Electromagnetic_attack#Known_a...

swordsmith 5y ago

The location of the electrodes aren't arbitrary. The region of implantation is likely the hand-motor cortex (which has thickness on the order of mm), the area associated with arm movements (check out somatotopy), so the signals acquired are actually targeted for the demo task. This relationship has been known in neuroscience for almost a century and has been validated in decades of brain-machine interface experiments. They are also recording from up to 1000 channels, which is probably overkill for mind pong, tbh. But you'll need that many implanted electrodes to study long term electrode biocompatibility.

mattkrause 5y ago
Even outside of primary motor/somatosensory areas, 2D cursor control is a pretty standard BMI paradigm. Not exactly “Hello, World” but maybe “ToDo List App”.
- caddemon 5y ago
  Yeah their implant hardware is really impressive but "mind pong" is a pretty common starter project for at home EEG hacking, let alone legit BCI research. The internet is littered with examples of mind pong for the Muse like this one: https://medium.com/@nayvelt.lina/playing-ping-pong-with-my-brain-49025f044b9f https://medium.com/@nayvelt.lina/playing-ping-pong-with-my-b... Granted it moves slower than actual Atari Pong, and takes a few minutes to get the hang of controlling the paddle. But IME it isn't much harder to control the up/down of the paddle with the EEG than it is to play a game like flappy bird, and that's using a couple random scalp electrodes from a consumer device. So yeah thanks for bringing this up, because I feel some of the comments here are acting like this is much closer to "mind reading" than it actually is. Not that it isn't cool, but the overhyping kinda kills it for me lol. Is this how robotics people feel when Boston Dynamics releases a new video?
- swordsmith 5y ago
  I'd argue it is the "Hello, World" in any BMI lab. Assuming existing implants and electronics, and non-naive monkey, advanced undergrads should get it in a semester.
  - mattkrause 5y ago
    That's why I went with To Do list :-) It's not totally trivial, but a reasonably skilled person should be able to get a not-too-janky version working with a bit of effort. (Also, where are you that undergrads get to interact with monkeys?!)
jonnycomputer 5y ago
I think the source of wonder here is that neural networks are very un-CPU like, in the level of correlation between the units. I'm not sure you could probe a CPU in a similar manner and be able to recover meaningful information. A binary 101X10 are entirely different numbers depending on what X is, for example, and you woulnd't necessarily expect X to correlate with the place values elsehwere (e.g. 1st and 5th bit). Similar arguments, apply, to probing different components, such as registers or instruction memory.
- swordsmith 5y ago
  Funny you should compare understanding biological neural networks to a microprocessor. Check out this paper (https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268 https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...), which systematically addresses how applying traditional neuroscience methods to analyzing a microprocessor would get you.
  - jonnycomputer 5y ago
    I've read that paper years ago. I'm afraid I find it unimpressive. It is not so hard, really, to fail, when you fail to try. I have a computer science degree; I've built CPUs (from all the sequential and combinational circuits on up); I also work in neuroscience, and do some of the kinds of analysis this paper tries to criticize. For example, a very basic approach is design experimental tasks that have contrasting conditions, or events to be predicted. They don't do that. Even something as simple as pressing the controller left vs right vs not moving at all in donkeykong would have been more interesting. In any case, brains and traditional computer circuits work on different principles as I noted in my OP. Furthermore, there is quite a lot more redundancy in brains--and partial-redundancy is a bit of how it works; for example, if you have a population of neurons with different tuning curves, then the output in response to a stimulus can be integrated, and a likelihood distribution obtained for the actual value of a stimulus; that's just fundamentally different than how, say, a 5-stage pipelined RISC cpu works.

ampdepolymerase 5y ago

The most interesting thing here is not the mind reading, it is the miniaturization of computing and sparse data transfer. Neuralink off-loaded the computation to the sensor, sending back aggregate data snapshots. Usually you will need massive ASIC/FPGA computation banks with a lot of messy to handle the data transfer and readout but they took a shortcut bye sending only counts of spikes that exceeds a certain threshold. There is nothing stopping you from using a cloth hangar. You just need to make sure it is impedance matched and properly sanitized for subdermal use. The signal will be much worse as a cloth hangar had gauge/diameter that is several magnitudes bigger than that of a neuron. Nothing here is particularly "novel" that it cannot be trivially replicated if you have the budget of one of the FAANGs. If you have 100million in capital, you can easily find poor neuroscience and EE PhDs who will be willing to work for you for <80K a year just for the opportunity to build something like this. The reason it is difficult and out of reach for smaller companies is because small amplifiers and electronic packages that can be subdermally implanted requires significant manufacturing capital as they fall under semiconductors and biomedical devices. Once the industry settle on a design expect the price to drop dramatically as the core chips and parts become commoditized.

toomuchtodo 5y ago
Is there anyway to get this signal non invasively (magnetic field detection or photonically)? Or are implanted electrodes table stakes for a neural interface?
- lwansbrough 5y ago
  Not really, that we know of. To appreciate the complexity you could read this post[0] from Wait But Why? Your brain has approximately 40,000 neurons in the space of a cubic millimeter. Each saying something completely different. It’s incomprehensible noise unless you start dividing it up like Neuralink has. They get a few thousand probes in the size of a penny, which is a far greater level of resolution than you could probably ever expect from a wireless solution. [0] https://waitbutwhy.com/2017/04/neuralink.html https://waitbutwhy.com/2017/04/neuralink.html

mhh__ 5y ago

MI5 were lifting crypto keys based on keypress acoustics in the 50s, I wouldn't be so certain. If you put the phone up an expensive enough tool you can at very least find side channels

JanSolo 5y ago

My understanding is that neurons are like a higher-order connected mesh. A bit like many fully-connected layers in a Machine Learning model. You can't just read fine motor signals from any neuron, but given enough neurons, since they're deeply connected, you can infer a close approximation of the intended motor signals given a known output. That's why they need a lot of sampling wires in the brain; to makes sure they have enough data to make that inference. Luckily, machine-learning is very good at figuring out patterns in this sort of data. My guess is that they train an ML model using the inputs and outputs when the joystick is connected. Then they can use the trained model to just generate the output directly from the inputs when the joystick is disconnected. This will work fine for one particular monkeybrain/interface combo; however, you cannot use the same model on other monkeys. You have to train a new one each time.

judge2020 5y ago
> Like, you can't just tape a metal coat hanger to an iPhone and start getting text messages through it. You could if we didn't encrypt connections. On that note, does anyone know how to upgrade libssl across my entire nervous system?
- croddin 5y ago
  > On that note, does anyone know how to upgrade libssl across my entire nervous system? ‘Inception’ shows someone who tries to do something like that, but there seems to be a vulnerability.
- meowface 5y ago
  Or have Bluetooth disabled by default for my brain. The game examples were super fascinating, but since I was already essentially familiar with the fundamental principles and technology, it didn't quite impact me as much as the researcher pairing his iPhone with a Bluetooth transceiver in the monkey's brain. That sent weird shivers down my spine.
babesh 5y ago
This reminds me of dropout where the neural network won’t be dependent on a specific connection. Instead, it will be statistical which will actually be a good fit for this approach. https://en.wikipedia.org/wiki/Dilution_(neural_networks) https://en.wikipedia.org/wiki/Dilution_(neural_networks)

NavinF 5y ago

>Like, you can't just tape a metal coat hanger to an iPhone and start getting text messages through it. You can get a 128 key out of most electronic devices in a reasonable amount of time: https://en.wikipedia.org/wiki/Electromagnetic_attack#Known_attacks https://en.wikipedia.org/wiki/Electromagnetic_attack#Known_a...