17 ms·

Brain implant translates paralyzed man's thoughts into text with 94% accuracy

449 points by wlkr 5y ago

HenryKissinger 5y ago
It wouldn't be particularly hard for a trained person to block their thoughts to a brain implant, if the police were trying to read their mind.
- funnyflamigo 5y ago
  If the police are able to implant a device into your brain to get information out of you, I think you're screwed either way as the good 'ol rubber hose method is likely on the table - at least until people commonly elect to do the surgery or the surgery somehow gets to legendary levels of safety in the dystopian future.
  - HenryKissinger 5y ago
    It wouldn't be any different from trying to force a confession through torture. You need to control your thoughts during the interrogation.
    - funnyflamigo 5y ago
      Oh that's interesting, make them think the thoughts at least for extraction. This first version of the tech requires the user to mentally imagine moving muscles to trace letters so yeah it's not likely to be an option... yet. I wonder what happens to these BCI results when the user is on psychedelics a la MK Ultra EDIT: Wait your point was slightly different then I first read but that makes perfect sense!
    - gus_massa 5y ago
      Ignoring the ethical and legal problems with torture, the main problem that is not usually portrayed in films is that people say whatever the interrogator wants to stop the pain. interrogator> We have been torturing you for 10 hours. If you spell "John" we will stop torturing you and torture John instead. interrogated> [The pain is too much. We have a deal. Sorry John.] JOHN A better interrogator can be more subtle, like interrogator> We have some evidence [1] that John is the one that put the bomb, and that you are innocent, but my boss is not sure. If you confirm this info we will send you immediately to your cell. [1] It's a lie! Also, John is innocent in case you are wondering.
  - WithinReason 5y ago
    I'm pretty sure torture would be seen as barbaric and inhumane while reading thoughts would not have the same associations
    - tremon 5y ago
      Yes, just wait until we can do the reading with on-skin electrodes or micro-needle patches.
    - funnyflamigo 5y ago
      If like the sibling said they develop on-skin electrodes or micro-needle patches I 100% agree. Currently it requires an invasive and risky brain surgery which AFAIK would be a definite no-go. At least in the US, this would likely be considered extremely inhumane unless it could be done without such significant risk of death or brain damage - given the nature of the surgery I find that unlikely. In the cases where an invasive brain surgery would be permitted, I imagine torture would already be an option. Keep in mind torture doesn't have nearly the same risk of death and permanent brain damage as an invasive brain surgery.
  - spywaregorilla 5y ago
    Nah. Generic brain reading is not possible in the forseeable future. Everyone's brain works differently. You have to train a machine learning model to recognize specific types of inputs. You likely cannot do pure thought reading.
- spywaregorilla 5y ago
  It wouldn't require any training at all. This tech cannot read your thoughts. It's reading motor inputs. The difference is on the level of hearing someone's chess moves dictated vs. reading their mind to understand how they perceive the game. It is entirely a consciously composed and intentionally sent message.
  - tremon 5y ago
    It wouldn't require any training at all. It's reading motor inputs. You're assuming that everyone's motor cortex/nervous system is wired exactly the same way. Given what we know about the variability of the human body, I wouldn't expect this to hold for the entire population.
    - spywaregorilla 5y ago
      I meant training on the human part to hide from the device. Every human is wired differently. You need to train an ML model to read signals specific to that human for this tech to work.
tyingq 5y ago
Interesting that the imagined letters are single-stroke, with no pen lifting, somewhat like most of the PalmOS graffiti letter forms.
adminscoffee 5y ago
if someone spoke 94% of a second language we would say that person is fluent in that second language. this is great news
grumple 5y ago
Very cool. The opportunity for people with physical impairments to continue communicating - and convey their ability to still experience the world - gives us the ability to help these people lead happier lives.
jonplackett 5y ago
The next truly massive tech revolution will have to be something even we tech peeps reject. And I think bodily embedded stuff would do the trick. It could be amazingly effective though if this is where we’re at already. Imagine the speed and enjoyment increase for anything from typing to gaming to driving a car. You’d get completely left behind if you rejected it.
- funnyflamigo 5y ago
  > In tests, the man was able to achieve writing speeds of 90 characters per minute (about 18 words per minute), with approximately 94 percent accuracy (and up to 99 percent accuracy with autocorrect enabled). Don't get me wrong I'm sure there will be advances. But this current tech is based off reading nerve data meant to be movement data - the user needs to mentally trace each letter. So I don't see this form of the tech at least being able to compete with qwerty let alone stenography. Actually to that point, stenography would allow people to input data (it must be language specific and error tolerant but most BCI is as well) at ~5x the average typing speed on qwerty but that hasn't proliferated. EDIT: On second thought I could see it matching physical movement, maybe _slightly_ outperforming it by a few % by skipping a few physical limitations. I think this should be essentially identical to any other physical motion based communication.
  - EGreg 5y ago
    Simple ... how about mentally tracing over a keyboard like Swype?
    - funnyflamigo 5y ago
      Question to anyone that knows - I know that actively "imagining" movements/activities is neurologically _very_ similar in many ways, which is why it works here for BCI. Do these "imagined" thoughts develop muscle memory? Anyway, assuming they do, I'm not sure if it'd be a real advantage over physically swyping with your finger (for those that can obviously) - it seems like they'd be roughly equivalent? Actually that'll be my second question - how is this system affected by things like tremors? I'd suspect they originate from your brain in which case your "mental movements" should have the exact same quirks and limitations as physical movements. The end game of course is not needing to use the movement system to interpret information.
GOONIMMUNE 5y ago
> In tests, the man was able to achieve writing speeds of 90 characters per minute (about 18 words per minute), with approximately 94 percent accuracy (and up to 99 percent accuracy with autocorrect enabled). I'd be interested in knowing how this metric changes over time as the user gains more experience with the BCI device. The article mentions that researchers recorded his neural activity while he was thinking about writing letters. Would the man eventually find that the system is more accurate or faster when he instead learns how to think "the thought that generates the letter A in my BCI device"? Fascinating stuff all around.
- gonehome 5y ago
  My hypothesis would be yes. You need to establish the initial feedback loop somehow and imagining writing is a good way to do it. But once you have it I’d suspect you could get faster doing what you’re describing.
  - mistermann 5y ago
    I agree, and I'm a bit excited/worried if brain interfaces turn out to be kinda like smart handheld devices, years of failures until the essential "recipe" is discovered like the iPhone, followed by a tsunami of innovation, functionality, and power - some realized, some not. Do we have adequate wisdom to wield the powers we are granting ourselves?
- striking 5y ago
  The article mentions that > electrodes implanted in his motor cortex recorded signals of his brain activity so I'm assuming other thoughts and interactions had little to no effect.
  - Ajedi32 5y ago
    This makes me wonder if "thinking about writing letters" is really an accurate description of what's happening. Was the subject merely thinking about writing letters, or was he actually trying to write them, such that if he weren't paralyzed his muscles would be moving to perform that task?
savant_penguin 5y ago
> (Don't think this nurse is hot! Don't think this nurse is hot! Holy shit she's hot.) > (You can do this) > (OMG, is it on already?) > (You can do this) > (What a nice bottom) > (Don't think bottom you idiot) > (She's entering the room, think something else, now, fast, bunny bunny) > (Bunny) > (Bunny) > (Bunny, you got this)
- LadyCailin 5y ago
  I was just thinking about how, once this technology is perfected, at how easy it will be to interrogate high value prisoners. Just pop one of these in their head, and tell them not to think about the top secret or incriminating stuff.
  - tester34 5y ago
    remember that it may be used against you once regime changes
  - david-cako 5y ago
    Don't think about top secret or incriminating stuff.
    - elliekelly 5y ago
      There are some people who, it seems, can “lie” to themselves. They’re so convinced of their own lies they end up truly believing it. Can you imagine a world where people train themselves to be delusional for the purpose of avoiding self-incrimination during this kind of “brain” interrogation?
  - alok-g 5y ago
    ... till I guess a deep fake thoughts arrive. ;-)
    - hwers 5y ago
      Adversarial thoughts that get misclassified into something safe.
shepik 5y ago
> In this case, the man – called T5 in the study, i wonder if it's a coincidence? https://huggingface.co/transformers/model_doc/t5.html https://huggingface.co/transformers/model_doc/t5.html
rasz 5y ago
His first words were "I need more computing power"
- mikepurvis 5y ago
  "How about sending me a fourth gimbal for Christmas."
  - abdulhaq 5y ago
    A friend of mine was a nurse on a mental health ward in the UK. There was a long term patient there who no-one could understand, until my friend, who was Mauritian, started working there. The patient was saying 'mes dents, mes dents'. (my false teeth, my false teeth..)
deleted 5y ago
[deleted]
cs702 5y ago
System diagram (how it works): https://github.com/fwillett/handwritingBCI/raw/main/systemDiagram.png https://github.com/fwillett/handwritingBCI/raw/main/systemDi... -- Code and data (for replicating results offline): https://github.com/fwillett/handwritingBCI https://github.com/fwillett/handwritingBCI -- Published paper (you can find its full contents online): https://www.nature.com/articles/s41586-021-03506-2 https://www.nature.com/articles/s41586-021-03506-2
- dnautics 5y ago
  yeesh it's an RNN and not even something like a transformer
  - cs702 5y ago
    ...which means that even a modestly sized transformer model could do significantly better.
    - dnautics 5y ago
      with a big heap of "maybe". Signal could be a limiting factor, but also come on, more than 99% correct? That's above the level of a non-professional human transcriber in most situations. I'm not sure that typing I get more than 99% correct, I've made at least six errors typing this sentence alone (though it's quick to fix). The real improvement, it seems would be to speed and latency, looking at the diagram, the sampling is over the course of 3 seconds, which is butt-slow. A good NN would be able to compressed patterns in the data stream and blat out more than one letter, or contextually learn letters in the word, or learn whole words, etc. But that is not, it appears, supported by the model.
      - cs702 5y ago
        That's why I wrote "could," as opposed to "would" :-)
        5y ago
rubylark 5y ago
I'm curious how his imagined writing compares to the handwriting he had before he was paralyzed. Was it always that messy or are the BCI controls difficult to use? For example, his comma seems exaggerated, as though he had to imagine grand gestures to get the SW to recognize it as a valid character. But on the other hand, his "m" and "n" look fairly normal. It's possible that's just what his handwriting looks like.
- ordu 5y ago
  Actual handwriting goes with a feedback, at least with kinesthetic, tactile and visual. Here he writes "blindly". I dont' know how much it contributes to the mess, but I'm sure that a noticeable amount of it can be explained by the lack of a feedback.
cblconfederate 5y ago
This is the metaverse we should be funding, not phones strapped on bulky blindfolds.
- charcircuit 5y ago
  Give it 5 years and VR will not be bulky at all.
  - throwaway946513 5y ago
    Full Dive VR coming soon
  - foxfluff 5y ago
    Five years ago, the avegant glyph seemed reasonably small for an HMD (well, if you could get it without the earphones). I feel like we've regressed from that point.
    - charcircuit 5y ago
      That HMD only had a FoV of 45 degrees. The current technology which is production ready is optics using pancake lenses. For an example look at Huawei's VR glasses or HTC's Vive Flow.
      - foxfluff 5y ago
        Yea, I actually consider the smaller FOV a feature. Doesn't necessarily have to be 45 degrees, but I'm not interested in covering my entire field of view. I want a monitor for work, not for games and movies. Immersion doesn't matter, but the comfort of not needing to cup my eyes sure does.
        maximus-decimus 5y ago
        The whole point of VR is immersion. It's literally called Virtual Reality. What you seem to want is just a screen strapped to your head with some lenses to make it look further. Why do you want that exactly? Portability? Space saving?
bawana 5y ago
This too will be weaponized. How? Capture a North Korean official, put one of these in his brain and siphon off all the juicy intel. Capture a narco trafficker. ditto. Capture a terrorist. ditto. Have we made the world better? It seems that whenever we develop ANY device for improved information processing, it disrupts the world we live in and displaces the uniquely human style of information processing-rendering humans less necessary. It is paradoxical. We make our lives 'easier' and human talent is lost.
- IggleSniggle 5y ago
  This doesn't work like this. Braingate (and all brain implants of this nature) require very active effort/cooperation in order to produce output. Here's what it feels like: - Imagine, very hard, the act of moving your arm to write a message. - Do everything that you would do to speak a word, except the actual act of articulating a word. Get as close as you can while stopping the actual muscles involved from firing. These are not mind-readers. They hook into your normal brain circuitry. It mostly works the same way that you can type out a message on your keyboard without thinking about the keyboard: it's a brain HCI, `/dev/input`, not kernel-space.
  - hwers 5y ago
    Really wouldn't surprise me if they figured out a way to do this in an unsupervised way though. (We might be decades from that though who knows.)
    - alboy 5y ago
      Maybe you could even do supervised by force-feeding known stimuli a la Clockwork Orange.
      - IggleSniggle 5y ago
        This bypasses the desired capability, that of getting info from the hard disk. If you connect input in a feedback loop with an output and train on a particular output, of course you’re going to get the output you’re training for. It’s just not going to bear much resemblance to the data that’s on the hard disk without cooperation from the host system.
clircle 5y ago
How do you measure accuracy? Is there another way the man can communicate? edit: The researchers compared the BCI output to a prompt that T5 was supposed to restate. I was thinking that T5 was communicating without prompts in the experiment. This isn't my idea of translation accuracy, but you've got to have some baseline.
- ceejayoz 5y ago
  "A man paralyzed from the neck down due to a spinal cord injury" typically can still talk. If not, there's blinking/mouth controls like Stephen Hawking.
  - Levitz 5y ago
    It would also be easy enough to make him memorize some text and attempt to replicate it, would it not?
    - cgriswald 5y ago
      No. You’re trying to measure whether the machine is accurately translating his thoughts into characters. He may make errors in spelling or memory. Success is whether the machine spits out the characters he intended, even if they weren’t the ones he was supposed to send.
  - excalibur 5y ago
    Surprised at how far I had to scroll to find Hawking's name. If he were still with us I bet he could type circles around this guy.
- graindcafe 5y ago
  Same way autospell is not working with 100% accuracy and you're still managing to express yourself, I believe
restalis 5y ago
Something even more impressive, IMHO, from the same source: https://www.sciencealert.com/a-brain-implant-has-allowed-a-blind-woman-to-see-simple-2d-shapes-and-letters https://www.sciencealert.com/a-brain-implant-has-allowed-a-b... It's Star Trek like technology (think TNG series' Geordi La Forge).
gonehome 5y ago
I wonder if it’d be faster if you imagined typing instead of writing (obviously would require the patient to be a proficient typist).
- deleted 5y ago
  [deleted]
- ThaJay 5y ago
  I don't think it works like that. Letters are shapes but keys just are a relative position. The software is reading gestures, specific keypress motions seems much less data to work with.
  - gonehome 5y ago
    I think it’s trained. “Imagine writing an A” Then they look at what fires and record it. Instead you’d ask “imagine typing an A” and then do the same thing. Eventually when brought training happens to capture variation you start to get visual feedback and can train faster.
- caymanjim 5y ago
  This was my first thought. Writing is incredibly slow. I can barely even operate a pen anymore. I don't see any reason why this mechanism couldn't be used on any thought patterns. If the subtle motions of typing aren't high-fidelity enough to be differentiated, I still wouldn't have chosen normal letter patterns; I'd design a new motion alphabet that is much easier and faster to "write" by thought. The article alludes to this: > the researchers say that alphabetical letters are very different from one another in shape, so the AI can decode the user's intention more rapidly as the characters are drawn, compared to other BCI systems that don't make use of dozens of different inputs in the same way The fact that it works so well on these complex motions means it can probably work better and faster if they use an alphabet with simpler--but still distinct--motions. Probably lots of lessons to be learned from shorthand and other rapid transcription techniques. Losing the ability to communicate scares the hell out of everyone. This is amazing progress. And it'll have plenty of applications even for able-bodied people.
AlwaysRock 5y ago
This is very very cool and interesting.
sam1r 5y ago
This is really awesome. Imagine getting paid to think one day...
CapmCrackaWaka 5y ago
Interesting that the user has to think of specific letters to spell out a word. I guess the 26 distinct English letters are much easier to parse and separate than the untold thousands of words, especially when those words could be used in different contexts. I bet the next step wouldn't be to even parse words to make a sentence. It seems to me the next low hanging fruit would be to enable this machine to parse common ideas. I wonder how complex it would be to translate full sentences like "Good Morning", "I gotta take a dump", or "I'm hungry". It doesn't seem like it would be that much of a leap, since the user already has to imagine the idea of different letters. Admittedly I have no idea how different those concepts are, or how they would express themselves in the brain to be interpreted by the machine.
- Communitivity 5y ago
  That next step is a leap. I view it as the difference between parsing and processing spoken English and parsing and processing spoken Mandarin Chinese. The letters and numbers are 36 symbols to understand, plus capitalization and punctuation. Understanding words means mapping out the brain pathways for each word. There is actually a path for this that's been done before, in a way. Dragon Naturally Speaking was evolved this way. As I understand it, that evolution took decades. In 1952 Bell Labs came up with Audrey (Automatic Digit Recognition). Voice specific, and could only recognize numbers 0-9. This is where the OP linked Brain Computer Interface (BCI) is. In 1962 IBM revealed Shoebox at the World Fair. Shoebox could understand 16 English words. It would listen to the words and complete an instruction for example adding up numbers and providing the result. Harpy came in 1971. Funded by Darpa and developed through a collaboration between CMU, Stanford and IBM. Harpy cold work with ordinary speech and pick out individual words, but it only had a vocabulary of around 1000 words. In 1974, Kurzweil forms Kurzweil Computer Products (KCP) for development of pattern recognition technology. In 1976, KCP introduces the Kurzweil Reading Machine, combining three technological firsts. In 1982 Dr's Jim and Janet Baker launched Dragon Systems and prototyped a voice recognition system that was based around mathematical models. The Bakers were mathematicians and the system they came up with was based a hidden Markov model – using statistics to predict words, phrases and sentences. In 1983, Kurzweil Music Systems launches a keyboard synthesizer that accurately reproduces the sounds of acoustic instruments. In 1985, Kurzweil Applied Intelligence introduces the first speech-to-text computer program. In 1990, Dragon Dictate was launched as the first general purpose large vocabulary speech to text dictation system. This was a groundbreaking product for Dragon, but it required users to pause between individual words. In 1994, KurzweilVoice for Windows 1.0 is launched, bringing discrete speech command technology to the personal computer environment. In 1995, Kurzweil Technologies is founded. By 1997, the problem of having to pause between words had been overcome and Dragon Naturally Speaking v1 was launched, 45 years after Audrey. In 1997, the Continuous Speech Natural Language Command and Control software is launched as Kurzweil Voice Commands; The Medical Learning Company is formed. In 2000, Kurzweil forms FAT KAT, Inc. to develop artificial intelligence that can make decisions about buying and selling on the stock market. Then in 2001 KTI introduced "Ramona," the virtual reality rock star. Yes, the last two have little (maybe even nothing) to do with speech recognition, but I found them interesting, so I thought you might too. The sources for the above are primarily: [1] http://www.fundinguniverse.com/company-histories/kurzweil-technologies-inc-history/ http://www.fundinguniverse.com/company-histories/kurzweil-te... [2] https://whatsnext.nuance.com/en-gb/dragon-professional/history-speech-recognition/ https://whatsnext.nuance.com/en-gb/dragon-professional/histo...
KhoomeiK 5y ago
Previous threads: https://news.ycombinator.com/item?id=27134049 https://news.ycombinator.com/item?id=27134049 https://news.ycombinator.com/item?id=27157369 https://news.ycombinator.com/item?id=27157369
- dang 5y ago
  Thanks! Macroexpanded: Neural implant lets paralyzed person type by imagining writing - https://news.ycombinator.com/item?id=27134049 https://news.ycombinator.com/item?id=27134049 - May 2021 (183 comments) Brain-Computer Interface User Types 90 Characters per Minute with Mind - https://news.ycombinator.com/item?id=27157369 https://news.ycombinator.com/item?id=27157369 - May 2021 (40 comments) This is pretty similar but I guess it's different work: Human use of high-bandwidth wireless brain-computer interface - https://news.ycombinator.com/item?id=26690126 https://news.ycombinator.com/item?id=26690126 - April 2021 (210 comments)
zivkovicp 5y ago
Amazing
codegladiator 5y ago
I can imagine a pretty good horror/scifi movie enabled by this.
deleted 5y ago
[deleted]
GhettoComputers 5y ago
I've spoken to Abe Caplan, one of the Braingate researchers a few times online on clubhouse. https://clubhousedb.com/user/abecaplan https://clubhousedb.com/user/abecaplan https://www.researchgate.net/scientific-contributions/Abraham-H-Caplan-33403213 https://www.researchgate.net/scientific-contributions/Abraha... He said it did not need a high resolution to work well off the top of my head (it did not need to be very accurate, it does not have to be a single method of working, there are many places it can be linked and still work, this refers to the signal processing on the BCI), companies like neuralink often repackages these research projects with slick marketing as original but it was simply rebuilding this project with another interface (they used clunkier probes while neuralink wants to do an implant, but that was before they filed a patent for an implant as well). This says it requires an implant, but I am not sure if its true, his contributions are much older though and they might not be the same as the current Braingate research, but they were also for aiding with disabled people with controlling prosthetics, and also with signal processing and calibration to the user, they have made text input before inplants were required, so I don't see why its required, its benefit is being more convenient than setting up the probes or as a wearable. https://www.frontiersin.org/articles/10.3389/fnins.2012.00072/full https://www.frontiersin.org/articles/10.3389/fnins.2012.0007...
pedrobtz 5y ago
Can this be used for interrogation technique?
qntty 5y ago
The title is clickbait. This isn't "reading thoughts", it's reading motor movements, which is a different part of the brain than cognition. Most people will assume thoughts = cognition reading the title.
swayvil 5y ago
This opens the door to bad-thought-monitoring. Of course it will become required equipment for all citizens. Mark my words
deleted 5y ago
[deleted]
rdevsrex 5y ago
Total dystopian thought, but I wonder if this could ever be used to extract info people unwillingly, as in some interrogation scenario. I mean, you have a lot more control over what you say than what you think.
- marviel 5y ago
  Perhaps someday. It seems the way this experiment worked was by "imagining writing the words by hand", which would indicate that this kind of transmission was dependent on an explicit act of will.
- Raidion 5y ago
  If you read the article it's clear that the researchers aren't pulling out the information from the guy's thoughts, they're reading the nerve signals from him "imagining" he's writing a letter, and using ML to map that to pen strokes which get mapped to words. So if James Bond gets captured, all he has to do is to not imagine writing the information, he can think it all he wants.
- capableweb 5y ago
  Most likely the brain implement is working against/with a persons consciousness. This "information extraction" device you talk about, would need to work against/with the sub-consciousness, otherwise you can just think "WALL WALL WALL WALL WALL" and the device would only be able to extract that. Not to say that this device was probably trained on the person a lot before they could reach that accuracy. A "information extraction" device would have to be trained on it's victim first, but why would they play along with the training? Maybe I'm wrong, but I think most people have control over what they consciously think, at least it works like that for me.
- 3r8riacz 5y ago
  Dont't think of your Gmail password now
renewiltord 5y ago
This is freaking awesome. Those mind-control drone pilots we have today mean there's an exciting near-future for mind-machine interfaces. It will be fantastic not just for people who are paralyzed, but also perhaps we will have better prostheses, and maybe in the metaverse we will have additional appendages to do more tasks. I am looking forward to our new world.
brazed_blotch 5y ago
I wonder how something like this would function with chorded typing? A keyboard like this [1] with only one key per finger I would imagine would be relatively easy (easier than handwriting even? Since it's not limited to only your finger muscles - you attach every easily controllable muscle in the body to a button on the 'keyboard', and it seems to be a binary value whether a finger is clicking down or not, instead of what letter your hand is writing) for a brain implant to register. And a lot faster than handwriting. [1] - https://www.gboards.ca/product/ginni https://www.gboards.ca/product/ginni

449 points by wlkr 5y ago

It wouldn't be particularly hard for a trained person to block their thoughts to a brain implant, if the police were trying to read their mind.

funnyflamigo 5y ago
If the police are able to implant a device into your brain to get information out of you, I think you're screwed either way as the good 'ol rubber hose method is likely on the table - at least until people commonly elect to do the surgery or the surgery somehow gets to legendary levels of safety in the dystopian future.
- HenryKissinger 5y ago
  It wouldn't be any different from trying to force a confession through torture. You need to control your thoughts during the interrogation.
  - funnyflamigo 5y ago
    Oh that's interesting, make them think the thoughts at least for extraction. This first version of the tech requires the user to mentally imagine moving muscles to trace letters so yeah it's not likely to be an option... yet. I wonder what happens to these BCI results when the user is on psychedelics a la MK Ultra EDIT: Wait your point was slightly different then I first read but that makes perfect sense!
  - gus_massa 5y ago
    Ignoring the ethical and legal problems with torture, the main problem that is not usually portrayed in films is that people say whatever the interrogator wants to stop the pain. interrogator> We have been torturing you for 10 hours. If you spell "John" we will stop torturing you and torture John instead. interrogated> [The pain is too much. We have a deal. Sorry John.] JOHN A better interrogator can be more subtle, like interrogator> We have some evidence [1] that John is the one that put the bomb, and that you are innocent, but my boss is not sure. If you confirm this info we will send you immediately to your cell. [1] It's a lie! Also, John is innocent in case you are wondering.
- WithinReason 5y ago
  I'm pretty sure torture would be seen as barbaric and inhumane while reading thoughts would not have the same associations
  - tremon 5y ago
    Yes, just wait until we can do the reading with on-skin electrodes or micro-needle patches.
  - funnyflamigo 5y ago
    If like the sibling said they develop on-skin electrodes or micro-needle patches I 100% agree. Currently it requires an invasive and risky brain surgery which AFAIK would be a definite no-go. At least in the US, this would likely be considered extremely inhumane unless it could be done without such significant risk of death or brain damage - given the nature of the surgery I find that unlikely. In the cases where an invasive brain surgery would be permitted, I imagine torture would already be an option. Keep in mind torture doesn't have nearly the same risk of death and permanent brain damage as an invasive brain surgery.
- spywaregorilla 5y ago
  Nah. Generic brain reading is not possible in the forseeable future. Everyone's brain works differently. You have to train a machine learning model to recognize specific types of inputs. You likely cannot do pure thought reading.
spywaregorilla 5y ago
It wouldn't require any training at all. This tech cannot read your thoughts. It's reading motor inputs. The difference is on the level of hearing someone's chess moves dictated vs. reading their mind to understand how they perceive the game. It is entirely a consciously composed and intentionally sent message.
- tremon 5y ago
  It wouldn't require any training at all. It's reading motor inputs. You're assuming that everyone's motor cortex/nervous system is wired exactly the same way. Given what we know about the variability of the human body, I wouldn't expect this to hold for the entire population.
  - spywaregorilla 5y ago
    I meant training on the human part to hide from the device. Every human is wired differently. You need to train an ML model to read signals specific to that human for this tech to work.

tyingq 5y ago

Interesting that the imagined letters are single-stroke, with no pen lifting, somewhat like most of the PalmOS graffiti letter forms.

adminscoffee 5y ago

if someone spoke 94% of a second language we would say that person is fluent in that second language. this is great news

grumple 5y ago

Very cool. The opportunity for people with physical impairments to continue communicating - and convey their ability to still experience the world - gives us the ability to help these people lead happier lives.

jonplackett 5y ago

The next truly massive tech revolution will have to be something even we tech peeps reject. And I think bodily embedded stuff would do the trick. It could be amazingly effective though if this is where we’re at already. Imagine the speed and enjoyment increase for anything from typing to gaming to driving a car. You’d get completely left behind if you rejected it.

funnyflamigo 5y ago
> In tests, the man was able to achieve writing speeds of 90 characters per minute (about 18 words per minute), with approximately 94 percent accuracy (and up to 99 percent accuracy with autocorrect enabled). Don't get me wrong I'm sure there will be advances. But this current tech is based off reading nerve data meant to be movement data - the user needs to mentally trace each letter. So I don't see this form of the tech at least being able to compete with qwerty let alone stenography. Actually to that point, stenography would allow people to input data (it must be language specific and error tolerant but most BCI is as well) at ~5x the average typing speed on qwerty but that hasn't proliferated. EDIT: On second thought I could see it matching physical movement, maybe _slightly_ outperforming it by a few % by skipping a few physical limitations. I think this should be essentially identical to any other physical motion based communication.
- EGreg 5y ago
  Simple ... how about mentally tracing over a keyboard like Swype?
  - funnyflamigo 5y ago
    Question to anyone that knows - I know that actively "imagining" movements/activities is neurologically _very_ similar in many ways, which is why it works here for BCI. Do these "imagined" thoughts develop muscle memory? Anyway, assuming they do, I'm not sure if it'd be a real advantage over physically swyping with your finger (for those that can obviously) - it seems like they'd be roughly equivalent? Actually that'll be my second question - how is this system affected by things like tremors? I'd suspect they originate from your brain in which case your "mental movements" should have the exact same quirks and limitations as physical movements. The end game of course is not needing to use the movement system to interpret information.

GOONIMMUNE 5y ago

> In tests, the man was able to achieve writing speeds of 90 characters per minute (about 18 words per minute), with approximately 94 percent accuracy (and up to 99 percent accuracy with autocorrect enabled). I'd be interested in knowing how this metric changes over time as the user gains more experience with the BCI device. The article mentions that researchers recorded his neural activity while he was thinking about writing letters. Would the man eventually find that the system is more accurate or faster when he instead learns how to think "the thought that generates the letter A in my BCI device"? Fascinating stuff all around.

gonehome 5y ago
My hypothesis would be yes. You need to establish the initial feedback loop somehow and imagining writing is a good way to do it. But once you have it I’d suspect you could get faster doing what you’re describing.
- mistermann 5y ago
  I agree, and I'm a bit excited/worried if brain interfaces turn out to be kinda like smart handheld devices, years of failures until the essential "recipe" is discovered like the iPhone, followed by a tsunami of innovation, functionality, and power - some realized, some not. Do we have adequate wisdom to wield the powers we are granting ourselves?
striking 5y ago
The article mentions that > electrodes implanted in his motor cortex recorded signals of his brain activity so I'm assuming other thoughts and interactions had little to no effect.
- Ajedi32 5y ago
  This makes me wonder if "thinking about writing letters" is really an accurate description of what's happening. Was the subject merely thinking about writing letters, or was he actually trying to write them, such that if he weren't paralyzed his muscles would be moving to perform that task?

savant_penguin 5y ago

> (Don't think this nurse is hot! Don't think this nurse is hot! Holy shit she's hot.) > (You can do this) > (OMG, is it on already?) > (You can do this) > (What a nice bottom) > (Don't think bottom you idiot) > (She's entering the room, think something else, now, fast, bunny bunny) > (Bunny) > (Bunny) > (Bunny, you got this)

LadyCailin 5y ago
I was just thinking about how, once this technology is perfected, at how easy it will be to interrogate high value prisoners. Just pop one of these in their head, and tell them not to think about the top secret or incriminating stuff.
- tester34 5y ago
  remember that it may be used against you once regime changes
- david-cako 5y ago
  Don't think about top secret or incriminating stuff.
  - elliekelly 5y ago
    There are some people who, it seems, can “lie” to themselves. They’re so convinced of their own lies they end up truly believing it. Can you imagine a world where people train themselves to be delusional for the purpose of avoiding self-incrimination during this kind of “brain” interrogation?
- alok-g 5y ago
  ... till I guess a deep fake thoughts arrive. ;-)
  - hwers 5y ago
    Adversarial thoughts that get misclassified into something safe.

shepik 5y ago

> In this case, the man – called T5 in the study, i wonder if it's a coincidence? https://huggingface.co/transformers/model_doc/t5.html https://huggingface.co/transformers/model_doc/t5.html

rasz 5y ago

His first words were "I need more computing power"

mikepurvis 5y ago
"How about sending me a fourth gimbal for Christmas."
- abdulhaq 5y ago
  A friend of mine was a nurse on a mental health ward in the UK. There was a long term patient there who no-one could understand, until my friend, who was Mauritian, started working there. The patient was saying 'mes dents, mes dents'. (my false teeth, my false teeth..)

deleted 5y ago

[deleted]

cs702 5y ago

System diagram (how it works): https://github.com/fwillett/handwritingBCI/raw/main/systemDiagram.png https://github.com/fwillett/handwritingBCI/raw/main/systemDi... -- Code and data (for replicating results offline): https://github.com/fwillett/handwritingBCI https://github.com/fwillett/handwritingBCI -- Published paper (you can find its full contents online): https://www.nature.com/articles/s41586-021-03506-2 https://www.nature.com/articles/s41586-021-03506-2

dnautics 5y ago
yeesh it's an RNN and not even something like a transformer
- cs702 5y ago
  ...which means that even a modestly sized transformer model could do significantly better.
  - dnautics 5y ago
    with a big heap of "maybe". Signal could be a limiting factor, but also come on, more than 99% correct? That's above the level of a non-professional human transcriber in most situations. I'm not sure that typing I get more than 99% correct, I've made at least six errors typing this sentence alone (though it's quick to fix). The real improvement, it seems would be to speed and latency, looking at the diagram, the sampling is over the course of 3 seconds, which is butt-slow. A good NN would be able to compressed patterns in the data stream and blat out more than one letter, or contextually learn letters in the word, or learn whole words, etc. But that is not, it appears, supported by the model.
    - cs702 5y ago
      That's why I wrote "could," as opposed to "would" :-)
      - 5y ago

rubylark 5y ago

I'm curious how his imagined writing compares to the handwriting he had before he was paralyzed. Was it always that messy or are the BCI controls difficult to use? For example, his comma seems exaggerated, as though he had to imagine grand gestures to get the SW to recognize it as a valid character. But on the other hand, his "m" and "n" look fairly normal. It's possible that's just what his handwriting looks like.

ordu 5y ago
Actual handwriting goes with a feedback, at least with kinesthetic, tactile and visual. Here he writes "blindly". I dont' know how much it contributes to the mess, but I'm sure that a noticeable amount of it can be explained by the lack of a feedback.

cblconfederate 5y ago

This is the metaverse we should be funding, not phones strapped on bulky blindfolds.

charcircuit 5y ago
Give it 5 years and VR will not be bulky at all.
- throwaway946513 5y ago
  Full Dive VR coming soon
- foxfluff 5y ago
  Five years ago, the avegant glyph seemed reasonably small for an HMD (well, if you could get it without the earphones). I feel like we've regressed from that point.
  - charcircuit 5y ago
    That HMD only had a FoV of 45 degrees. The current technology which is production ready is optics using pancake lenses. For an example look at Huawei's VR glasses or HTC's Vive Flow.
    - foxfluff 5y ago
      Yea, I actually consider the smaller FOV a feature. Doesn't necessarily have to be 45 degrees, but I'm not interested in covering my entire field of view. I want a monitor for work, not for games and movies. Immersion doesn't matter, but the comfort of not needing to cup my eyes sure does.
      - maximus-decimus 5y ago
        The whole point of VR is immersion. It's literally called Virtual Reality. What you seem to want is just a screen strapped to your head with some lenses to make it look further. Why do you want that exactly? Portability? Space saving?

bawana 5y ago

This too will be weaponized. How? Capture a North Korean official, put one of these in his brain and siphon off all the juicy intel. Capture a narco trafficker. ditto. Capture a terrorist. ditto. Have we made the world better? It seems that whenever we develop ANY device for improved information processing, it disrupts the world we live in and displaces the uniquely human style of information processing-rendering humans less necessary. It is paradoxical. We make our lives 'easier' and human talent is lost.

IggleSniggle 5y ago
This doesn't work like this. Braingate (and all brain implants of this nature) require very active effort/cooperation in order to produce output. Here's what it feels like: - Imagine, very hard, the act of moving your arm to write a message. - Do everything that you would do to speak a word, except the actual act of articulating a word. Get as close as you can while stopping the actual muscles involved from firing. These are not mind-readers. They hook into your normal brain circuitry. It mostly works the same way that you can type out a message on your keyboard without thinking about the keyboard: it's a brain HCI, `/dev/input`, not kernel-space.
- hwers 5y ago
  Really wouldn't surprise me if they figured out a way to do this in an unsupervised way though. (We might be decades from that though who knows.)
  - alboy 5y ago
    Maybe you could even do supervised by force-feeding known stimuli a la Clockwork Orange.
    - IggleSniggle 5y ago
      This bypasses the desired capability, that of getting info from the hard disk. If you connect input in a feedback loop with an output and train on a particular output, of course you’re going to get the output you’re training for. It’s just not going to bear much resemblance to the data that’s on the hard disk without cooperation from the host system.

clircle 5y ago

How do you measure accuracy? Is there another way the man can communicate? edit: The researchers compared the BCI output to a prompt that T5 was supposed to restate. I was thinking that T5 was communicating without prompts in the experiment. This isn't my idea of translation accuracy, but you've got to have some baseline.

ceejayoz 5y ago
"A man paralyzed from the neck down due to a spinal cord injury" typically can still talk. If not, there's blinking/mouth controls like Stephen Hawking.
- Levitz 5y ago
  It would also be easy enough to make him memorize some text and attempt to replicate it, would it not?
  - cgriswald 5y ago
    No. You’re trying to measure whether the machine is accurately translating his thoughts into characters. He may make errors in spelling or memory. Success is whether the machine spits out the characters he intended, even if they weren’t the ones he was supposed to send.
- excalibur 5y ago
  Surprised at how far I had to scroll to find Hawking's name. If he were still with us I bet he could type circles around this guy.
graindcafe 5y ago
Same way autospell is not working with 100% accuracy and you're still managing to express yourself, I believe

restalis 5y ago

Something even more impressive, IMHO, from the same source: https://www.sciencealert.com/a-brain-implant-has-allowed-a-blind-woman-to-see-simple-2d-shapes-and-letters https://www.sciencealert.com/a-brain-implant-has-allowed-a-b... It's Star Trek like technology (think TNG series' Geordi La Forge).

gonehome 5y ago

I wonder if it’d be faster if you imagined typing instead of writing (obviously would require the patient to be a proficient typist).

deleted 5y ago
[deleted]
ThaJay 5y ago
I don't think it works like that. Letters are shapes but keys just are a relative position. The software is reading gestures, specific keypress motions seems much less data to work with.
- gonehome 5y ago
  I think it’s trained. “Imagine writing an A” Then they look at what fires and record it. Instead you’d ask “imagine typing an A” and then do the same thing. Eventually when brought training happens to capture variation you start to get visual feedback and can train faster.
caymanjim 5y ago
This was my first thought. Writing is incredibly slow. I can barely even operate a pen anymore. I don't see any reason why this mechanism couldn't be used on any thought patterns. If the subtle motions of typing aren't high-fidelity enough to be differentiated, I still wouldn't have chosen normal letter patterns; I'd design a new motion alphabet that is much easier and faster to "write" by thought. The article alludes to this: > the researchers say that alphabetical letters are very different from one another in shape, so the AI can decode the user's intention more rapidly as the characters are drawn, compared to other BCI systems that don't make use of dozens of different inputs in the same way The fact that it works so well on these complex motions means it can probably work better and faster if they use an alphabet with simpler--but still distinct--motions. Probably lots of lessons to be learned from shorthand and other rapid transcription techniques. Losing the ability to communicate scares the hell out of everyone. This is amazing progress. And it'll have plenty of applications even for able-bodied people.

AlwaysRock 5y ago

This is very very cool and interesting.

sam1r 5y ago

This is really awesome. Imagine getting paid to think one day...

CapmCrackaWaka 5y ago

Interesting that the user has to think of specific letters to spell out a word. I guess the 26 distinct English letters are much easier to parse and separate than the untold thousands of words, especially when those words could be used in different contexts. I bet the next step wouldn't be to even parse words to make a sentence. It seems to me the next low hanging fruit would be to enable this machine to parse common ideas. I wonder how complex it would be to translate full sentences like "Good Morning", "I gotta take a dump", or "I'm hungry". It doesn't seem like it would be that much of a leap, since the user already has to imagine the idea of different letters. Admittedly I have no idea how different those concepts are, or how they would express themselves in the brain to be interpreted by the machine.

Communitivity 5y ago
That next step is a leap. I view it as the difference between parsing and processing spoken English and parsing and processing spoken Mandarin Chinese. The letters and numbers are 36 symbols to understand, plus capitalization and punctuation. Understanding words means mapping out the brain pathways for each word. There is actually a path for this that's been done before, in a way. Dragon Naturally Speaking was evolved this way. As I understand it, that evolution took decades. In 1952 Bell Labs came up with Audrey (Automatic Digit Recognition). Voice specific, and could only recognize numbers 0-9. This is where the OP linked Brain Computer Interface (BCI) is. In 1962 IBM revealed Shoebox at the World Fair. Shoebox could understand 16 English words. It would listen to the words and complete an instruction for example adding up numbers and providing the result. Harpy came in 1971. Funded by Darpa and developed through a collaboration between CMU, Stanford and IBM. Harpy cold work with ordinary speech and pick out individual words, but it only had a vocabulary of around 1000 words. In 1974, Kurzweil forms Kurzweil Computer Products (KCP) for development of pattern recognition technology. In 1976, KCP introduces the Kurzweil Reading Machine, combining three technological firsts. In 1982 Dr's Jim and Janet Baker launched Dragon Systems and prototyped a voice recognition system that was based around mathematical models. The Bakers were mathematicians and the system they came up with was based a hidden Markov model – using statistics to predict words, phrases and sentences. In 1983, Kurzweil Music Systems launches a keyboard synthesizer that accurately reproduces the sounds of acoustic instruments. In 1985, Kurzweil Applied Intelligence introduces the first speech-to-text computer program. In 1990, Dragon Dictate was launched as the first general purpose large vocabulary speech to text dictation system. This was a groundbreaking product for Dragon, but it required users to pause between individual words. In 1994, KurzweilVoice for Windows 1.0 is launched, bringing discrete speech command technology to the personal computer environment. In 1995, Kurzweil Technologies is founded. By 1997, the problem of having to pause between words had been overcome and Dragon Naturally Speaking v1 was launched, 45 years after Audrey. In 1997, the Continuous Speech Natural Language Command and Control software is launched as Kurzweil Voice Commands; The Medical Learning Company is formed. In 2000, Kurzweil forms FAT KAT, Inc. to develop artificial intelligence that can make decisions about buying and selling on the stock market. Then in 2001 KTI introduced "Ramona," the virtual reality rock star. Yes, the last two have little (maybe even nothing) to do with speech recognition, but I found them interesting, so I thought you might too. The sources for the above are primarily: [1] http://www.fundinguniverse.com/company-histories/kurzweil-technologies-inc-history/ http://www.fundinguniverse.com/company-histories/kurzweil-te... [2] https://whatsnext.nuance.com/en-gb/dragon-professional/history-speech-recognition/ https://whatsnext.nuance.com/en-gb/dragon-professional/histo...

KhoomeiK 5y ago

Previous threads: https://news.ycombinator.com/item?id=27134049 https://news.ycombinator.com/item?id=27134049 https://news.ycombinator.com/item?id=27157369 https://news.ycombinator.com/item?id=27157369

dang 5y ago
Thanks! Macroexpanded: Neural implant lets paralyzed person type by imagining writing - https://news.ycombinator.com/item?id=27134049 https://news.ycombinator.com/item?id=27134049 - May 2021 (183 comments) Brain-Computer Interface User Types 90 Characters per Minute with Mind - https://news.ycombinator.com/item?id=27157369 https://news.ycombinator.com/item?id=27157369 - May 2021 (40 comments) This is pretty similar but I guess it's different work: Human use of high-bandwidth wireless brain-computer interface - https://news.ycombinator.com/item?id=26690126 https://news.ycombinator.com/item?id=26690126 - April 2021 (210 comments)

zivkovicp 5y ago

Amazing

codegladiator 5y ago

I can imagine a pretty good horror/scifi movie enabled by this.

deleted 5y ago

[deleted]

GhettoComputers 5y ago

I've spoken to Abe Caplan, one of the Braingate researchers a few times online on clubhouse. https://clubhousedb.com/user/abecaplan https://clubhousedb.com/user/abecaplan https://www.researchgate.net/scientific-contributions/Abraham-H-Caplan-33403213 https://www.researchgate.net/scientific-contributions/Abraha... He said it did not need a high resolution to work well off the top of my head (it did not need to be very accurate, it does not have to be a single method of working, there are many places it can be linked and still work, this refers to the signal processing on the BCI), companies like neuralink often repackages these research projects with slick marketing as original but it was simply rebuilding this project with another interface (they used clunkier probes while neuralink wants to do an implant, but that was before they filed a patent for an implant as well). This says it requires an implant, but I am not sure if its true, his contributions are much older though and they might not be the same as the current Braingate research, but they were also for aiding with disabled people with controlling prosthetics, and also with signal processing and calibration to the user, they have made text input before inplants were required, so I don't see why its required, its benefit is being more convenient than setting up the probes or as a wearable. https://www.frontiersin.org/articles/10.3389/fnins.2012.00072/full https://www.frontiersin.org/articles/10.3389/fnins.2012.0007...

pedrobtz 5y ago

Can this be used for interrogation technique?

qntty 5y ago

The title is clickbait. This isn't "reading thoughts", it's reading motor movements, which is a different part of the brain than cognition. Most people will assume thoughts = cognition reading the title.

swayvil 5y ago

This opens the door to bad-thought-monitoring. Of course it will become required equipment for all citizens. Mark my words

deleted 5y ago

[deleted]

rdevsrex 5y ago

Total dystopian thought, but I wonder if this could ever be used to extract info people unwillingly, as in some interrogation scenario. I mean, you have a lot more control over what you say than what you think.

marviel 5y ago
Perhaps someday. It seems the way this experiment worked was by "imagining writing the words by hand", which would indicate that this kind of transmission was dependent on an explicit act of will.
Raidion 5y ago
If you read the article it's clear that the researchers aren't pulling out the information from the guy's thoughts, they're reading the nerve signals from him "imagining" he's writing a letter, and using ML to map that to pen strokes which get mapped to words. So if James Bond gets captured, all he has to do is to not imagine writing the information, he can think it all he wants.
capableweb 5y ago
Most likely the brain implement is working against/with a persons consciousness. This "information extraction" device you talk about, would need to work against/with the sub-consciousness, otherwise you can just think "WALL WALL WALL WALL WALL" and the device would only be able to extract that. Not to say that this device was probably trained on the person a lot before they could reach that accuracy. A "information extraction" device would have to be trained on it's victim first, but why would they play along with the training? Maybe I'm wrong, but I think most people have control over what they consciously think, at least it works like that for me.
3r8riacz 5y ago
Dont't think of your Gmail password now

renewiltord 5y ago

This is freaking awesome. Those mind-control drone pilots we have today mean there's an exciting near-future for mind-machine interfaces. It will be fantastic not just for people who are paralyzed, but also perhaps we will have better prostheses, and maybe in the metaverse we will have additional appendages to do more tasks. I am looking forward to our new world.

brazed_blotch 5y ago

I wonder how something like this would function with chorded typing? A keyboard like this [1] with only one key per finger I would imagine would be relatively easy (easier than handwriting even? Since it's not limited to only your finger muscles - you attach every easily controllable muscle in the body to a button on the 'keyboard', and it seems to be a binary value whether a finger is clicking down or not, instead of what letter your hand is writing) for a brain implant to register. And a lot faster than handwriting. [1] - https://www.gboards.ca/product/ginni https://www.gboards.ca/product/ginni