11 ms·
Brain signals translated into speech using AI
- 781 8y agoWhy are advances like this always presented as useful in a medical context? It seems to me like some sort of fear of talking about commercial use cases - for example in this case the ability to talk in your mind with your phone, person-phone telepathy if you wish
- aeontech 8y agoBecause a slight increase in convenience to be able to talk to your phone subvocally is trivial compared to granting capability for vocal communication to someone who is incapable of regular speech. One is a slight improvement in convenience for an otherwise able person. The other is life-changing.
- 781 8y agoTrue, but many people weigh the convenience by the number of uses. In that case, 0.0001 * 1 million >> 1.0 * 100
- SmellyGeekBoy 8y agoAre there really that many people willing to have electrodes implanted in their brain just to... type on their phone!?
- solarkraft 8y agoHonestly, for a few thousand Euros, if it worked really well (and was a well supported product software wise, with open APIs) ... I might do it - And I do think that while it wouldn't be everyone, significantly many people willing to do it could be found.
- notahacker 8y agoOnly on Hacker News could people's main concern about whether to have invasive surgery to put electrodes on the surface of the speech centres of the brain be ensuring the APIs of the software it transmits messages to are open!
- Dylan16807 8y agoNot right now, but imagine if the level of invasiveness/danger was on the same level as a birth control implant, or botox, or lasik. I've had surgery to remove drainage tubes from my eardrums just because they were being inconvenient. I'd rather put up with that in exchange for being able to do 150wpm hands-free.
- stubish 8y agoWhat are you going to do with your hands while you are concentrating on your writing? They are going to be dangling there, useless for nothing except reflex actions like picking your nose or driving your car (!). I already have attached devices that let me enter text faster than I can think it. I think the tech only gets interesting for the handy-capable when it goes beyond words, capturing images or sounds as you imagine them, and that is a long way of.
- Dylan16807 8y agoI might be holding my phone, I might just not be near a keyboard, I might be controlling a video game and use it for extra input... And while I can type sentences pretty fast, sometimes I get bogged down with things like code and entering special characters and being able to subvocalize a single syllable to activate macros would be pretty nice. (Yes I know you could do that without a brain interface, but the point is that hands are limited.) Edit: why does this comment get a downvote?
- stubish 8y agoThese use cases are all great for speech recognition, which is decades ahead of brain interfaces and less invasive (and just as quiet if you want by sub-vocalizing into a throat mike). Again, not particularly interesting to most of us but very interesting to people with various disabilities.
- OkGoDoIt 8y agoI'm looking forward to the day that a technology like this is safe or simple enough for widespread commercial use. I would be concerned about the safety of brain surgery, but lots of people get lasik which uses lasers to cut your eyes, which in my head is much scarier. Hopefully someday this could be a fairly routine procedure.
- umeshunni 8y agoIf this was consumer-grade technology, you wouldn't need to type on your phone, you could simply communicate with others directly like telepathy.
- gknoy 8y agoI think Star Trek II put it well, in that the needs of the many are not always more important than the needs of the few.
- kuschku 8y agoI think you mean Star Trek III: The Search for Spock? :)
- gknoy 8y agoYou're right - I had mis-remembered it. The scene I was thinking of was the one near the end of Wrath of Khan where Spock and Kirk are discussing sacrifice, but that's not where the quote is from. I'll blame it on the scene in II being so much more memorable. ;)
- bdamm 8y agoSelling subvocal control could fund the disability case. Finding users willing to pay to build the technology you want is very much in vogue.
- wodenokoto 8y agoI believe it is about grants. If your research has medical implications, there may be more grants available.
- AlexCoventry 8y agoIt's where the money's at: > This work was supported by grants from the NIH (DP2 OD008627 and U01 NS098971-01). E.F.C. is a New York Stem Cell Foundation-Robertson Investigator. This research was also supported by The William K. Bowes Foundation, the Howard Hughes Medical Institute, The New York Stem Cell Foundation and The Shurl and Kay Curci Foundation (Acknowledgements section, p. 498.)
- lamename 8y agoIn many cases there are multiple avenues for application, but this paper says the research was specifically funded by the National Institute of Health, which means the PR focus will be toward human health/disease.
- o10449366 8y agoAs other people have already said, it's where the grant money is. That's why many of the top "statistics" programs in the United States are really biostatistics programs - They get more funding and research opportunities and all of the benefits that come with those.
- gervase 8y agoAnother factor that no one has mentioned is that this approach seems to require invasive brain surgery. This is likely not a large risk for someone suffering from locked-in syndrome, but would be for someone who wants to improve their typing speed. As such, most early use cases would be restricted to people with few alternatives, such as medical cases.
- VikingCoder 8y agoPer another comment [1] "This technology uses an electrode array referred to as ECoG, which needs to be surgically placed on the surface of the brain. Current technology does not make it possible to have the signal quality from non-invasive methods." [1] : https://news.ycombinator.com/item?id=19742479 https://news.ycombinator.com/item?id=19742479
- anigbrowl 8y agoI predict that you'll see this deployable by nanobots within a decade.
- aperrien 8y agoOr with neural lace some time sooner...
- _visgean 8y ago"Musk defined the neuro lace as a "digital layer above the cortex" that would not necessarily imply extensive surgical insertion but ideally an implant through a vein or artery" well it is still pretty undefined. Essentially neural lace can be anything enabling this kind of communication.
- xeromal 8y agoI'd bet you 100USD that isn't true.
- return1 8y ago> Why are advances like this always presented as useful in a medical context? You don't think that people who have lost the ability to speak, e.g. from stroke would find it useful?
- dekhn 8y agoyou can't get funding for these sorts of things unless you show medical context first. It may change- billionaires funding brain scanning companies are an exmaple, but even then they tend to work with quadriplegics. If you cure a quadriplegic, then getting approval for the general public tends to be easy (it's an easy sell) but the other way around... not so much.
- est31 8y agoI think the study of cortical implants is interesting for this question because they touch the same questions basically. Cortical implants basically form the opposite side by providing a direct interface between machines and the brain. They are already available and deployed in medical contexts. Why aren't they deployed in healthy people as well? They have pretty cool advantages. With them, you can listen to music, etc. without being affected by outside noises (unless you want to). Imagine having a phone call in a noisy environment and you can actually hear stuff. Imagine your neighbours having a loud party but you can still sleep tight because you turned it into sleep mode where only fire alarms may disturb you. How many conflicts you could avoid! However, cortical implants have heavy disadvantages: They give you worse hearing quality than normal hearing, they require surgery, and also need devices on the outside that are in contact with them. That's why they aren't useful for healthy humans yet. But maybe, with improvement, one day they will become. Basically, the problems of this technology will be the same: worse quality, requirement for surgery, requirement for extra devices to carry around.
- swordsmith 8y agoBecause commercial use requiring brain surgery is most likely not going to work
- sjg007 8y agoYou could imagine Neuro link is doing something similar.
- melling 8y agoI saw the article in the NYT. It mentions that the system is capable of 150 wpm, compared to older systems that could only do 8 wpm. “Previous implant-based communication systems have produced about eight words a minute. The new program generates about 150 a minute, the pace of natural speech.” https://www.nytimes.com/2019/04/24/health/artificial-speech-brain-injury.html https://www.nytimes.com/2019/04/24/health/artificial-speech-...
- dang 8y agoThanks—since the paper is paywalled and HN tends to prefer the best popular article on such a topic, we've changed the URL to that from https://www.nature.com/articles/s41586-019-1119-1 https://www.nature.com/articles/s41586-019-1119-1. A pdf of the paper itself is linked from elsewhere in this thread.
- return1 8y agoNature has an editorial on the paper that is better and free https://www.nature.com/articles/d41586-019-01328-x https://www.nature.com/articles/d41586-019-01328-x
- dang 8y agoOk, we've switched to that from https://www.nytimes.com/2019/04/24/health/artificial-speech-brain-injury.html https://www.nytimes.com/2019/04/24/health/artificial-speech-.... Thanks!
- airstrike 8y agoSadly the comment section on nytimes.com is absolutely appalling...
- oceliker 8y agoI usually find the NYTimes comments mostly fine, save for the expected ideological bias. I read your comment, went back to check comments, and couldn't believe my eyes. People think this is mind-reading. > the article is using a benevolence trope ... to cloak the real purpose of this research — interfaces for perfectly healthy (and likely rich) people to control maybe their car or television with their minds (which would be cool!). > Soon employers will be able to require that all employees will have their thoughts recorded while performing any work-related task, as "performance of these tasks carries no expectations of privacy." So many people missing the fact that the technique (ECoG) requires a surgical procedure. It's not a consumer product, and it won't be cheap or easy any time soon. Is there a Gell-Mann amnesia effect [1] for comments sections? [1] https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect
- gwern 8y agoMirror: https://www.gwern.net/docs/ai/2019-anumanchipalli.pdf https://www.gwern.net/docs/ai/2019-anumanchipalli.pdf
- godelmachine 8y agoMerci beaucoup :)
- fundamental 8y agoAfter working on the neural speech-prosthesis problem in the single (or nearly single) electrode case, this seems to be a promising result from the ECoG side of things. While there's a few more results I'd like to see about how things would be expected to generalize more broadly over time/neural-input, it does look like the results are fairly solid. Hopefully there can be some further development from here into some practical applications.
- thisisananth 8y agoI haven't read the full paper to know what hardware it needs, but if it can be put into a headphone like wearable and integrated into phone to be output as text, then we can type on phone as fast as we can think. Looking forward to it! Also once of the major impediments to using voice assistants in public is to not want to shout into the phone and if it can be directly integrated into the assistant, the voice assistant use will explode. The possibilities are amazing!
- fundamental 8y agoThis technology uses an electrode array referred to as ECoG, which needs to be surgically placed on the surface of the brain. Current technology does not make it possible to have the signal quality from non-invasive methods.
- thisisananth 8y agoOk. Then all those applications will have to wait!
- Sharlin 8y agoI'm sure this augmentation will be offered by your friendly neighborhood illegal cybertech clinic sooner or later. Oh, wait, this wasn't the Cyberpunk 2020 timeline. Never mind then.
- king07828 8y agoSounds like a neural network that converts EEG signals to ECoG signals would be an interesting idea.
- sagarm 8y agoIf there is enough information in EEG signals to generate speech, an intervening layer to "convert" EEG signals into ECoG signals is an unnecessary complication.
- deleted 8y ago[deleted]
- solarkraft 8y agoI don't fully understand. > As each participant recited hundreds of sentences, the electrodes recorded the firing patterns of neurons in the motor cortex. The researchers associated those patterns with the subtle movements of the patient’s lips, tongue, larynx and jaw that occur during natural speech. The team then translated those movements into spoken sentences. So this means the audio example is speech synthesized from data gemerated while the person was actually reading out loud, right? Why does the text under the headline claim "no muscle movement needed", which would imply audible speech synthesized from mere thoughts?
- fundamental 8y agoParticipants spoke sentences while their neural activity was recorded. The acoustic speech was associated with motor movement which was in turn associated with learned neural patterns. As such the final result is a multi-stage model which can translate from patient specific (at a minimum in the sense of electrode configuration) neural activity into audio. All neural activity (from what I saw in the paper) is neural activity while the individual is reading sentences out loud.
- return1 8y agoYes, they are synthesizing speech from thought while the person mimicked speaking moving their mouth but making no sound. > Although synthesis performance for mimed speech was inferior to the performance for audible speech—which is probably due to absence of phonation signals during miming—this demonstrates that it is possible to decode important spectral features of speech that were never audibly uttered (P < 1 × 10−11 compared to chance, n = 58; Wilcoxon signedrank test) and that the decoder did not rely on auditory feedback
- solarkraft 8y agoOkay. I now understand what's actually significant about this - and conclude that "no muscle movement needed" is a lie.
- maebert 8y agoFormer neuroscientist working on similar stuff here. While the flashy part is the “speech synthesis”, the science breakthrough is actually better frames as an machine learning problem. Imagine you record someone moving their hand across a canvas. The hand movement becomes the input, the drawing the output. The ML problem they solved is to reconstruct the output (or at least something that resembles it) from the input. In the study’s case, that’s efferent motor signals. There is a long history of mapping these kind of signals in the sensorimotor homunculus to the respective muscles they control downstream (and some really cool stuff like prosthetic limbs can already be controlled with it), but speaking is a notoriously hard motor task and requires a lot of muscles to work in unison in very precise ways. When you implant these multi-electrode arrays, you get a few hundred more or less random single neurons, astrocytes, and local field potentials from the nether in between. Nonsense, noisy data. Being able to map this back to the result they produce in the body is technically as complex as astonishing!
- avisser 8y agoI'd love to see if it works when people are dreaming. Or people who have Locked-in Syndrome. Or even people who are in a coma. https://en.wikipedia.org/wiki/Locked-in_syndrome https://en.wikipedia.org/wiki/Locked-in_syndrome
- chanakya 8y agoIt doesn't try to read thoughts. It uses commands that are sent to the tongue/mouth to form syllables. We're nowhere near being able to trace / decode thoughts.
- sagartewari01 8y ago10 years ago I watched a national geographic documentary about something quite similar to this one. The results were also apperantly impressive. It's weird they haven't progressed much during this period. Sadly I can't find any resources about that project anywhere.
- kevin2107 8y agoProbably because of this little important detailed not mentioned in the title "Five participants underwent chronic implantation of a high-density, subdural electrode array over the lateral surface of the brain as part of their clinical treatment for epilepsy"
- joshvm 8y agoActually you can, in limited cases. If you have prior knowledge of the person's brain activity when looking at an image, it's possible to predict (reconstruct an image of) what they're looking at. https://journals.plos.org/ploscompbiol/article/figures?id=10.1371/journal.pcbi.1006633 https://journals.plos.org/ploscompbiol/article/figures?id=10...
- Engineering-MD 8y agoThat is not reading thoughts. That is via the occipital Cortex which has a highly structured representation of what is projected on to the retina. This is much easier as the organisation of the data is maintained relative to the retina, just regrouped according to visual field rather than eye.
- deleted 8y ago[deleted]
- chuckschemer 8y agoI read this as scientist creates free speech from brain signals.
- person_of_color 8y agoDoes this require mouth movement though?
- superconformist 8y agoWe have ways of making you talk. Or at least reading words from your brain.
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- oldgun 8y agoI wish Stephen Hawking were still around.
- jsnider3 8y agoYeah. It's unfortunate he didn't get to see the black hole picture from a few weeks ago.
- leafario2 8y agoCan't wait to control vim using my brain some day
- rainhacker 8y agoIf/when driverless cars are reliable, human's learning driving skills may become obsolete. Wonder what are the implications if we are able to reliably communicate our thoughts using AI. Can training vocal cords to speak become an obsolete skill too? On this trajectory - I also wonder what other core capabilities can be delegated to AI and what role this could plays in evolution of future generations.