5 ms·

Co-senior author here. I'm a long-time hackernews reader. Very excited to see that this work made the front page. I'm happy to take questions. The Scientific A

Co-senior author here. I'm a long-time hackernews reader. Very excited to see that this work made the front page. I'm happy to take questions. The Scientific American article was written about our lab's first major paper, "An Accurate and Rapidly Calibrating Speech Neuroprosthesis" published 14 August 2024 in the New England Journal of Medicine [0]. This video nicely summarizes the project [1]. Some additional coverage can be found in the New York Times[2] and Bloomberg[3]. Our lab's website can be found here (written in elm) [4]. The software platform that this BCI uses can be found here [5]. [0] https://www.nejm.org/doi/full/10.1056/NEJMoa2314132 https://www.nejm.org/doi/full/10.1056/NEJMoa2314132 [1] https://www.youtube.com/watch?v=thPhBDVSxz0 https://www.youtube.com/watch?v=thPhBDVSxz0 [2] https://www.nytimes.com/2024/08/14/health/als-ai-brain-implants.html https://www.nytimes.com/2024/08/14/health/als-ai-brain-impla... [3] https://www.bloomberg.com/news/articles/2024-08-14/brain-tech-breakthrough-restores-als-patient-s-ability-to-speak https://www.bloomberg.com/news/articles/2024-08-14/brain-tec... [4] https://neuroprosthetics.science/ https://neuroprosthetics.science/ [5] https://iopscience.iop.org/article/10.1088/1741-2552/ad3b3a https://iopscience.iop.org/article/10.1088/1741-2552/ad3b3a

malwrar 2y ago
Congratulations on achieving this, I teared up watching that man’s face in your linked summary video when he was able to communicate with your device! It looks like there are three components: 1.) the device itself 2.) a microphone 3.) eye tracking (??) Is that right? If so, to what degree does a person’s ability to make at least utterances into the microphone matter to the overall system’s performance? Are you heading in the direction of purely using the device itself to determine what patients are intending to say?
- drdavidbrandman 2y ago
  Thank you for the kind words! A brain computer interface is a device that records brain signals, and allows people with paralysis to control objects in their environment using their thoughts. Here, we demonstrate how the BCI can help restore communication to a man with ALS using only neural signals recorded from the speech-related motor areas of the brain (ventral precentral gyrus). As he tries to speak, our decoding system translates the neural activity into words on a screen. Hence, to answer your question, we are only using brain signals to determine what he's trying to say. The microphone is not involved in interpreting his speech at all (and in fact, speech-to-text algorithms applied to the audio signals are nonsensical). One can hear him speak in the supplementary videos in the NEJM paper, and in the video produced by UC Davis. The eye-tracker is used to help control the GUI, much like an able-bodied person would use a mouse.
  - malwrar 2y ago
    Fascinating, thank you for your response. Best of luck as your team’s work progresses!
hazmazlaz 2y ago
This is so fascinating and amazing, congratulations on the achievements you and your team have made. What are some current limitations of this technology? Where do you think the trajectory will lead in 5 years in terms of capability? Are there long term health and safety risks that have been discovered through the various implementations of this technology since 2004?
- drdavidbrandman 2y ago
  I'm optimistic for the future! Basic science laboratories have provided the foundational insights needed for clinical trials to explore BCI technology in people with paralysis. As academics have de-risked BCI development, there are now multiple companies that are developing fully-implanted devices, including Neuralink, Paradromics, Synchron, and Precision. I hope that we'll be at a point in the next 5 years where I will be able to prescribe one (or perhaps one of many!) of these devices to patients. The progress described in the NEJM paper is happening as part of an ongoing clinical trial called BrainGate2. The clinical trial has been ongoing for over 20 years. The interim safety analysis was recently published about the first 17 years of the clinical trial [0]. https://pubmed.ncbi.nlm.nih.gov/36639237/ https://pubmed.ncbi.nlm.nih.gov/36639237/
Incipient 2y ago
Does the device/algorithm have to be trained on a specific person's brain/physical parameters? Or could this be swapped to another arbitrary person and work essentially the same? The thought behind the question: how similar is everyone's process to construct the word "happy"?

malwrar 2y ago

Congratulations on achieving this, I teared up watching that man’s face in your linked summary video when he was able to communicate with your device! It looks like there are three components: 1.) the device itself 2.) a microphone 3.) eye tracking (??) Is that right? If so, to what degree does a person’s ability to make at least utterances into the microphone matter to the overall system’s performance? Are you heading in the direction of purely using the device itself to determine what patients are intending to say?

drdavidbrandman 2y ago
Thank you for the kind words! A brain computer interface is a device that records brain signals, and allows people with paralysis to control objects in their environment using their thoughts. Here, we demonstrate how the BCI can help restore communication to a man with ALS using only neural signals recorded from the speech-related motor areas of the brain (ventral precentral gyrus). As he tries to speak, our decoding system translates the neural activity into words on a screen. Hence, to answer your question, we are only using brain signals to determine what he's trying to say. The microphone is not involved in interpreting his speech at all (and in fact, speech-to-text algorithms applied to the audio signals are nonsensical). One can hear him speak in the supplementary videos in the NEJM paper, and in the video produced by UC Davis. The eye-tracker is used to help control the GUI, much like an able-bodied person would use a mouse.
- malwrar 2y ago
  Fascinating, thank you for your response. Best of luck as your team’s work progresses!

hazmazlaz 2y ago

This is so fascinating and amazing, congratulations on the achievements you and your team have made. What are some current limitations of this technology? Where do you think the trajectory will lead in 5 years in terms of capability? Are there long term health and safety risks that have been discovered through the various implementations of this technology since 2004?

drdavidbrandman 2y ago
I'm optimistic for the future! Basic science laboratories have provided the foundational insights needed for clinical trials to explore BCI technology in people with paralysis. As academics have de-risked BCI development, there are now multiple companies that are developing fully-implanted devices, including Neuralink, Paradromics, Synchron, and Precision. I hope that we'll be at a point in the next 5 years where I will be able to prescribe one (or perhaps one of many!) of these devices to patients. The progress described in the NEJM paper is happening as part of an ongoing clinical trial called BrainGate2. The clinical trial has been ongoing for over 20 years. The interim safety analysis was recently published about the first 17 years of the clinical trial [0]. https://pubmed.ncbi.nlm.nih.gov/36639237/ https://pubmed.ncbi.nlm.nih.gov/36639237/

Incipient 2y ago

Does the device/algorithm have to be trained on a specific person's brain/physical parameters? Or could this be swapped to another arbitrary person and work essentially the same? The thought behind the question: how similar is everyone's process to construct the word "happy"?