5 ms·

Work on using AI to brain activity into speech

72 points by Osiris30 8y ago

degenerate 8y ago
The eeriness of the second audio clip reminds me of the first recorded voice by Edouard-Leon Scott de Martinville: https://www.youtube.com/watch?v=q7Gi6j4w3DY https://www.youtube.com/watch?v=q7Gi6j4w3DY
myth_drannon 8y ago
How does it compare with what https://www.ctrl-labs.com/ https://www.ctrl-labs.com/ are doing? It seems that they get the intent just from having a device attached to the arm, so is it really necessary to attach the electrodes to the brain? Because if the person is reading silently you can still get the signal going from the brain to the muscles in the throat/mouth, can it be just attached to the neck?
- fundamental 8y ago
  Typically the end target for applications like this are for patients who have some impairment which prevents normal vocalization which would mean that the EMG around the throat/mouth would not be reliably available.
elisharobinson 8y ago
Stephen hawking: AAAAHHHH you got be kidding me
- jacobtwotwo 8y ago
  This had me legitimately laughing out loud. Thank you ^_^
withhighprod 8y ago
Given some cleverly crafted hint, seems this can be used to extract sensitive information!
fundamental 8y ago
After working on the same task ~3 years back I will say: Be skeptical about results. Neural activity shifts over the course of days, and experiments without long term recordings will produce results which look a lot better than what can be obtained in realistic scenarios. As the article also mentions, there is a sizable question about the differences in signals when vocalization is not possible. I've seen some of that data, and in my opinion we're still a good whiles away from being able to use that data in any practical sense.
- lvbu 8y ago
  Iam very interested in brain computer interfaces. It would be great if you can share any of your work here..
  - fundamental 8y ago
    Poster from extracting speech articulation events using local field potentials (electrodes inserted into brain matter) http://fundamental-code.com/tmp/2015-neurosig.pdf http://fundamental-code.com/tmp/2015-neurosig.pdf
AnIdiotOnTheNet 8y ago
With these kinds of MMIs, I often wonder if it wouldn't be better to train the mind to the machine instead of the other way around. What I mean is, have the machine convert certain brain activity into phonemes or some other primitive and allow the human mind to do what it is already pretty good at: learn to use a tool. Since I'm not an expert in this domain, I assume there are reasons this isn't how it is done. Is anyone aware of what they are?
- fundamental 8y ago
  To a degree there is some learning on both sides. I don't know if I've seen it mentioned in detail in papers that I've read, but it was a general tone at a few confs when dealing with coarse motor functionality. As per why the same approach is not well suited for speech, one (of many) components is the data bandwidth you need for speech to happen at a reasonable rate. For many applications the ML involved in the brain computer interface is only able to extract a few bits per second. It works for applications where the output is coarse (e.g. up/down/left/right) or where large delays are acceptable, but it's not the best fit for speech.
  - 0xdeadbeefbabe 8y ago
    > extract a few bits per second That's surprising. I thought it would be a lot higher than 3.3 bits for 10 fingers (probably not worth considering in this context).
- 0xdeadbeefbabe 8y ago
  To me, this is the only hope. Herf says it is one approach at the end of the article. > If they can hear the computer's speech interpretation in real time, they may be able to adjust their thoughts to get the result they want. I wonder if thought adjustments are intuitive or tricky.
deleted 8y ago
[deleted]

72 points by Osiris30 8y ago

The eeriness of the second audio clip reminds me of the first recorded voice by Edouard-Leon Scott de Martinville: https://www.youtube.com/watch?v=q7Gi6j4w3DY https://www.youtube.com/watch?v=q7Gi6j4w3DY

myth_drannon 8y ago

How does it compare with what https://www.ctrl-labs.com/ https://www.ctrl-labs.com/ are doing? It seems that they get the intent just from having a device attached to the arm, so is it really necessary to attach the electrodes to the brain? Because if the person is reading silently you can still get the signal going from the brain to the muscles in the throat/mouth, can it be just attached to the neck?

fundamental 8y ago
Typically the end target for applications like this are for patients who have some impairment which prevents normal vocalization which would mean that the EMG around the throat/mouth would not be reliably available.

elisharobinson 8y ago

Stephen hawking: AAAAHHHH you got be kidding me

jacobtwotwo 8y ago
This had me legitimately laughing out loud. Thank you ^_^

withhighprod 8y ago

Given some cleverly crafted hint, seems this can be used to extract sensitive information!

fundamental 8y ago

After working on the same task ~3 years back I will say: Be skeptical about results. Neural activity shifts over the course of days, and experiments without long term recordings will produce results which look a lot better than what can be obtained in realistic scenarios. As the article also mentions, there is a sizable question about the differences in signals when vocalization is not possible. I've seen some of that data, and in my opinion we're still a good whiles away from being able to use that data in any practical sense.

lvbu 8y ago
Iam very interested in brain computer interfaces. It would be great if you can share any of your work here..
- fundamental 8y ago
  Poster from extracting speech articulation events using local field potentials (electrodes inserted into brain matter) http://fundamental-code.com/tmp/2015-neurosig.pdf http://fundamental-code.com/tmp/2015-neurosig.pdf

AnIdiotOnTheNet 8y ago

With these kinds of MMIs, I often wonder if it wouldn't be better to train the mind to the machine instead of the other way around. What I mean is, have the machine convert certain brain activity into phonemes or some other primitive and allow the human mind to do what it is already pretty good at: learn to use a tool. Since I'm not an expert in this domain, I assume there are reasons this isn't how it is done. Is anyone aware of what they are?

fundamental 8y ago
To a degree there is some learning on both sides. I don't know if I've seen it mentioned in detail in papers that I've read, but it was a general tone at a few confs when dealing with coarse motor functionality. As per why the same approach is not well suited for speech, one (of many) components is the data bandwidth you need for speech to happen at a reasonable rate. For many applications the ML involved in the brain computer interface is only able to extract a few bits per second. It works for applications where the output is coarse (e.g. up/down/left/right) or where large delays are acceptable, but it's not the best fit for speech.
- 0xdeadbeefbabe 8y ago
  > extract a few bits per second That's surprising. I thought it would be a lot higher than 3.3 bits for 10 fingers (probably not worth considering in this context).
0xdeadbeefbabe 8y ago
To me, this is the only hope. Herf says it is one approach at the end of the article. > If they can hear the computer's speech interpretation in real time, they may be able to adjust their thoughts to get the result they want. I wonder if thought adjustments are intuitive or tricky.

deleted 8y ago

[deleted]