4 ms·

Interesting that the user has to think of specific letters to spell out a word. I guess the 26 distinct English letters are much easier to parse and separate th

Interesting that the user has to think of specific letters to spell out a word. I guess the 26 distinct English letters are much easier to parse and separate than the untold thousands of words, especially when those words could be used in different contexts. I bet the next step wouldn't be to even parse words to make a sentence. It seems to me the next low hanging fruit would be to enable this machine to parse common ideas. I wonder how complex it would be to translate full sentences like "Good Morning", "I gotta take a dump", or "I'm hungry". It doesn't seem like it would be that much of a leap, since the user already has to imagine the idea of different letters. Admittedly I have no idea how different those concepts are, or how they would express themselves in the brain to be interpreted by the machine.

Communitivity 5y ago
That next step is a leap. I view it as the difference between parsing and processing spoken English and parsing and processing spoken Mandarin Chinese. The letters and numbers are 36 symbols to understand, plus capitalization and punctuation. Understanding words means mapping out the brain pathways for each word. There is actually a path for this that's been done before, in a way. Dragon Naturally Speaking was evolved this way. As I understand it, that evolution took decades. In 1952 Bell Labs came up with Audrey (Automatic Digit Recognition). Voice specific, and could only recognize numbers 0-9. This is where the OP linked Brain Computer Interface (BCI) is. In 1962 IBM revealed Shoebox at the World Fair. Shoebox could understand 16 English words. It would listen to the words and complete an instruction for example adding up numbers and providing the result. Harpy came in 1971. Funded by Darpa and developed through a collaboration between CMU, Stanford and IBM. Harpy cold work with ordinary speech and pick out individual words, but it only had a vocabulary of around 1000 words. In 1974, Kurzweil forms Kurzweil Computer Products (KCP) for development of pattern recognition technology. In 1976, KCP introduces the Kurzweil Reading Machine, combining three technological firsts. In 1982 Dr's Jim and Janet Baker launched Dragon Systems and prototyped a voice recognition system that was based around mathematical models. The Bakers were mathematicians and the system they came up with was based a hidden Markov model – using statistics to predict words, phrases and sentences. In 1983, Kurzweil Music Systems launches a keyboard synthesizer that accurately reproduces the sounds of acoustic instruments. In 1985, Kurzweil Applied Intelligence introduces the first speech-to-text computer program. In 1990, Dragon Dictate was launched as the first general purpose large vocabulary speech to text dictation system. This was a groundbreaking product for Dragon, but it required users to pause between individual words. In 1994, KurzweilVoice for Windows 1.0 is launched, bringing discrete speech command technology to the personal computer environment. In 1995, Kurzweil Technologies is founded. By 1997, the problem of having to pause between words had been overcome and Dragon Naturally Speaking v1 was launched, 45 years after Audrey. In 1997, the Continuous Speech Natural Language Command and Control software is launched as Kurzweil Voice Commands; The Medical Learning Company is formed. In 2000, Kurzweil forms FAT KAT, Inc. to develop artificial intelligence that can make decisions about buying and selling on the stock market. Then in 2001 KTI introduced "Ramona," the virtual reality rock star. Yes, the last two have little (maybe even nothing) to do with speech recognition, but I found them interesting, so I thought you might too. The sources for the above are primarily: [1] http://www.fundinguniverse.com/company-histories/kurzweil-technologies-inc-history/ http://www.fundinguniverse.com/company-histories/kurzweil-te... [2] https://whatsnext.nuance.com/en-gb/dragon-professional/history-speech-recognition/ https://whatsnext.nuance.com/en-gb/dragon-professional/histo...
stevenwoo 5y ago
A paralyzed man wrote a book by blinking when someone pointed at the right letter (sort of like how the guy stranded on Mars in book/movie The Martian sent messages to earth). This story was later made into the movie The Diving Bell and the Butterfly. https://en.wikipedia.org/wiki/The_Diving_Bell_and_the_Butterfly https://en.wikipedia.org/wiki/The_Diving_Bell_and_the_Butter...
- tdeck 5y ago
  This process seems like it would be so much more efficient if he'd learned to blink in Morse code instead.
LynxInLA 5y ago
I think the next step would be predictive text. Have 3+ symbols that correspond to a screen that is in the subject's eye line. Basically just leverage the current tech to streamline this.
cookingrobot 5y ago
Note that this doesn’t work by detecting his “imagined” letters. It’s detecting him trying to move his paralyzed hand through the motion of drawing the letters on paper. By measuring the motor cortex activity it seems to be looking at something closer to an “output” of the brain rather than an internal representation. So detecting imagined letters or words or ideas seems like a different problem than this achievement.
- spiffytech 5y ago
  Does this mean it's plausible to detect sign language instead of handwriting? I'd expect the WPM to be much higher there.
smegger001 5y ago
hmm, I wonder what would be the fastst method wether teaching it the idea of "good morning" or the words "good" and "morning" I would imagine eventually you would end up with system where less common words or complex nuanced ideas would need spelled out and commonly used thought words or sentences could be recognized.

Communitivity 5y ago

That next step is a leap. I view it as the difference between parsing and processing spoken English and parsing and processing spoken Mandarin Chinese. The letters and numbers are 36 symbols to understand, plus capitalization and punctuation. Understanding words means mapping out the brain pathways for each word. There is actually a path for this that's been done before, in a way. Dragon Naturally Speaking was evolved this way. As I understand it, that evolution took decades. In 1952 Bell Labs came up with Audrey (Automatic Digit Recognition). Voice specific, and could only recognize numbers 0-9. This is where the OP linked Brain Computer Interface (BCI) is. In 1962 IBM revealed Shoebox at the World Fair. Shoebox could understand 16 English words. It would listen to the words and complete an instruction for example adding up numbers and providing the result. Harpy came in 1971. Funded by Darpa and developed through a collaboration between CMU, Stanford and IBM. Harpy cold work with ordinary speech and pick out individual words, but it only had a vocabulary of around 1000 words. In 1974, Kurzweil forms Kurzweil Computer Products (KCP) for development of pattern recognition technology. In 1976, KCP introduces the Kurzweil Reading Machine, combining three technological firsts. In 1982 Dr's Jim and Janet Baker launched Dragon Systems and prototyped a voice recognition system that was based around mathematical models. The Bakers were mathematicians and the system they came up with was based a hidden Markov model – using statistics to predict words, phrases and sentences. In 1983, Kurzweil Music Systems launches a keyboard synthesizer that accurately reproduces the sounds of acoustic instruments. In 1985, Kurzweil Applied Intelligence introduces the first speech-to-text computer program. In 1990, Dragon Dictate was launched as the first general purpose large vocabulary speech to text dictation system. This was a groundbreaking product for Dragon, but it required users to pause between individual words. In 1994, KurzweilVoice for Windows 1.0 is launched, bringing discrete speech command technology to the personal computer environment. In 1995, Kurzweil Technologies is founded. By 1997, the problem of having to pause between words had been overcome and Dragon Naturally Speaking v1 was launched, 45 years after Audrey. In 1997, the Continuous Speech Natural Language Command and Control software is launched as Kurzweil Voice Commands; The Medical Learning Company is formed. In 2000, Kurzweil forms FAT KAT, Inc. to develop artificial intelligence that can make decisions about buying and selling on the stock market. Then in 2001 KTI introduced "Ramona," the virtual reality rock star. Yes, the last two have little (maybe even nothing) to do with speech recognition, but I found them interesting, so I thought you might too. The sources for the above are primarily: [1] http://www.fundinguniverse.com/company-histories/kurzweil-technologies-inc-history/ http://www.fundinguniverse.com/company-histories/kurzweil-te... [2] https://whatsnext.nuance.com/en-gb/dragon-professional/history-speech-recognition/ https://whatsnext.nuance.com/en-gb/dragon-professional/histo...

stevenwoo 5y ago

A paralyzed man wrote a book by blinking when someone pointed at the right letter (sort of like how the guy stranded on Mars in book/movie The Martian sent messages to earth). This story was later made into the movie The Diving Bell and the Butterfly. https://en.wikipedia.org/wiki/The_Diving_Bell_and_the_Butterfly https://en.wikipedia.org/wiki/The_Diving_Bell_and_the_Butter...

tdeck 5y ago
This process seems like it would be so much more efficient if he'd learned to blink in Morse code instead.

LynxInLA 5y ago

I think the next step would be predictive text. Have 3+ symbols that correspond to a screen that is in the subject's eye line. Basically just leverage the current tech to streamline this.

cookingrobot 5y ago

Note that this doesn’t work by detecting his “imagined” letters. It’s detecting him trying to move his paralyzed hand through the motion of drawing the letters on paper. By measuring the motor cortex activity it seems to be looking at something closer to an “output” of the brain rather than an internal representation. So detecting imagined letters or words or ideas seems like a different problem than this achievement.

spiffytech 5y ago
Does this mean it's plausible to detect sign language instead of handwriting? I'd expect the WPM to be much higher there.

smegger001 5y ago

hmm, I wonder what would be the fastst method wether teaching it the idea of "good morning" or the words "good" and "morning" I would imagine eventually you would end up with system where less common words or complex nuanced ideas would need spelled out and commonly used thought words or sentences could be recognized.