4 ms·

Whether you think it's a good idea or not, there are those of us who can't function without voice programming/input. This video demonstrates full coding profic

Whether you think it's a good idea or not, there are those of us who can't function without voice programming/input. This video demonstrates full coding proficiency with my Talon project, which still needs a lot of work (e.g. no auto-spacing yet) - https://youtu.be/ddFI63dgpaI?t=30 https://youtu.be/ddFI63dgpaI?t=30 - this demo took me around 9min vs a 90wpm typist doing it in a little over 6min. I've also benchmarked inputting over 280 commands per minute with no false recognitions. This is my project, with which I aim to change minds about voice input (even making it viable for real-time gaming): https://talonvoice.com/ https://talonvoice.com/ The Perl video is embarrassing, but that voice engine is only designed for dictating english text. voicecode.io has some major architectural limitations (no dynamic grammars so it requires a Dragon restart to change commands or word lists, adds commands by adding them to the english vocabulary instead of a strict command tree so recognition seriously suffers), aenea, caster, dragonfly are built on a 90's Python project called natlink and have their own issues. "Continuous command recognition" is pretty much a must, which Tavis Rudd's video lacks (notice the pausing between words).

fpig 9y ago
That looks great! Both the speech recognition and the eye tracking look really impressive. You should make this into a commercial project or something. Is there any popular application that already does this stuff and is as good (I am not familiar with this type of software)? I had no idea eye tracking could work this well.
- lunixbochs 9y ago
  As far as I'm concerned, there's nothing remotely in Talon's ballpark for voice, and these demos are just a sort of raw baseline. Every part of Talon will see significant improvements as I complete more of the upcoming features. Tech like Siri/Alexa/Google Assistant is too natural language processing focused to be reflective of the performance I've already demonstrated from a precise control scheme. To my knowledge I am the only available high performance eye mousing project to not require a separate head tracker (I only require a $150 Tobii 4C eye tracker, and I do not use their integrated webcam head tracking system as it is not responsive enough). I'm additionally planning to remove the ~$50-150 Dragon requirement for Mac users in the mid future. I'm very committed to keeping the baseline software not just free, but a net negative cost over using other free solutions. As far as making it commercial, my current plan is to eventually charge a small subscription fee for extremely well integrated video game control systems, and a moderate subscription for professional input schemes like CADD, A/V products, and entire language models built around natural language programming that would ideally enable people who become injured to continue to work without much of any downtime. These are some of the competition: Voice: - http://voicecode.io/ http://voicecode.io/ - https://github.com/dictation-toolbox/aenea https://github.com/dictation-toolbox/aenea - https://github.com/synkarius/caster https://github.com/synkarius/caster - https://www.voicebot.net/ https://www.voicebot.net/ Eye/Head Tracking: - http://precisiongazemouse.com/ http://precisiongazemouse.com/ - http://iris.xcessity.at/ http://iris.xcessity.at/ - https://github.com/trishume/PolyMouse https://github.com/trishume/PolyMouse - https://github.com/trishume/FusionMouse https://github.com/trishume/FusionMouse (huge shout out to trishume, who was very helpful and a big inspiration for my tracking system) One of my upcoming goals is to get a hands-free world-record game speedrun into the Awesome Games Done Quick stream.
mcphage 9y ago
Your speed and accuracy are impressive, but that looks very unpleasant to recite. Human speech tends to have rhythm and cadence, but that sounded to be like reading a phone book. Very precisely spoken, very choppy, no comfortable flow.
- lunixbochs 9y ago
  First, it doesn't feel unpleasant as I do it. There are a few reasons why it might look that way. I have very little practice at this point, especially considering I'm still changing my grammars regularly, so the editing video there is more a technical demo of the software's performance in spite of my own awkwardness while learning. That was my first and only take. I'm basically learning to type again, with cache misses as I try to remember commands and keep track of my position (like trying to remember where keys are, or the next note in a song as you pluck it out). It's also possible the most accurate form of voice input does not sound like natural language. Finally, it's a different mental exercise to copy text that is already written versus writing and editing my own code, so I'm triply out of my element when recording these early videos. Compare to Tavis Rudd's video from 2013, where he pauses between every command: https://youtu.be/8SkdfdXWYaI?t=1050 https://youtu.be/8SkdfdXWYaI?t=1050 I've recorded my own version of his demo today for comparison: https://youtu.be/wt4PR5j7vBE https://youtu.be/wt4PR5j7vBE

fpig 9y ago

That looks great! Both the speech recognition and the eye tracking look really impressive. You should make this into a commercial project or something. Is there any popular application that already does this stuff and is as good (I am not familiar with this type of software)? I had no idea eye tracking could work this well.

lunixbochs 9y ago
As far as I'm concerned, there's nothing remotely in Talon's ballpark for voice, and these demos are just a sort of raw baseline. Every part of Talon will see significant improvements as I complete more of the upcoming features. Tech like Siri/Alexa/Google Assistant is too natural language processing focused to be reflective of the performance I've already demonstrated from a precise control scheme. To my knowledge I am the only available high performance eye mousing project to not require a separate head tracker (I only require a $150 Tobii 4C eye tracker, and I do not use their integrated webcam head tracking system as it is not responsive enough). I'm additionally planning to remove the ~$50-150 Dragon requirement for Mac users in the mid future. I'm very committed to keeping the baseline software not just free, but a net negative cost over using other free solutions. As far as making it commercial, my current plan is to eventually charge a small subscription fee for extremely well integrated video game control systems, and a moderate subscription for professional input schemes like CADD, A/V products, and entire language models built around natural language programming that would ideally enable people who become injured to continue to work without much of any downtime. These are some of the competition: Voice: - http://voicecode.io/ http://voicecode.io/ - https://github.com/dictation-toolbox/aenea https://github.com/dictation-toolbox/aenea - https://github.com/synkarius/caster https://github.com/synkarius/caster - https://www.voicebot.net/ https://www.voicebot.net/ Eye/Head Tracking: - http://precisiongazemouse.com/ http://precisiongazemouse.com/ - http://iris.xcessity.at/ http://iris.xcessity.at/ - https://github.com/trishume/PolyMouse https://github.com/trishume/PolyMouse - https://github.com/trishume/FusionMouse https://github.com/trishume/FusionMouse (huge shout out to trishume, who was very helpful and a big inspiration for my tracking system) One of my upcoming goals is to get a hands-free world-record game speedrun into the Awesome Games Done Quick stream.

mcphage 9y ago

Your speed and accuracy are impressive, but that looks very unpleasant to recite. Human speech tends to have rhythm and cadence, but that sounded to be like reading a phone book. Very precisely spoken, very choppy, no comfortable flow.

lunixbochs 9y ago
First, it doesn't feel unpleasant as I do it. There are a few reasons why it might look that way. I have very little practice at this point, especially considering I'm still changing my grammars regularly, so the editing video there is more a technical demo of the software's performance in spite of my own awkwardness while learning. That was my first and only take. I'm basically learning to type again, with cache misses as I try to remember commands and keep track of my position (like trying to remember where keys are, or the next note in a song as you pluck it out). It's also possible the most accurate form of voice input does not sound like natural language. Finally, it's a different mental exercise to copy text that is already written versus writing and editing my own code, so I'm triply out of my element when recording these early videos. Compare to Tavis Rudd's video from 2013, where he pauses between every command: https://youtu.be/8SkdfdXWYaI?t=1050 https://youtu.be/8SkdfdXWYaI?t=1050 I've recorded my own version of his demo today for comparison: https://youtu.be/wt4PR5j7vBE https://youtu.be/wt4PR5j7vBE