5 ms·

> In the AR and VR future we won’t be typing. Oh please god no full stop imagine how hard it would be to even write a simple hello world program example ellips

> In the AR and VR future we won’t be typing. Oh please god no full stop imagine how hard it would be to even write a simple hello world program example ellipsis linebreak linebreak three backticks no not that delete word delete word backtick backtick backtick c linebreak void main open bracket int argc comma char asterisk argv close bracket space open curly bracket line break tab print-EFF open bracket double-quote percent d backslash n double-quote comma argc close bracket semi-colon line break backtick backtick backtick Chance of success? Zero. Time taken? Three minutes.

m4tthumphrey 9y ago
I found this to be very ironic based on the time it probably took you to type all this out haha!
- shabbyrobe 9y ago
  Hehe yep but the major difference here is that I was chuckling the whole time I was typing. Can you imagine how bad chuckling would make the end result if you were speaking it? Chance of success: -1 billion.
icebraining 9y ago
I won't be switching any time soon, but it's certainly possible: https://www.youtube.com/watch?v=8SkdfdXWYaI https://www.youtube.com/watch?v=8SkdfdXWYaI
bryanrasmussen 9y ago
A voice interface to programming would either necessitate different programming languages, or macros - maybe gestural or vocal - of some sort to take care of the various ordering syntax that languages like { that languages require. As well as absolutely perfect voice recognition, including adjusting to when you have a sore throat or other problems.
- wernsey 9y ago
  I foresee a problem with maintaining code that has been written with gestures or voice. How would that work? Would your IDE play back the voice or display animations of the gestures to you. It just feels as if text is the simplest solution, but I'm open to suggestions.
  - bryanrasmussen 9y ago
    I agree, just trying to think of how it would have to work if it was ever to exist. I would suppose that voice/gesture written code would be stored as text, when reading you could edit via keyboard or your voice/gesture interface. This comes back to languages written for voice - probably if this became feasible a lot of these languages would be compilers from voice to another traditional text target. But like many people here I have a hard time actually believing this would ever be useful.
- soylentcola 9y ago
  In seriousness though, while voice input would not be how you do complex programming, you could argue that there will always be more specific and specialized tools for that. For people who spend most of their time clicking with a mouse or pointing at pictures on a phone screen, I could see voice working with things like IFTTT to create simple "programs" or actions. That's essentially what happens when I say "hey Google, turn on living room lights at 6pm". Someone needed to program the framework for that to happen, but someone also needed to develop the Arduino IDE that I use when I engage in the very limited hobbyist programming that I do. I get to use a simpler interface because someone more specialized and experienced created it with lower level tools. In a sense, that "if time=18:00 then set livingroom_lights=on" command is a very simple program entered via voice command.
drunkkcunt 9y ago
That's why you use Python.
lokedhs 9y ago
This one is old, but I don't think things have improved much since then: https://www.youtube.com/watch?v=MzJ0CytAsec https://www.youtube.com/watch?v=MzJ0CytAsec It's worth a watch. Hilarious, despite the bad sound quality.
- woolvalley 9y ago
  There are improvements on it :) https://voicecode.io/ https://voicecode.io/
  - lunixbochs 9y ago
    and improvements on that: https://talonvoice.com/ https://talonvoice.com/
pvdebbe 9y ago
That's why only brain interfaces can replace keyboards
lunixbochs 9y ago
Whether you think it's a good idea or not, there are those of us who can't function without voice programming/input. This video demonstrates full coding proficiency with my Talon project, which still needs a lot of work (e.g. no auto-spacing yet) - https://youtu.be/ddFI63dgpaI?t=30 https://youtu.be/ddFI63dgpaI?t=30 - this demo took me around 9min vs a 90wpm typist doing it in a little over 6min. I've also benchmarked inputting over 280 commands per minute with no false recognitions. This is my project, with which I aim to change minds about voice input (even making it viable for real-time gaming): https://talonvoice.com/ https://talonvoice.com/ The Perl video is embarrassing, but that voice engine is only designed for dictating english text. voicecode.io has some major architectural limitations (no dynamic grammars so it requires a Dragon restart to change commands or word lists, adds commands by adding them to the english vocabulary instead of a strict command tree so recognition seriously suffers), aenea, caster, dragonfly are built on a 90's Python project called natlink and have their own issues. "Continuous command recognition" is pretty much a must, which Tavis Rudd's video lacks (notice the pausing between words).
- fpig 9y ago
  That looks great! Both the speech recognition and the eye tracking look really impressive. You should make this into a commercial project or something. Is there any popular application that already does this stuff and is as good (I am not familiar with this type of software)? I had no idea eye tracking could work this well.
  - lunixbochs 9y ago
    As far as I'm concerned, there's nothing remotely in Talon's ballpark for voice, and these demos are just a sort of raw baseline. Every part of Talon will see significant improvements as I complete more of the upcoming features. Tech like Siri/Alexa/Google Assistant is too natural language processing focused to be reflective of the performance I've already demonstrated from a precise control scheme. To my knowledge I am the only available high performance eye mousing project to not require a separate head tracker (I only require a $150 Tobii 4C eye tracker, and I do not use their integrated webcam head tracking system as it is not responsive enough). I'm additionally planning to remove the ~$50-150 Dragon requirement for Mac users in the mid future. I'm very committed to keeping the baseline software not just free, but a net negative cost over using other free solutions. As far as making it commercial, my current plan is to eventually charge a small subscription fee for extremely well integrated video game control systems, and a moderate subscription for professional input schemes like CADD, A/V products, and entire language models built around natural language programming that would ideally enable people who become injured to continue to work without much of any downtime. These are some of the competition: Voice: - http://voicecode.io/ http://voicecode.io/ - https://github.com/dictation-toolbox/aenea https://github.com/dictation-toolbox/aenea - https://github.com/synkarius/caster https://github.com/synkarius/caster - https://www.voicebot.net/ https://www.voicebot.net/ Eye/Head Tracking: - http://precisiongazemouse.com/ http://precisiongazemouse.com/ - http://iris.xcessity.at/ http://iris.xcessity.at/ - https://github.com/trishume/PolyMouse https://github.com/trishume/PolyMouse - https://github.com/trishume/FusionMouse https://github.com/trishume/FusionMouse (huge shout out to trishume, who was very helpful and a big inspiration for my tracking system) One of my upcoming goals is to get a hands-free world-record game speedrun into the Awesome Games Done Quick stream.
melling 9y ago
While you were yucking it up, someone already did it. I'd say it takes about a minute: https://youtu.be/YRyYIIFKsdU?t=9m9s https://youtu.be/YRyYIIFKsdU?t=9m9s Of course, there's plenty of room for improvement. Why not have the assistant understand keywords and perform high-level tasks? "import" "create function named main" "return" "Add two int arguments"

m4tthumphrey 9y ago

I found this to be very ironic based on the time it probably took you to type all this out haha!

shabbyrobe 9y ago
Hehe yep but the major difference here is that I was chuckling the whole time I was typing. Can you imagine how bad chuckling would make the end result if you were speaking it? Chance of success: -1 billion.

icebraining 9y ago

I won't be switching any time soon, but it's certainly possible: https://www.youtube.com/watch?v=8SkdfdXWYaI https://www.youtube.com/watch?v=8SkdfdXWYaI

bryanrasmussen 9y ago

A voice interface to programming would either necessitate different programming languages, or macros - maybe gestural or vocal - of some sort to take care of the various ordering syntax that languages like { that languages require. As well as absolutely perfect voice recognition, including adjusting to when you have a sore throat or other problems.

wernsey 9y ago
I foresee a problem with maintaining code that has been written with gestures or voice. How would that work? Would your IDE play back the voice or display animations of the gestures to you. It just feels as if text is the simplest solution, but I'm open to suggestions.
- bryanrasmussen 9y ago
  I agree, just trying to think of how it would have to work if it was ever to exist. I would suppose that voice/gesture written code would be stored as text, when reading you could edit via keyboard or your voice/gesture interface. This comes back to languages written for voice - probably if this became feasible a lot of these languages would be compilers from voice to another traditional text target. But like many people here I have a hard time actually believing this would ever be useful.
soylentcola 9y ago
In seriousness though, while voice input would not be how you do complex programming, you could argue that there will always be more specific and specialized tools for that. For people who spend most of their time clicking with a mouse or pointing at pictures on a phone screen, I could see voice working with things like IFTTT to create simple "programs" or actions. That's essentially what happens when I say "hey Google, turn on living room lights at 6pm". Someone needed to program the framework for that to happen, but someone also needed to develop the Arduino IDE that I use when I engage in the very limited hobbyist programming that I do. I get to use a simpler interface because someone more specialized and experienced created it with lower level tools. In a sense, that "if time=18:00 then set livingroom_lights=on" command is a very simple program entered via voice command.

drunkkcunt 9y ago

That's why you use Python.

lokedhs 9y ago

This one is old, but I don't think things have improved much since then: https://www.youtube.com/watch?v=MzJ0CytAsec https://www.youtube.com/watch?v=MzJ0CytAsec It's worth a watch. Hilarious, despite the bad sound quality.

woolvalley 9y ago
There are improvements on it :) https://voicecode.io/ https://voicecode.io/
- lunixbochs 9y ago
  and improvements on that: https://talonvoice.com/ https://talonvoice.com/

pvdebbe 9y ago

That's why only brain interfaces can replace keyboards

lunixbochs 9y ago

Whether you think it's a good idea or not, there are those of us who can't function without voice programming/input. This video demonstrates full coding proficiency with my Talon project, which still needs a lot of work (e.g. no auto-spacing yet) - https://youtu.be/ddFI63dgpaI?t=30 https://youtu.be/ddFI63dgpaI?t=30 - this demo took me around 9min vs a 90wpm typist doing it in a little over 6min. I've also benchmarked inputting over 280 commands per minute with no false recognitions. This is my project, with which I aim to change minds about voice input (even making it viable for real-time gaming): https://talonvoice.com/ https://talonvoice.com/ The Perl video is embarrassing, but that voice engine is only designed for dictating english text. voicecode.io has some major architectural limitations (no dynamic grammars so it requires a Dragon restart to change commands or word lists, adds commands by adding them to the english vocabulary instead of a strict command tree so recognition seriously suffers), aenea, caster, dragonfly are built on a 90's Python project called natlink and have their own issues. "Continuous command recognition" is pretty much a must, which Tavis Rudd's video lacks (notice the pausing between words).

fpig 9y ago
That looks great! Both the speech recognition and the eye tracking look really impressive. You should make this into a commercial project or something. Is there any popular application that already does this stuff and is as good (I am not familiar with this type of software)? I had no idea eye tracking could work this well.
- lunixbochs 9y ago
  As far as I'm concerned, there's nothing remotely in Talon's ballpark for voice, and these demos are just a sort of raw baseline. Every part of Talon will see significant improvements as I complete more of the upcoming features. Tech like Siri/Alexa/Google Assistant is too natural language processing focused to be reflective of the performance I've already demonstrated from a precise control scheme. To my knowledge I am the only available high performance eye mousing project to not require a separate head tracker (I only require a $150 Tobii 4C eye tracker, and I do not use their integrated webcam head tracking system as it is not responsive enough). I'm additionally planning to remove the ~$50-150 Dragon requirement for Mac users in the mid future. I'm very committed to keeping the baseline software not just free, but a net negative cost over using other free solutions. As far as making it commercial, my current plan is to eventually charge a small subscription fee for extremely well integrated video game control systems, and a moderate subscription for professional input schemes like CADD, A/V products, and entire language models built around natural language programming that would ideally enable people who become injured to continue to work without much of any downtime. These are some of the competition: Voice: - http://voicecode.io/ http://voicecode.io/ - https://github.com/dictation-toolbox/aenea https://github.com/dictation-toolbox/aenea - https://github.com/synkarius/caster https://github.com/synkarius/caster - https://www.voicebot.net/ https://www.voicebot.net/ Eye/Head Tracking: - http://precisiongazemouse.com/ http://precisiongazemouse.com/ - http://iris.xcessity.at/ http://iris.xcessity.at/ - https://github.com/trishume/PolyMouse https://github.com/trishume/PolyMouse - https://github.com/trishume/FusionMouse https://github.com/trishume/FusionMouse (huge shout out to trishume, who was very helpful and a big inspiration for my tracking system) One of my upcoming goals is to get a hands-free world-record game speedrun into the Awesome Games Done Quick stream.

melling 9y ago

While you were yucking it up, someone already did it. I'd say it takes about a minute: https://youtu.be/YRyYIIFKsdU?t=9m9s https://youtu.be/YRyYIIFKsdU?t=9m9s Of course, there's plenty of room for improvement. Why not have the assistant understand keywords and perform high-level tasks? "import" "create function named main" "return" "Add two int arguments"