5 ms·

Voice Assistant for VSCode

125 points by b4rtaz__ 5y ago

sandoche 5y ago
Looks very cool well done!
ExcavateGrandMa 5y ago
it's me or you missing a driver for your cursor movements? I don't imagine myself talking to computer... but I might try that for the fun with my awfulite editor :)... gl.
farbodsaraf 5y ago
This makes it more accessible for many!
oaiey 5y ago
How is the multilinguality achieved? By code snippets or is there some language server support?
- Xevi 5y ago
  I'm also wondering this. He says that it works with _every_ language, which leads me to believe that all it does is to paste in code snippets.
- b4rtaz__ 5y ago
  You put voice-assistant.json with your snippets to the root folder of the project. You can define in this file what you want. Check this example: https://github.com/b4rtaz/voice-assistant/blob/master/media/voice-assistant.json https://github.com/b4rtaz/voice-assistant/blob/master/media/...
  - oaiey 5y ago
    I think the next step is getting a huge catalogue of these actions defined. We speak tons of "codes" already in our life's, learning some more to speak (programming) code will not be too hard I guess.
traspler 5y ago
This is pretty nice :) My first thought was that our eyes and hands do all the work; our mouth and ears are untapped resources in the quest to become true 100x engineers ;) All joking aside, I am interested in how well this might work outside of a11y use-cases. Speaking is just so natural. It doesn't have to be used exclusively but I do want to find out if there are cases where it's just nicer to say a command during coding than remembering all kinds of keyboard shortcuts. I always wonder if a more hybrid approach of using touch, speaking and typing for various situations could feel better than keyboard all the way.
- halsom 5y ago
  Would be nice to use for opening files if that could be configured to work in a suitable fashion.
- sumnole 5y ago
  Don't forget to use your feet https://news.ycombinator.com/item?id=26430466 https://news.ycombinator.com/item?id=26430466
  - croon 5y ago
    Dance Dance Recursion
- itsbits 5y ago
  hybrid approach should really be handy. Don't think only voice coding for enabled, becoming standard anytime soon.
- prawn 5y ago
  I've often imagined a voice-command layer for an OS or apps that doesn't steal focus. So, you could be working in a particular app, and be giving instructions by voice to prepare other parts of your workflow.
  - melling 5y ago
    A voice layer on an iPad could make it a better productivity device.
learc83 5y ago
This looks pretty cool, but it’s windows only and doesn’t work outside of VS code. If you want something more full features that works everywhere, I’ve used https://talonvoice.com/ https://talonvoice.com/ for a while now.
- deepstack 5y ago
  This is suppose to Free/Libre. Is the source code some where? Can't seem to find it. Also the homebrew seems to be broken. https://talonvoice.com/update/pgUuEYK3vzmYQtF2PMgOyK/appcast.xml https://talonvoice.com/update/pgUuEYK3vzmYQtF2PMgOyK/appcast...
  - Xevi 5y ago
    Talon is not open source, but it's free to use.
  - dwiel 5y ago
    The author of talon doesnt support homebrew installation. Other people try to add it, but it doesn't work to install that way. Have you tried the installer on the website?
  - Centigonal 5y ago
    Talon is free/gratis - It's not open source.
- melling 5y ago
  at least it appears to be high level. For years people would always comment “I can type faster”, not realizing that we should be able also make it smarter than word by word, or character. https://youtu.be/hGPNs5C1Lp0 https://youtu.be/hGPNs5C1Lp0 Notice this guy is also using his “hat” as a pointing device
  - tmccrary55 5y ago
    Neat but it seems like your neck would pay a price. The voice commands are also cool but needing to pause between each one seems like a huge drawback, compared to typing where I can just blaze through.
lazyresearcher 5y ago
Is this using IBM Watson for STT? Are you assuming the costs? Or does it use some windows dictation api?
- CrazyStat 5y ago
  It's using the Windows api.
twh270 5y ago
Incompatible with "open plan" offices, one or the other will win. Guess which one I'm rooting for. :-)
therealunreal 5y ago
Looks nice! Here's the source for the server component: https://github.com/b4rtaz/voice-assistant-net-server/blob/master/VoiceAssistant.Server/SpeechToText/SpeechToTextService.cs#L17 https://github.com/b4rtaz/voice-assistant-net-server/blob/ma...
MauranKilom 5y ago
We have come a long way: https://www.youtube.com/watch?v=MzJ0CytAsec https://www.youtube.com/watch?v=MzJ0CytAsec
- melling 5y ago
  that’s 14 years ago. i would have expected voice programming to be solved by now. The original iPhone was introduced in 2007. Ray Kurzweil’s predictions are taking longer than expected https://singularityhub.com/2015/01/26/ray-kurzweils-mind-boggling-predictions-for-the-next-25-years/ https://singularityhub.com/2015/01/26/ray-kurzweils-mind-bog...
doitLP 5y ago
If this this interesting, check out https://serenade.ai/ https://serenade.ai/ I’ve only taken it for a test run but it seems really good and smooth.
danpalmer 5y ago
There was a great talk from 2013 about coding with voice commands: https://www.youtube.com/watch?v=8SkdfdXWYaI https://www.youtube.com/watch?v=8SkdfdXWYaI It's by a developer who developed RSI and had to find another way to write code. He uses a combination of Dragon and custom Python scripts to control Emacs. The fascinating bit for me was the language he created around text navigation and manipulation. Lots of custom short words to optimise the amount of speaking he actually had to do. Really worth a watch for anyone interested in this. If you want a quick demo, this part of the video is fairly representative: https://youtu.be/8SkdfdXWYaI?t=1034 https://youtu.be/8SkdfdXWYaI?t=1034
r0b05 5y ago
I cannot get it to recognise any commands unfortunately.

125 points by b4rtaz__ 5y ago

Looks very cool well done!

it's me or you missing a driver for your cursor movements? I don't imagine myself talking to computer... but I might try that for the fun with my awfulite editor :)... gl.

farbodsaraf 5y ago

This makes it more accessible for many!

oaiey 5y ago

How is the multilinguality achieved? By code snippets or is there some language server support?

Xevi 5y ago
I'm also wondering this. He says that it works with _every_ language, which leads me to believe that all it does is to paste in code snippets.
b4rtaz__ 5y ago
You put voice-assistant.json with your snippets to the root folder of the project. You can define in this file what you want. Check this example: https://github.com/b4rtaz/voice-assistant/blob/master/media/voice-assistant.json https://github.com/b4rtaz/voice-assistant/blob/master/media/...
- oaiey 5y ago
  I think the next step is getting a huge catalogue of these actions defined. We speak tons of "codes" already in our life's, learning some more to speak (programming) code will not be too hard I guess.

traspler 5y ago

This is pretty nice :) My first thought was that our eyes and hands do all the work; our mouth and ears are untapped resources in the quest to become true 100x engineers ;) All joking aside, I am interested in how well this might work outside of a11y use-cases. Speaking is just so natural. It doesn't have to be used exclusively but I do want to find out if there are cases where it's just nicer to say a command during coding than remembering all kinds of keyboard shortcuts. I always wonder if a more hybrid approach of using touch, speaking and typing for various situations could feel better than keyboard all the way.

halsom 5y ago
Would be nice to use for opening files if that could be configured to work in a suitable fashion.
sumnole 5y ago
Don't forget to use your feet https://news.ycombinator.com/item?id=26430466 https://news.ycombinator.com/item?id=26430466
- croon 5y ago
  Dance Dance Recursion
itsbits 5y ago
hybrid approach should really be handy. Don't think only voice coding for enabled, becoming standard anytime soon.
prawn 5y ago
I've often imagined a voice-command layer for an OS or apps that doesn't steal focus. So, you could be working in a particular app, and be giving instructions by voice to prepare other parts of your workflow.
- melling 5y ago
  A voice layer on an iPad could make it a better productivity device.

learc83 5y ago

This looks pretty cool, but it’s windows only and doesn’t work outside of VS code. If you want something more full features that works everywhere, I’ve used https://talonvoice.com/ https://talonvoice.com/ for a while now.

deepstack 5y ago
This is suppose to Free/Libre. Is the source code some where? Can't seem to find it. Also the homebrew seems to be broken. https://talonvoice.com/update/pgUuEYK3vzmYQtF2PMgOyK/appcast.xml https://talonvoice.com/update/pgUuEYK3vzmYQtF2PMgOyK/appcast...
- Xevi 5y ago
  Talon is not open source, but it's free to use.
- dwiel 5y ago
  The author of talon doesnt support homebrew installation. Other people try to add it, but it doesn't work to install that way. Have you tried the installer on the website?
- Centigonal 5y ago
  Talon is free/gratis - It's not open source.
melling 5y ago
at least it appears to be high level. For years people would always comment “I can type faster”, not realizing that we should be able also make it smarter than word by word, or character. https://youtu.be/hGPNs5C1Lp0 https://youtu.be/hGPNs5C1Lp0 Notice this guy is also using his “hat” as a pointing device
- tmccrary55 5y ago
  Neat but it seems like your neck would pay a price. The voice commands are also cool but needing to pause between each one seems like a huge drawback, compared to typing where I can just blaze through.

lazyresearcher 5y ago

Is this using IBM Watson for STT? Are you assuming the costs? Or does it use some windows dictation api?

CrazyStat 5y ago
It's using the Windows api.

twh270 5y ago

Incompatible with "open plan" offices, one or the other will win. Guess which one I'm rooting for. :-)

therealunreal 5y ago

Looks nice! Here's the source for the server component: https://github.com/b4rtaz/voice-assistant-net-server/blob/master/VoiceAssistant.Server/SpeechToText/SpeechToTextService.cs#L17 https://github.com/b4rtaz/voice-assistant-net-server/blob/ma...

MauranKilom 5y ago

We have come a long way: https://www.youtube.com/watch?v=MzJ0CytAsec https://www.youtube.com/watch?v=MzJ0CytAsec

melling 5y ago
that’s 14 years ago. i would have expected voice programming to be solved by now. The original iPhone was introduced in 2007. Ray Kurzweil’s predictions are taking longer than expected https://singularityhub.com/2015/01/26/ray-kurzweils-mind-boggling-predictions-for-the-next-25-years/ https://singularityhub.com/2015/01/26/ray-kurzweils-mind-bog...

doitLP 5y ago

If this this interesting, check out https://serenade.ai/ https://serenade.ai/ I’ve only taken it for a test run but it seems really good and smooth.

danpalmer 5y ago

There was a great talk from 2013 about coding with voice commands: https://www.youtube.com/watch?v=8SkdfdXWYaI https://www.youtube.com/watch?v=8SkdfdXWYaI It's by a developer who developed RSI and had to find another way to write code. He uses a combination of Dragon and custom Python scripts to control Emacs. The fascinating bit for me was the language he created around text navigation and manipulation. Lots of custom short words to optimise the amount of speaking he actually had to do. Really worth a watch for anyone interested in this. If you want a quick demo, this part of the video is fairly representative: https://youtu.be/8SkdfdXWYaI?t=1034 https://youtu.be/8SkdfdXWYaI?t=1034

r0b05 5y ago

I cannot get it to recognise any commands unfortunately.