6 ms·

Numen: Voice Control for Handsfree Computing

100 points by memorable 4y ago

jzellis 4y ago
Dunno what the video is, but it's broken on Firefox mobile at least.
- pimlottc 4y ago
  Broken in Safari on iPhone as well
- pfortuny 4y ago
  Doesn’t work on ipad safari either…
- 58x14 4y ago
  Same here - but I bet it’s the HN hug
- gebbie 4y ago
  I only found this thread today but that's good to know. Here's the original screencast on a peertube: https://diode.zone/w/7ZjccgJ5EJCsES3x3yrkpQ https://diode.zone/w/7ZjccgJ5EJCsES3x3yrkpQ There's also videos of my phone and pi: https://peertube.tv/w/uzMMQ5nbmsHMkDGGVcS1ZB https://peertube.tv/w/uzMMQ5nbmsHMkDGGVcS1ZB https://peertube.tv/w/miurmjVygd6C71EfPk19QU https://peertube.tv/w/miurmjVygd6C71EfPk19QU
gramiro 4y ago
Interesting project for providing better accessibility! Reminded me a bit of those scenes on Blade Runner where Deckard is asking the computer to zoom in a certain area and enhance image :D
- noduerme 4y ago
  > Deckard It does. Bloody awesome. I'm re-watching this video trying to understand some of the shorthand being used. There's "bang" for exclamation mark; "cap drum" (?) for `cd`. I can't figure out what words he uses to invoke `git clone` at 1:27 but it's incredibly futuristic. I wish my daily driver wasn't a Mac these days =(
  - ArchieMaclean 4y ago
    It looks like they have a word (or multiple) for each letter of the alphabet. So CD is "change drum", git clone is " guest ice traps space cap look [Ctrl right - autocomplete]", where you can read the commands from the first letters of each word. Edit: the default 'phrases' are here: https://git.sr.ht/~geb/numen/tree/master/item/phrases https://git.sr.ht/~geb/numen/tree/master/item/phrases
    - simultsop 4y ago
      Find it counter intuitive, like we have to memorize new constants that a programer defines... But having this with english words like next line or page down or page up, gamechanger.
nathias 4y ago
this looks much better than any voice control I've seen so far, I wonder if it requires tiles or you can integrate it with other tiling managers
- gebbie 4y ago
  I only found this thread today but it works everywhere a keyboard would so any desktop environment or in the virtual consoles. I also have a raspberry pi which I use to install operating systems and the like. https://peertube.tv/w/miurmjVygd6C71EfPk19QU https://peertube.tv/w/miurmjVygd6C71EfPk19QU
teknopaul 4y ago
This is soo needed. All big techs use of voice has so far required Internet access and is creepy. Googles is apawling in that it changes so things that did work, stop working. What voice needed was for humans to adjust a little to make the computer work easier. e.g. "Computer" "file save" is much more efficient all round than sending off audio to the bork for AI to try work out what it means.
cube2222 4y ago
It's worth mentioning Talon[0] here, which is a system for offline voice control as well, with great python-based scripting (and also supports eye tracking, though I haven't used it myself). Using your computer or programming with it works like a charm, with some interesting and impressive projects based on it coming out as well, like Cursorless[1]. There's a great strangeloop talk[2] demonstrating talon and the actual state of voice coding, which is how I discovered it (hint: it's much better than you'd expect, and straightforward to learn at that). [0]: https://talonvoice.com/ https://talonvoice.com/ [1]: https://github.com/cursorless-dev/cursorless https://github.com/cursorless-dev/cursorless [2]: https://youtu.be/YKuRkGkf5HU https://youtu.be/YKuRkGkf5HU Disclaimer: not affiliated, just a happy occasional user
- comfypotato 4y ago
  Was hoping for a comparison to Talon. Talon is incredible. I’m particularly interested to see if any project spawns focused around augmenting the keyboard as opposed to replacing it in a programming context.
  - rom-antics 4y ago
    You might be interested in Cursorless's experimental keyboard mode: https://www.cursorless.org/docs/user/experimental/keyboard/modal/ https://www.cursorless.org/docs/user/experimental/keyboard/m...
- yewenjie 4y ago
  Last time I checked Talon's models were very bad at recognizing my voice. Does it support better models now, for example OpenAI's Whisper?
  - caternoster 4y ago
    The creator of Talon has tested the Whisper models extensively[0]. [0]: https://twitter.com/lunixbochs/status/1574848899897884672 https://twitter.com/lunixbochs/status/1574848899897884672
ArchieMaclean 4y ago
I like this a lot. This is built upon Vosk [0], open source voice recognition. I must try it for some of my own projects! [0] https://alphacephei.com/vosk/ https://alphacephei.com/vosk/
Xevi 4y ago
Impressive, I'm looking forward to seeing more of this project. Did you draw inspiration from Talon? There are a lot of similarities when it comes to the voice commands.
- gebbie 4y ago
  Thank you! I only found this thread today. I was inspired by talon but all I really wanted was some phrases that worked everywhere and to use normal tools like vim and a tiling window manager, and hoped it could all be foss. I think talon's more about app specific phrases and trying to take the role of your text editor and window manager.
deleted 4y ago
[deleted]
CodexArcana 4y ago
I'm only interested if you have to activate it by saying "Hello... Numen..." ala Seinfeld.
- TheHumanist 4y ago
  Hello, Jerry...
zerop 4y ago
Intresting. Why will someone use handsfree computing. It's slow, i would rather type.
- replicanteven 4y ago
  TBF, you need working hands to use hands-on computers. Plus, the offline part could make a good starting point for a DIY personal assistant. That said, their "getting started" sounds...esoteric. >There normally isn't any output but you should be able to type "hey" by saying "hoof eve yank" and transcribe a sentence after saying "scribe". You can terminate it by pressing Ctrl+c or saying "troll cap".
- simplyinfinity 4y ago
  Because they might be physically impaired in some way. Or have severe Repetitive Strain Injury (RSI).
- deleted 4y ago
  [deleted]
- nova38 4y ago
  In addition to the other points above, i have found that in some workflows I have found that using talon and cursorless i am faster than I was using just a keyboard and mouse. I have had some bad RSI flare ups over the past few years and they have been a lifesaver getting me though my undergrad and most of my masters. Even during the times in-between flare ups i found that using them to editing a latex document was much more convenient being able to reference paragraphs/sentences/words and then change them or move them around.
penjelly 4y ago
interesting... i just broke my arm so this is potentially useful for me. The words you use will take some getting used to though.

100 points by memorable 4y ago

jzellis 4y ago

Dunno what the video is, but it's broken on Firefox mobile at least.

pimlottc 4y ago
Broken in Safari on iPhone as well
pfortuny 4y ago
Doesn’t work on ipad safari either…
58x14 4y ago
Same here - but I bet it’s the HN hug
gebbie 4y ago
I only found this thread today but that's good to know. Here's the original screencast on a peertube: https://diode.zone/w/7ZjccgJ5EJCsES3x3yrkpQ https://diode.zone/w/7ZjccgJ5EJCsES3x3yrkpQ There's also videos of my phone and pi: https://peertube.tv/w/uzMMQ5nbmsHMkDGGVcS1ZB https://peertube.tv/w/uzMMQ5nbmsHMkDGGVcS1ZB https://peertube.tv/w/miurmjVygd6C71EfPk19QU https://peertube.tv/w/miurmjVygd6C71EfPk19QU

gramiro 4y ago

Interesting project for providing better accessibility! Reminded me a bit of those scenes on Blade Runner where Deckard is asking the computer to zoom in a certain area and enhance image :D

noduerme 4y ago
> Deckard It does. Bloody awesome. I'm re-watching this video trying to understand some of the shorthand being used. There's "bang" for exclamation mark; "cap drum" (?) for `cd`. I can't figure out what words he uses to invoke `git clone` at 1:27 but it's incredibly futuristic. I wish my daily driver wasn't a Mac these days =(
- ArchieMaclean 4y ago
  It looks like they have a word (or multiple) for each letter of the alphabet. So CD is "change drum", git clone is " guest ice traps space cap look [Ctrl right - autocomplete]", where you can read the commands from the first letters of each word. Edit: the default 'phrases' are here: https://git.sr.ht/~geb/numen/tree/master/item/phrases https://git.sr.ht/~geb/numen/tree/master/item/phrases
  - simultsop 4y ago
    Find it counter intuitive, like we have to memorize new constants that a programer defines... But having this with english words like next line or page down or page up, gamechanger.

nathias 4y ago

this looks much better than any voice control I've seen so far, I wonder if it requires tiles or you can integrate it with other tiling managers

gebbie 4y ago
I only found this thread today but it works everywhere a keyboard would so any desktop environment or in the virtual consoles. I also have a raspberry pi which I use to install operating systems and the like. https://peertube.tv/w/miurmjVygd6C71EfPk19QU https://peertube.tv/w/miurmjVygd6C71EfPk19QU

teknopaul 4y ago

This is soo needed. All big techs use of voice has so far required Internet access and is creepy. Googles is apawling in that it changes so things that did work, stop working. What voice needed was for humans to adjust a little to make the computer work easier. e.g. "Computer" "file save" is much more efficient all round than sending off audio to the bork for AI to try work out what it means.

cube2222 4y ago

It's worth mentioning Talon[0] here, which is a system for offline voice control as well, with great python-based scripting (and also supports eye tracking, though I haven't used it myself). Using your computer or programming with it works like a charm, with some interesting and impressive projects based on it coming out as well, like Cursorless[1]. There's a great strangeloop talk[2] demonstrating talon and the actual state of voice coding, which is how I discovered it (hint: it's much better than you'd expect, and straightforward to learn at that). [0]: https://talonvoice.com/ https://talonvoice.com/ [1]: https://github.com/cursorless-dev/cursorless https://github.com/cursorless-dev/cursorless [2]: https://youtu.be/YKuRkGkf5HU https://youtu.be/YKuRkGkf5HU Disclaimer: not affiliated, just a happy occasional user

comfypotato 4y ago
Was hoping for a comparison to Talon. Talon is incredible. I’m particularly interested to see if any project spawns focused around augmenting the keyboard as opposed to replacing it in a programming context.
- rom-antics 4y ago
  You might be interested in Cursorless's experimental keyboard mode: https://www.cursorless.org/docs/user/experimental/keyboard/modal/ https://www.cursorless.org/docs/user/experimental/keyboard/m...
yewenjie 4y ago
Last time I checked Talon's models were very bad at recognizing my voice. Does it support better models now, for example OpenAI's Whisper?
- caternoster 4y ago
  The creator of Talon has tested the Whisper models extensively[0]. [0]: https://twitter.com/lunixbochs/status/1574848899897884672 https://twitter.com/lunixbochs/status/1574848899897884672

ArchieMaclean 4y ago

I like this a lot. This is built upon Vosk [0], open source voice recognition. I must try it for some of my own projects! [0] https://alphacephei.com/vosk/ https://alphacephei.com/vosk/

Xevi 4y ago

Impressive, I'm looking forward to seeing more of this project. Did you draw inspiration from Talon? There are a lot of similarities when it comes to the voice commands.

gebbie 4y ago
Thank you! I only found this thread today. I was inspired by talon but all I really wanted was some phrases that worked everywhere and to use normal tools like vim and a tiling window manager, and hoped it could all be foss. I think talon's more about app specific phrases and trying to take the role of your text editor and window manager.

deleted 4y ago

[deleted]

CodexArcana 4y ago

I'm only interested if you have to activate it by saying "Hello... Numen..." ala Seinfeld.

TheHumanist 4y ago
Hello, Jerry...

zerop 4y ago

Intresting. Why will someone use handsfree computing. It's slow, i would rather type.

replicanteven 4y ago
TBF, you need working hands to use hands-on computers. Plus, the offline part could make a good starting point for a DIY personal assistant. That said, their "getting started" sounds...esoteric. >There normally isn't any output but you should be able to type "hey" by saying "hoof eve yank" and transcribe a sentence after saying "scribe". You can terminate it by pressing Ctrl+c or saying "troll cap".
simplyinfinity 4y ago
Because they might be physically impaired in some way. Or have severe Repetitive Strain Injury (RSI).
deleted 4y ago
[deleted]
nova38 4y ago
In addition to the other points above, i have found that in some workflows I have found that using talon and cursorless i am faster than I was using just a keyboard and mouse. I have had some bad RSI flare ups over the past few years and they have been a lifesaver getting me though my undergrad and most of my masters. Even during the times in-between flare ups i found that using them to editing a latex document was much more convenient being able to reference paragraphs/sentences/words and then change them or move them around.

penjelly 4y ago

interesting... i just broke my arm so this is potentially useful for me. The words you use will take some getting used to though.