6 ms·

This post starts out talking about expecting to spend around $1,000. There are at least two cross-platform projects where the biggest expense is a microphone i

This post starts out talking about expecting to spend around $1,000. There are at least two cross-platform projects where the biggest expense is a microphone instead of software. 1. My project, Talon. Windows/Linux/Mac support, and a first party local speech recognition engine that is pretty good and getting better. It’s free, but the engine is in a private beta (which is $15/mo to support development, optional if there’s a financial issue). 2. Serenade. They are VC backed. Currently free, unsure about their longer term plans. They use cloud based recognition.

pumanoir 7y ago
I have intermittent RSI and I've been using Talon for 1.5 years for programming (web) with intellij and spacemacs. I reckon I'm as productive as using my hands if I use only voice. When my hands don't hurt and i mix talon with my hands I feel i can do more than I could do with my hands alone. Thanks, lunixbochs. Talon is great.
- bobakanoosh 7y ago
  Can you give some examples of how you intermix the two? I'm curious in trying this out.
  - pumanoir 7y ago
    When using vim keys for example, to jump to the beginning of the line I press ^ then press 'dw' keys delete first word in a line. So I do the caret with my voice and the other press with my hand which is faster than doing it by hand alone.
    - lunixbochs 7y ago
      Interesting, is that something you’d consider using the noise input for once I finish that? (As noise will be lower latency, you’ll have about 40 noises to map, and it’s “out of band” with speech)
      - pumanoir 7y ago
        Sure! Id love that. I only use one noise at the moment for mouse clicking (since I only know one sound). Could you point me to a video that shows an example of each of the 40 available sounds?
        lunixbochs 7y ago
        They aren’t available for use yet. https://noise.talonvoice.com https://noise.talonvoice.com is the submission collection site for the training data. Each noise there has a description and audio example. When they’re available you’ll be able to bind them just like voice commands.
tucosan 7y ago
Actual links to the projects would be useful. ;)
- femiagbabiaka 7y ago
  1. https://talonvoice.com/ https://talonvoice.com/ 2. https://serenade.ai/beta https://serenade.ai/beta
tucosan 7y ago
Here's a video outlining Talon's [1] UX: https://youtu.be/ddFI63dgpaI https://youtu.be/ddFI63dgpaI [1]: https://talonvoice.com/ https://talonvoice.com/
- lunixbochs 7y ago
  That video is fairly old. I’ve polished my workflow a fair bit since then. My advent of code videos described here [1] are probably more representative, and the later videos in the playlist even show the keys that are pressed with each command. [1] https://news.ycombinator.com/item?id=22405178 https://news.ycombinator.com/item?id=22405178 Fun tidbit: Most of the commands I use in vim are just the Talon alphabet.
iudqnolq 7y ago
I have a non-technical friend who says Dragon is too unreliable to be usable. My intuition was that the problem is probably somewhere else and that if Dragon doesn't work nothing will. I'm under the assumption that competing voice recognition software competes on price. Is that assumption wrong? (For example, I suspect the microphone their school supplies them with may be no good)
- daanzu 7y ago
  For what it's worth, my voice is quite abnormal, so most untrained speech recognition is terrible for me, and even performing the normal "training" for Dragon still resulted in very poor accuracy. However, apparently their training is quite limited, because once I developed my Kaldi Active Grammar [1], and did my own direct training, the results were fantastic in comparison, with orders of magnitude better accuracy. Also, Dragon has many commands that cannot be disabled, and if your accuracy is already low, more commands available means more possible things to get wrong. Something like Dragonfly with KaldiAG could allow you to reduce the command set, and improve the practical accuracy. So, more personalized training/software may work better. [1] https://github.com/daanzu/kaldi-active-grammar https://github.com/daanzu/kaldi-active-grammar
  - iudqnolq 7y ago
    Very interesting. I'll take a look and see if I can easily package that in a way someone non-technical could use
    - daanzu 7y ago
      If they are using Windows, packaging is quite easy: see the winpython distribution of KaldiAG. The more difficult part is writing the commands: it is not hard programming, but it is technical. But if they can describe what they would want, someone else may be able to write it easily. The personalized training is still pretty new and raw, and it needs a lot of setup to do the training itself. Without knowing more about what your friend found problematic, it is hard to say what could help the most.
daanzu 7y ago
This comment fails to mention Dragonfly with my Kaldi Active Grammar backend [1], which is cross platform (Windows/Linux now and Mac functional and to be released soon), completely free with no private beta features (although I do accept donations), and 100% open source (unlike Talon). The speech recognition is local, with extremely low latency. See the video demonstration [2] on the project page. I think the underlying Kaldi engine delivers unmatched accuracy as a free non-commercial engine. I created Kaldi Active Grammar because I didn't trust relying on closed source software for something so crucial to my productivity, where a decision by an outside party determines whether I can function. As a bonus, open source means I can make it work better to fit my needs than closed source ever could. Furthermore, the original article mentions Caster (which is built on Dragonfly), but doesn't mention that KaldiAG works with it, and that work is underway to expand Caster's platform support. [1] https://github.com/daanzu/kaldi-active-grammar https://github.com/daanzu/kaldi-active-grammar [2] https://youtu.be/Qk1mGbIJx3s https://youtu.be/Qk1mGbIJx3s
- lunixbochs 7y ago
  I think to state "unmatched accuracy" in good faith we should actually come up with a common benchmark and measure against it. I believe there aren't really any clean benchmarks for command accuracy floating around (it would be ideal if we used a strict grammar to properly measure the command decoders), and a wav2letter model holds the state of the art for librispeech WER% as of 2019. I found your measurement here [1] which is against an unknown wav2letter acoustic+language model pair, as the web demo is at any given point in time running an arbitrary model based on having users test in-progress models, and it has never been running the model I am currently shipping with Talon. (As a small example, the unfinished wav2letter experiment I am training right now has a 3.17% WER on speech commands, and 6.86% WER on librispeech clean, both numbers without using a language model) [1] https://github.com/daanzu/kaldi-active-grammar/blob/master/docs/models.md https://github.com/daanzu/kaldi-active-grammar/blob/master/d...
  - daanzu 7y ago
    I am all for devising a good, fair apples to apples comparison. If you have any suggestions, let me know. In lieu of that, I use what I have available. While accuracy numbers from papers are informative and interesting, I don't think they directly apply to our usage particularly well. I would prefer to use numbers from actual usage.
_-___________-_ 7y ago
On the homepage for Talon it lists macOS under dependencies. I've actually come across the homepage before and didn't look into it further because I thought it was Mac-only.
- _-___________-_ 7y ago
  Hm, it does seem to be Mac-only? You mention "Windows/Linux/Mac support" in your comment; is Linux support released yet?
  - lunixbochs 7y ago
    Windows/Linux are in earlier beta, see the patreon posts for more info on status. There are about 20 full time users for each OS beta. I really should update the site, thanks for reminding me it's confusing.
GordonS 7y ago
Oh, it looks like Talon is finally available for Windows! I've been waiting to try Talon for ages - glad I can give it a try now.

pumanoir 7y ago

I have intermittent RSI and I've been using Talon for 1.5 years for programming (web) with intellij and spacemacs. I reckon I'm as productive as using my hands if I use only voice. When my hands don't hurt and i mix talon with my hands I feel i can do more than I could do with my hands alone. Thanks, lunixbochs. Talon is great.

bobakanoosh 7y ago
Can you give some examples of how you intermix the two? I'm curious in trying this out.
- pumanoir 7y ago
  When using vim keys for example, to jump to the beginning of the line I press ^ then press 'dw' keys delete first word in a line. So I do the caret with my voice and the other press with my hand which is faster than doing it by hand alone.
  - lunixbochs 7y ago
    Interesting, is that something you’d consider using the noise input for once I finish that? (As noise will be lower latency, you’ll have about 40 noises to map, and it’s “out of band” with speech)
    - pumanoir 7y ago
      Sure! Id love that. I only use one noise at the moment for mouse clicking (since I only know one sound). Could you point me to a video that shows an example of each of the 40 available sounds?
      - lunixbochs 7y ago
        They aren’t available for use yet. https://noise.talonvoice.com https://noise.talonvoice.com is the submission collection site for the training data. Each noise there has a description and audio example. When they’re available you’ll be able to bind them just like voice commands.

tucosan 7y ago

Actual links to the projects would be useful. ;)

femiagbabiaka 7y ago
1. https://talonvoice.com/ https://talonvoice.com/ 2. https://serenade.ai/beta https://serenade.ai/beta

tucosan 7y ago

Here's a video outlining Talon's [1] UX: https://youtu.be/ddFI63dgpaI https://youtu.be/ddFI63dgpaI [1]: https://talonvoice.com/ https://talonvoice.com/

lunixbochs 7y ago
That video is fairly old. I’ve polished my workflow a fair bit since then. My advent of code videos described here [1] are probably more representative, and the later videos in the playlist even show the keys that are pressed with each command. [1] https://news.ycombinator.com/item?id=22405178 https://news.ycombinator.com/item?id=22405178 Fun tidbit: Most of the commands I use in vim are just the Talon alphabet.

iudqnolq 7y ago

I have a non-technical friend who says Dragon is too unreliable to be usable. My intuition was that the problem is probably somewhere else and that if Dragon doesn't work nothing will. I'm under the assumption that competing voice recognition software competes on price. Is that assumption wrong? (For example, I suspect the microphone their school supplies them with may be no good)

daanzu 7y ago
For what it's worth, my voice is quite abnormal, so most untrained speech recognition is terrible for me, and even performing the normal "training" for Dragon still resulted in very poor accuracy. However, apparently their training is quite limited, because once I developed my Kaldi Active Grammar [1], and did my own direct training, the results were fantastic in comparison, with orders of magnitude better accuracy. Also, Dragon has many commands that cannot be disabled, and if your accuracy is already low, more commands available means more possible things to get wrong. Something like Dragonfly with KaldiAG could allow you to reduce the command set, and improve the practical accuracy. So, more personalized training/software may work better. [1] https://github.com/daanzu/kaldi-active-grammar https://github.com/daanzu/kaldi-active-grammar
- iudqnolq 7y ago
  Very interesting. I'll take a look and see if I can easily package that in a way someone non-technical could use
  - daanzu 7y ago
    If they are using Windows, packaging is quite easy: see the winpython distribution of KaldiAG. The more difficult part is writing the commands: it is not hard programming, but it is technical. But if they can describe what they would want, someone else may be able to write it easily. The personalized training is still pretty new and raw, and it needs a lot of setup to do the training itself. Without knowing more about what your friend found problematic, it is hard to say what could help the most.

daanzu 7y ago

This comment fails to mention Dragonfly with my Kaldi Active Grammar backend [1], which is cross platform (Windows/Linux now and Mac functional and to be released soon), completely free with no private beta features (although I do accept donations), and 100% open source (unlike Talon). The speech recognition is local, with extremely low latency. See the video demonstration [2] on the project page. I think the underlying Kaldi engine delivers unmatched accuracy as a free non-commercial engine. I created Kaldi Active Grammar because I didn't trust relying on closed source software for something so crucial to my productivity, where a decision by an outside party determines whether I can function. As a bonus, open source means I can make it work better to fit my needs than closed source ever could. Furthermore, the original article mentions Caster (which is built on Dragonfly), but doesn't mention that KaldiAG works with it, and that work is underway to expand Caster's platform support. [1] https://github.com/daanzu/kaldi-active-grammar https://github.com/daanzu/kaldi-active-grammar [2] https://youtu.be/Qk1mGbIJx3s https://youtu.be/Qk1mGbIJx3s

lunixbochs 7y ago
I think to state "unmatched accuracy" in good faith we should actually come up with a common benchmark and measure against it. I believe there aren't really any clean benchmarks for command accuracy floating around (it would be ideal if we used a strict grammar to properly measure the command decoders), and a wav2letter model holds the state of the art for librispeech WER% as of 2019. I found your measurement here [1] which is against an unknown wav2letter acoustic+language model pair, as the web demo is at any given point in time running an arbitrary model based on having users test in-progress models, and it has never been running the model I am currently shipping with Talon. (As a small example, the unfinished wav2letter experiment I am training right now has a 3.17% WER on speech commands, and 6.86% WER on librispeech clean, both numbers without using a language model) [1] https://github.com/daanzu/kaldi-active-grammar/blob/master/docs/models.md https://github.com/daanzu/kaldi-active-grammar/blob/master/d...
- daanzu 7y ago
  I am all for devising a good, fair apples to apples comparison. If you have any suggestions, let me know. In lieu of that, I use what I have available. While accuracy numbers from papers are informative and interesting, I don't think they directly apply to our usage particularly well. I would prefer to use numbers from actual usage.

_-___________-_ 7y ago

On the homepage for Talon it lists macOS under dependencies. I've actually come across the homepage before and didn't look into it further because I thought it was Mac-only.

_-___________-_ 7y ago
Hm, it does seem to be Mac-only? You mention "Windows/Linux/Mac support" in your comment; is Linux support released yet?
- lunixbochs 7y ago
  Windows/Linux are in earlier beta, see the patreon posts for more info on status. There are about 20 full time users for each OS beta. I really should update the site, thanks for reminding me it's confusing.

GordonS 7y ago

Oh, it looks like Talon is finally available for Windows! I've been waiting to try Talon for ages - glad I can give it a try now.