5 ms·

Hi, I'm the dev behind https://talonvoice.com https://talonvoice.com I've been working with Facebook's wav2letter project and the results (speed on CPU, comman

Hi, I'm the dev behind https://talonvoice.com https://talonvoice.com I've been working with Facebook's wav2letter project and the results (speed on CPU, command accuracy) are extremely good in my experience. They also hold the "state of the art" for librispeech (a common benchmark) on wer_are_we [1]. Granted, that's with a 2GB model that doesn't run very well on CPU, but I think most of the fully "state of the art" models are computationally expensive and expected to run on GPU. Wav2letter has other models that are very fast on CPU and still extremely accurate. You can run their "Streaming ConvNets" model on CPU to transcribe multiple live audio streams in parallel, see their wav2letter@anywhere post for more info [2] I am getting very good accuracy on the in-progress model I am training for command recognition (3.7% word error rate on librispeech clean, about 8% WER on librispeech other, 20% WER on common voice, 3% WER on "speech commands"). I plan to release it alongside my other models here [5] once I'm done working on it. There's a simple WER comparison between some of the command engines here [3] Between this and wer_are_we [1] it should give you a general idea of what to expect when talking about Word Error Rate (WER). (Note the wav2letter-talonweb entry in [3] is a rather old model I trained, known to have worse accuracy, it's not even the same NN architecture). ---- As far as constraining the vocabulary, you can try train a kenlm language model for kaldi, deepspeech, and wav2letter by grabbing KenLM and piping normalized (probably lowercase it and remove everything but ascii and quotes) text into lmplz: cat corpus.txt | kenlm/build/bin/lmplz -o 4 > model.arpa And you can turn it into a compressed binary model for wav2letter like this: kenlm/build/bin/build_binary -a 22 -q 8 -b 8 trie model.arpa model.bin There are other options, like using a "strict command grammar", but I don't have enough context as to how you want to program this to guide you there. I also have tooling I wrote around wav2letter, such as wav2train [4] which builds wav2letter training and runtime data files for you. I'm generally happy to talk more and answer any questions. ---- [1] https://github.com/syhw/wer_are_we https://github.com/syhw/wer_are_we [2] https://ai.facebook.com/blog/online-speech-recognition-with-wav2letteranywhere/ https://ai.facebook.com/blog/online-speech-recognition-with-... [3] https://github.com/daanzu/kaldi-active-grammar/blob/master/docs/models.md https://github.com/daanzu/kaldi-active-grammar/blob/master/d... [4] https://github.com/talonvoice/wav2train https://github.com/talonvoice/wav2train [5] https://talonvoice.com/research/ https://talonvoice.com/research/

rs23296008n1 7y ago
Looks interesting. I've got a few GPUs I could use if the CPU is too much of a bottleneck. We've identified a dictionary of the types of commands and words we use and have a recording of all our amazon and other commands. Training wave files are not an issue. Have you had any issues with recognising multiple languages? Thanks!
- lunixbochs 7y ago
  Are you talking about recognizing multiple languages at once (e.g. you don't know which language the user will speak and you want it to react appropriately to all of them)? You're going to have a harder time doing that with any system, but it is possible. You would likely need to run multiple models and pick between their output, or train a specialized separate model that can classify the language, then use it to pick the model. I haven't personally tested wav2letter with other languages yet. I know zamia-speech trained a german model, and some users have been talking about training for other languages. I've been helping someone who is training several other languages and they've reported great success as well. If you want to make a new model from scratch in any language, you'll probably want a couple hundred hours of transcribed speech for it, but it doesn't need to be your own speech. Common Voice is a good data source for that.
  - rs23296008n1 7y ago
    I'd expected some training required for each separately so we've got a decent collection of voice examples. Some of us mix languages within a sentence but that's a bad habit anyway so unsupported. I don't see figuring out the language as being a problem as my proof-of-concept already handles that well enough with around 90% accuracy. Once its selected a language then the appropriate model can be used. To make it fast we'd likely just keep the whole thing in RAM. Might need more however. The issue I see with talon is its currently mac only. That would however still help one of us who lives on wheels (got a 16" macbook IIRC and a mac mini as well). Different set of use-cases so things would be more relaxed. I see some hints about a linux version however. I've got windows / linux VMs on the server but no other macs. GPUs will be installed soon when I decom some old gaming rigs. Plenty to think about.

rs23296008n1 7y ago

Looks interesting. I've got a few GPUs I could use if the CPU is too much of a bottleneck. We've identified a dictionary of the types of commands and words we use and have a recording of all our amazon and other commands. Training wave files are not an issue. Have you had any issues with recognising multiple languages? Thanks!

lunixbochs 7y ago
Are you talking about recognizing multiple languages at once (e.g. you don't know which language the user will speak and you want it to react appropriately to all of them)? You're going to have a harder time doing that with any system, but it is possible. You would likely need to run multiple models and pick between their output, or train a specialized separate model that can classify the language, then use it to pick the model. I haven't personally tested wav2letter with other languages yet. I know zamia-speech trained a german model, and some users have been talking about training for other languages. I've been helping someone who is training several other languages and they've reported great success as well. If you want to make a new model from scratch in any language, you'll probably want a couple hundred hours of transcribed speech for it, but it doesn't need to be your own speech. Common Voice is a good data source for that.
- rs23296008n1 7y ago
  I'd expected some training required for each separately so we've got a decent collection of voice examples. Some of us mix languages within a sentence but that's a bad habit anyway so unsupported. I don't see figuring out the language as being a problem as my proof-of-concept already handles that well enough with around 90% accuracy. Once its selected a language then the appropriate model can be used. To make it fast we'd likely just keep the whole thing in RAM. Might need more however. The issue I see with talon is its currently mac only. That would however still help one of us who lives on wheels (got a 16" macbook IIRC and a mac mini as well). Different set of use-cases so things would be more relaxed. I see some hints about a linux version however. I've got windows / linux VMs on the server but no other macs. GPUs will be installed soon when I decom some old gaming rigs. Plenty to think about.