4 ms·

I do not understand how to use Deepspeech even in the most simple use case. 1. I want to teach it ten words. How do I do this? 2. I want to speak into my micr

by bmn__ 7y ago

I do not understand how to use Deepspeech even in the most simple use case. 1. I want to teach it ten words. How do I do this? 2. I want to speak into my microphone (available as a Pulseaudio device) and recognise the words and output the words as a text stream on stdout. How do I do this? This is the documentation: https://deepspeech.readthedocs.io/en/v0.6.0/Python-Examples.html https://deepspeech.readthedocs.io/en/v0.6.0/Python-Examples.... https://deepspeech.readthedocs.io/en/v0.6.0/Python-API.html https://deepspeech.readthedocs.io/en/v0.6.0/Python-API.html It does not answer the questions I have. https://deepspeech.readthedocs.io/en/v0.6.0/DeepSpeech.html https://deepspeech.readthedocs.io/en/v0.6.0/DeepSpeech.html The introduction page is full of incomprehensible jargon.

reubenmorais 7y ago
> 1. I want to teach it ten words. How do I do this? I'm assuming you mean "my target vocabulary is only ten words". In that case, you have a couple of options: 1. Just use it as is and see how it does. 2. Create a language model containing just those 10 words so that the engine is constrained to that vocabulary, which should improve accuracy. 3. Train a (smaller) model from scratch, which requires (audio, transcript) pairs. This is basically research work, there's no recipe you can just follow. The advantage is that if you can make it work, it can be a much smaller model than a model trained for general English dictation. > 2. I want to speak into my microphone (available as a Pulseaudio device) and recognise the words and output the words as a text stream on stdout. How do I do this? We have a few examples with microphone input, but they're all based on voice activity detection: https://github.com/mozilla/DeepSpeech/tree/master/examples https://github.com/mozilla/DeepSpeech/tree/master/examples I should write an example that is just continuous streaming and output without any voice activity detection. I'd be happy to discuss further on Discourse: https://discourse.mozilla.org/c/deep-speech https://discourse.mozilla.org/c/deep-speech
- na85 7y ago
  >Just use it as is and see how it does. How? The docs seem to be suffering from the same problem as the git docs: they assume the reader is already a domain expert, which would seem to run counter to the stated goal of being simple (and ubiquitous). I tried to figure out how to "just use" deepspeech and failed.
  - reubenmorais 7y ago
    There's a basic example here transcribing from an audio file: https://github.com/mozilla/DeepSpeech/tree/v0.6.0/#project-deepspeech https://github.com/mozilla/DeepSpeech/tree/v0.6.0/#project-d... There are also examples transcribing from the microphone as I mentioned above: https://github.com/mozilla/DeepSpeech/tree/master/examples/ https://github.com/mozilla/DeepSpeech/tree/master/examples/ I also have a small GUI example here [0], click once to start recording, once again to stop and show transcript. It receives the same command line arguments as the ones shown in the README link above (namely --model, --lm and --trie). The README starts with a very basic installation and usage guide, I don't see how that's assuming any expertise. [0] https://github.com/reuben/ds-qttest https://github.com/reuben/ds-qttest
    - mncharity 7y ago
      > There are also examples Fyi, the other 4 example links there are 404. Also in the root README. The linked README's are missing, or diversely suffixed and mislinked.
      - sgt101 7y ago
        You are right, and I expect Reuben and team will fix this and are grateful for the heads up. I was able to find the right links though, for example the WAV transcriber is at : https://github.com/mozilla/DeepSpeech/blob/v0.6.0/examples/vad_transcriber/wavTranscription.md https://github.com/mozilla/DeepSpeech/blob/v0.6.0/examples/v...
        mncharity 7y ago
        (Nod. I may have edited that out from under you - sorry. And thanks.)
      - reubenmorais 7y ago
        Ugh, sorry. I fixed it on master. We'll probably cut a v0.6.1 soon to publish small fixes like these.
IshKebab 7y ago
I think you're being a little harsh. 1. Train it using the training dataset but change all word labels that aren't in your 10 word list to "OTHER_WORD" or whatever. There's probably not much point doing this. Alternatively I guess you can restrict the beam search to only look for your 10 words. Yes this is really complicated - it's a speech recognition engine - they are really complicated! 2. The API docs look pretty clear to me: > createStream()[source] > Create a new streaming inference state. The streaming state returned by this function can then be passed to feedAudioContent() and finishStream(). > feedAudioContent(args, kwargs)[source] > Feed audio samples to an ongoing streaming inference. > intermediateDecode(args, kwargs)[source] > Compute the intermediate decoding of an ongoing streaming inference. This is an expensive process as the decoder implementation isn’t currently capable of streaming, so it always starts from the beginning of the audio. Huh well that's a flaw the neglected to mention! I guess you can't really do what you want yet.
- reubenmorais 7y ago
  > Huh well that's a flaw the neglected to mention! I guess you can't really do what you want yet. The documentation is outdated. That is no longer true, intermediateDecode is cheap. Thanks for noticing, I'll fix it.
sillysaurusx 7y ago
So, funny story: I wanted #2 and did it myself. I was similarly frustrated with the lack of documentation. It's been a little while since I got it running, but I basically got a siri clone working. If you want to test it out, I can try to answer questions / whatever problems pop up. The code is here: https://github.com/shawwn/DeepSpeech/commit/01f5cf8d39c356aecf84b9daaa5b38747d64c209 https://github.com/shawwn/DeepSpeech/commit/01f5cf8d39c356ae... As far as I know, you can simply run speech_to_text.sh. It will connect to your microphone and start dumping out transcribed audio to stdout. It wasn't super easy, but once you spend a little time with the code you can sort of figure out ways to get it to do what you want. EDIT: By the way, people will try to convince you that you're nuts and that the documentation is crystal clear and so on. Know this: It's not just you. I had the exact same experience. It's a recurring theme in AI programming. The only thing to do is to either find someone else who feels similarly, or roll up your sleeves and dive into the code.
- logicallee 7y ago
  >By the way, people will try to convince you that you're nuts and that the documentation is crystal clear and so on No it really is simple: the original implementation had a base of prefabulated amulition grammar, this was surmounted by a malleable logarithmic function in such a way that the two main spurving vocabularies were in a direct line with the panametric grammar fields. The latter simply consists of marzlevanes fitted to the ambifacient morpheme waneshaft to eliminate side utterances. This main winding is of the normal lotus-o-deltoid type, just placed in panendermic semi-boloid stators. Basically every second conductor is connected by a nonreversible tremmie pipe to the differential grammeters. All these words are in: https://en.wikipedia.org/wiki/Turboencabulator https://en.wikipedia.org/wiki/Turboencabulator

by bmn__ 7y ago

reubenmorais 7y ago

> 1. I want to teach it ten words. How do I do this? I'm assuming you mean "my target vocabulary is only ten words". In that case, you have a couple of options: 1. Just use it as is and see how it does. 2. Create a language model containing just those 10 words so that the engine is constrained to that vocabulary, which should improve accuracy. 3. Train a (smaller) model from scratch, which requires (audio, transcript) pairs. This is basically research work, there's no recipe you can just follow. The advantage is that if you can make it work, it can be a much smaller model than a model trained for general English dictation. > 2. I want to speak into my microphone (available as a Pulseaudio device) and recognise the words and output the words as a text stream on stdout. How do I do this? We have a few examples with microphone input, but they're all based on voice activity detection: https://github.com/mozilla/DeepSpeech/tree/master/examples https://github.com/mozilla/DeepSpeech/tree/master/examples I should write an example that is just continuous streaming and output without any voice activity detection. I'd be happy to discuss further on Discourse: https://discourse.mozilla.org/c/deep-speech https://discourse.mozilla.org/c/deep-speech

na85 7y ago
>Just use it as is and see how it does. How? The docs seem to be suffering from the same problem as the git docs: they assume the reader is already a domain expert, which would seem to run counter to the stated goal of being simple (and ubiquitous). I tried to figure out how to "just use" deepspeech and failed.
- reubenmorais 7y ago
  There's a basic example here transcribing from an audio file: https://github.com/mozilla/DeepSpeech/tree/v0.6.0/#project-deepspeech https://github.com/mozilla/DeepSpeech/tree/v0.6.0/#project-d... There are also examples transcribing from the microphone as I mentioned above: https://github.com/mozilla/DeepSpeech/tree/master/examples/ https://github.com/mozilla/DeepSpeech/tree/master/examples/ I also have a small GUI example here [0], click once to start recording, once again to stop and show transcript. It receives the same command line arguments as the ones shown in the README link above (namely --model, --lm and --trie). The README starts with a very basic installation and usage guide, I don't see how that's assuming any expertise. [0] https://github.com/reuben/ds-qttest https://github.com/reuben/ds-qttest
  - mncharity 7y ago
    > There are also examples Fyi, the other 4 example links there are 404. Also in the root README. The linked README's are missing, or diversely suffixed and mislinked.
    - sgt101 7y ago
      You are right, and I expect Reuben and team will fix this and are grateful for the heads up. I was able to find the right links though, for example the WAV transcriber is at : https://github.com/mozilla/DeepSpeech/blob/v0.6.0/examples/vad_transcriber/wavTranscription.md https://github.com/mozilla/DeepSpeech/blob/v0.6.0/examples/v...
      - mncharity 7y ago
        (Nod. I may have edited that out from under you - sorry. And thanks.)
    - reubenmorais 7y ago
      Ugh, sorry. I fixed it on master. We'll probably cut a v0.6.1 soon to publish small fixes like these.

IshKebab 7y ago

I think you're being a little harsh. 1. Train it using the training dataset but change all word labels that aren't in your 10 word list to "OTHER_WORD" or whatever. There's probably not much point doing this. Alternatively I guess you can restrict the beam search to only look for your 10 words. Yes this is really complicated - it's a speech recognition engine - they are really complicated! 2. The API docs look pretty clear to me: > createStream()[source] > Create a new streaming inference state. The streaming state returned by this function can then be passed to feedAudioContent() and finishStream(). > feedAudioContent(args, kwargs)[source] > Feed audio samples to an ongoing streaming inference. > intermediateDecode(args, kwargs)[source] > Compute the intermediate decoding of an ongoing streaming inference. This is an expensive process as the decoder implementation isn’t currently capable of streaming, so it always starts from the beginning of the audio. Huh well that's a flaw the neglected to mention! I guess you can't really do what you want yet.

reubenmorais 7y ago
> Huh well that's a flaw the neglected to mention! I guess you can't really do what you want yet. The documentation is outdated. That is no longer true, intermediateDecode is cheap. Thanks for noticing, I'll fix it.

sillysaurusx 7y ago

So, funny story: I wanted #2 and did it myself. I was similarly frustrated with the lack of documentation. It's been a little while since I got it running, but I basically got a siri clone working. If you want to test it out, I can try to answer questions / whatever problems pop up. The code is here: https://github.com/shawwn/DeepSpeech/commit/01f5cf8d39c356aecf84b9daaa5b38747d64c209 https://github.com/shawwn/DeepSpeech/commit/01f5cf8d39c356ae... As far as I know, you can simply run speech_to_text.sh. It will connect to your microphone and start dumping out transcribed audio to stdout. It wasn't super easy, but once you spend a little time with the code you can sort of figure out ways to get it to do what you want. EDIT: By the way, people will try to convince you that you're nuts and that the documentation is crystal clear and so on. Know this: It's not just you. I had the exact same experience. It's a recurring theme in AI programming. The only thing to do is to either find someone else who feels similarly, or roll up your sleeves and dive into the code.

logicallee 7y ago
>By the way, people will try to convince you that you're nuts and that the documentation is crystal clear and so on No it really is simple: the original implementation had a base of prefabulated amulition grammar, this was surmounted by a malleable logarithmic function in such a way that the two main spurving vocabularies were in a direct line with the panametric grammar fields. The latter simply consists of marzlevanes fitted to the ambifacient morpheme waneshaft to eliminate side utterances. This main winding is of the normal lotus-o-deltoid type, just placed in panendermic semi-boloid stators. Basically every second conductor is connected by a nonreversible tremmie pipe to the differential grammeters. All these words are in: https://en.wikipedia.org/wiki/Turboencabulator https://en.wikipedia.org/wiki/Turboencabulator