7 ms·

This really makes me want to build a Amazon Echo/Google Nest/etc replacement that's open hardware, open source and most importantly recognises voice completely

by dom96 4y ago

This really makes me want to build a Amazon Echo/Google Nest/etc replacement that's open hardware, open source and most importantly recognises voice completely offline. I find that I don't use these smart devices for much more than setting timers anyway so this seems like an easy project. I just wonder what system requirements Whisper has and whether there are open source voice recognition models that are specifically built for embedded devices.

suyash 4y ago
This is only one side of the coin, you still need really good models for Speech Synthesis and then be able to have it all working in almost real time, ideally locally on device.
- ricopags 4y ago
  As far as TTS goes, Mycroft.ai[0] has released a decent offline one. [0]https://mycroft.ai/ https://mycroft.ai/
  - smcameron 4y ago
    pico2wave with "-l=en-GB" option to get the British lady voice is pretty decent (way better than the other voices it does for some reason).
  - arbol 4y ago
    I'm pretty sure mycroft sends your speech snippets to Google for processing so it's not exactly offline. https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customizations/stt-engine#default-engine https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz... I'm currently trying to setup a deepspeech server on my raspberry pi to see if it works ok for commanding spotify. Edit: just realised you said `TTS` not `STT`
MacsHeadroom 4y ago
I really want all this too. The smallest model is ~80mb and the largest is 3gb. Not sure about system requirements yet; but models that small suggest this may be doable locally on a single board computer. Edit: According to this comment[0] the base model runs in real time on an M1 CPU. The tiny model apparently decodes an audio file twice as fast. These are promising results. [0] https://news.ycombinator.com/item?id=32927360#32929739 https://news.ycombinator.com/item?id=32927360#32929739
- dom96 4y ago
  I'd be interested to see how well it performs on something like an RPi. M1 is pretty beefy.
- lunixbochs 4y ago
  For an offline (non-streaming) model, 1x realtime is actually kind of bad, because you need to wait for the audio to be available before you can start processing it. So if you wait 10 seconds for someone to finish speaking, you won't have the result until 10 seconds after that. You could use really small chunk sizes and process them in a streaming fashion, but that would impact accuracy, as you're significantly limiting available context.
- olao99 4y ago
  To be more precise the original comment said "M1 Max" which in itself is significantly beefier a bare "M1"
solarkraft 4y ago
Are you thinking about reimplementing Mycroft? The Mycroft has done a lot of cool and important work in the field to ship an actual personal assistant product (stuff like wake word detection).
- dom96 4y ago
  hah, of course someone had the idea already and executed on it. But yeah, basically that but without the screen (probably would go a long way to decrease the cost, $299 is pretty steep for such a device)
  - sheepybloke 4y ago
    One thing they don't touch much on is the STT, as they use models from third parties. You could definitely do something that utilizes this model and then feeds the tokens to some of their parsing code. I've been working on something similar to this, but burned out around adding the STT portion [0]. [0]: https://github.com/Sheepybloke2-0/trashbot https://github.com/Sheepybloke2-0/trashbot - It was called trashbot because the final implementation was going to look like oscar the grouch in a trashcan displaying the reminders.
  - MayeulC 4y ago
    Well, you can always install Mycroft on a Pi, or on your computer. Almond is also interesting as a voice assistant, though I think it doesn't perform speech recognition itself.

by dom96 4y ago

suyash 4y ago

This is only one side of the coin, you still need really good models for Speech Synthesis and then be able to have it all working in almost real time, ideally locally on device.

ricopags 4y ago
As far as TTS goes, Mycroft.ai[0] has released a decent offline one. [0]https://mycroft.ai/ https://mycroft.ai/
- smcameron 4y ago
  pico2wave with "-l=en-GB" option to get the British lady voice is pretty decent (way better than the other voices it does for some reason).
- arbol 4y ago
  I'm pretty sure mycroft sends your speech snippets to Google for processing so it's not exactly offline. https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customizations/stt-engine#default-engine https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz... I'm currently trying to setup a deepspeech server on my raspberry pi to see if it works ok for commanding spotify. Edit: just realised you said `TTS` not `STT`

MacsHeadroom 4y ago

I really want all this too. The smallest model is ~80mb and the largest is 3gb. Not sure about system requirements yet; but models that small suggest this may be doable locally on a single board computer. Edit: According to this comment[0] the base model runs in real time on an M1 CPU. The tiny model apparently decodes an audio file twice as fast. These are promising results. [0] https://news.ycombinator.com/item?id=32927360#32929739 https://news.ycombinator.com/item?id=32927360#32929739

dom96 4y ago
I'd be interested to see how well it performs on something like an RPi. M1 is pretty beefy.
lunixbochs 4y ago
For an offline (non-streaming) model, 1x realtime is actually kind of bad, because you need to wait for the audio to be available before you can start processing it. So if you wait 10 seconds for someone to finish speaking, you won't have the result until 10 seconds after that. You could use really small chunk sizes and process them in a streaming fashion, but that would impact accuracy, as you're significantly limiting available context.
olao99 4y ago
To be more precise the original comment said "M1 Max" which in itself is significantly beefier a bare "M1"

solarkraft 4y ago

Are you thinking about reimplementing Mycroft? The Mycroft has done a lot of cool and important work in the field to ship an actual personal assistant product (stuff like wake word detection).

dom96 4y ago
hah, of course someone had the idea already and executed on it. But yeah, basically that but without the screen (probably would go a long way to decrease the cost, $299 is pretty steep for such a device)
- sheepybloke 4y ago
  One thing they don't touch much on is the STT, as they use models from third parties. You could definitely do something that utilizes this model and then feeds the tokens to some of their parsing code. I've been working on something similar to this, but burned out around adding the STT portion [0]. [0]: https://github.com/Sheepybloke2-0/trashbot https://github.com/Sheepybloke2-0/trashbot - It was called trashbot because the final implementation was going to look like oscar the grouch in a trashcan displaying the reminders.
- MayeulC 4y ago
  Well, you can always install Mycroft on a Pi, or on your computer. Almond is also interesting as a voice assistant, though I think it doesn't perform speech recognition itself.