4 ms·

Does anyone know just how much of the total functionality of Mycroft is actually running on the Raspberry Pi? I asked this question four years ago on Reddit (I

Does anyone know just how much of the total functionality of Mycroft is actually running on the Raspberry Pi? I asked this question four years ago on Reddit (I'll paste the response below) and now I wonder if things have changed, particularly with regard to speech to text. There are several ‘layers’ to a voice assistant; Wake Word - that detects when you are speaking to the device. This is local to the device and we use PocketSphinx. Speech to text - that detects what you say to determine Intents - we currently use a cloud service for this Intent matching - this is done locally using our own open source software - Adapt and Padatious Skills - Intents then match to Skills. Some Skills require internet connectivity. Text to Speech - We use our own software called Mimic for this, it’s local to the device.

krisgesling 4y ago
If you download Mycroft today, currently by default everything except the STT is running on the Pi. There are options to do this locally: https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customizations/stt-engine https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz... however they have their limitations. Before the Mark II ships in September, local STT will be available by default!
- Quequau 4y ago
  thanks for the info!
- diggernet 4y ago
  Thanks for jumping in here. > In order to provide an additional layer of privacy for our users, we proxy all STT requests through Mycroft's servers. This prevents Google's service from profiling Mycroft users or connecting voice recordings to their identities. Only the voice recording is sent to Google, no other identifying information is included in the request. Therefore Google's STT service does not know if an individual person is making thousands of requests, or if thousands of people are making a small number of requests each. Well, unless Google does voiceprint analysis. But Google wouldn't do that, would they? /s Beyond that, if I'm reading right, local STT will still require a separate STT server. It won't run on the Mark II itself, right?
capableweb 4y ago
The blogpost linked in this submission says the following: > Mimic 3: Mycroft’s newer, better, privacy-focused neural text-to-speech (TTS) engine. In human terms, that means it can run completely offline and sounds great. To top it all off, it’s open source. If "skills" are what I think they are (something like external commands, for example "Play X on Spotify"), then my understanding would be that everything but those runs offline and local-only. But if things like `speech to text` requires internet connection and sends the data to some cloud service, then the entire value proposition of this product falls apart. I hope that's really not the case, as that would be outright lying and false advertisement.
- krisgesling 4y ago
  On device STT will be available before the Mark II ships in September! There are some options for that available if you're running Mycroft already: https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customizations/stt-engine https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz... But these are not what will be shipped on the Mark II...
- bluGill 4y ago
  The raspberry pi 3 that is used in older products doesn't have enough power to be all offline. Maybe you could setup a server at home (but they won't help you!), but you cannot do it on the hardware they have. The next gen mycroft 2 (should ship this fall - first announced many years ago) will have a pi 4 which might have enough power to run offline, this isn't clear yet.
  - capableweb 4y ago
    > might have enough power to run offline, this isn't clear yet It's very unclear and misleading to put "it can run completely offline" if you're not 100% sure it can actually run "completely offline", hardware be damned.
notahacker 4y ago
If they're marketing it as "fully offline", they ought to be doing the speech to text bit locally now. Worked on part of a platform which could use Rasa for this a couple of years ago, but running on something a bit more powerful than a Raspberry Pi!
- krisgesling 4y ago
  On device STT will be available before the Mark II ships in September!
diggernet 4y ago
According to their privacy policy (https://mycroft.ai/embed-privacy-policy/ https://mycroft.ai/embed-privacy-policy/): "*When you use our Services including the Mycroft Voice Assistant, your voice and audio commands are transmitted to our Servers for processing.*" So it appears that STT is still cloud-based, which is a pity. That's the only thing keeping me from ordering one today.
- krisgesling 4y ago
  On device STT will be available before the Mark II ships in September! As will a new privacy policy that better reflects what we actually do.
- suyash 4y ago
  Interesting so they run the services on cloud and advertise their company as on device, what a bunch of crooks.
  - krisgesling 4y ago
    On device STT will be available before the Mark II ships in September! Note: I'm from Mycroft
    - whitten 4y ago
      Kris Gesling I’ve seen you say that Mark II is going to be able to be run non-internet. Will I be able to be run on my local network, even if I have to dedicate a machine to just speech to text (STT) ?

krisgesling 4y ago

If you download Mycroft today, currently by default everything except the STT is running on the Pi. There are options to do this locally: https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customizations/stt-engine https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz... however they have their limitations. Before the Mark II ships in September, local STT will be available by default!

Quequau 4y ago
thanks for the info!
diggernet 4y ago
Thanks for jumping in here. > In order to provide an additional layer of privacy for our users, we proxy all STT requests through Mycroft's servers. This prevents Google's service from profiling Mycroft users or connecting voice recordings to their identities. Only the voice recording is sent to Google, no other identifying information is included in the request. Therefore Google's STT service does not know if an individual person is making thousands of requests, or if thousands of people are making a small number of requests each. Well, unless Google does voiceprint analysis. But Google wouldn't do that, would they? /s Beyond that, if I'm reading right, local STT will still require a separate STT server. It won't run on the Mark II itself, right?

capableweb 4y ago

The blogpost linked in this submission says the following: > Mimic 3: Mycroft’s newer, better, privacy-focused neural text-to-speech (TTS) engine. In human terms, that means it can run completely offline and sounds great. To top it all off, it’s open source. If "skills" are what I think they are (something like external commands, for example "Play X on Spotify"), then my understanding would be that everything but those runs offline and local-only. But if things like `speech to text` requires internet connection and sends the data to some cloud service, then the entire value proposition of this product falls apart. I hope that's really not the case, as that would be outright lying and false advertisement.

krisgesling 4y ago
On device STT will be available before the Mark II ships in September! There are some options for that available if you're running Mycroft already: https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customizations/stt-engine https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz... But these are not what will be shipped on the Mark II...
bluGill 4y ago
The raspberry pi 3 that is used in older products doesn't have enough power to be all offline. Maybe you could setup a server at home (but they won't help you!), but you cannot do it on the hardware they have. The next gen mycroft 2 (should ship this fall - first announced many years ago) will have a pi 4 which might have enough power to run offline, this isn't clear yet.
- capableweb 4y ago
  > might have enough power to run offline, this isn't clear yet It's very unclear and misleading to put "it can run completely offline" if you're not 100% sure it can actually run "completely offline", hardware be damned.

notahacker 4y ago

If they're marketing it as "fully offline", they ought to be doing the speech to text bit locally now. Worked on part of a platform which could use Rasa for this a couple of years ago, but running on something a bit more powerful than a Raspberry Pi!

krisgesling 4y ago
On device STT will be available before the Mark II ships in September!

diggernet 4y ago

According to their privacy policy (https://mycroft.ai/embed-privacy-policy/ https://mycroft.ai/embed-privacy-policy/): "*When you use our Services including the Mycroft Voice Assistant, your voice and audio commands are transmitted to our Servers for processing.*" So it appears that STT is still cloud-based, which is a pity. That's the only thing keeping me from ordering one today.

krisgesling 4y ago
On device STT will be available before the Mark II ships in September! As will a new privacy policy that better reflects what we actually do.
suyash 4y ago
Interesting so they run the services on cloud and advertise their company as on device, what a bunch of crooks.
- krisgesling 4y ago
  On device STT will be available before the Mark II ships in September! Note: I'm from Mycroft
  - whitten 4y ago
    Kris Gesling I’ve seen you say that Mark II is going to be able to be run non-internet. Will I be able to be run on my local network, even if I have to dedicate a machine to just speech to text (STT) ?