4 ms·

I'm also very excited. I've had some ESP32 microphones before, but they were not really able to understand the wake word, sometimes even when it was quiet and y

by pimeys 2y ago

I'm also very excited. I've had some ESP32 microphones before, but they were not really able to understand the wake word, sometimes even when it was quiet and you were sitting next to the mic. This one looks like it can recognize your voice very well, even when music is playing. Because... when it works, it's amazing. You get that Star Trek wake word (KHUM-PUTER!), you can connect your favorite LLM to it (ChatGPT, Claude Sonnet, Ollama), you can control your home automation with it and it's as private as you want. I ordered two of these, if they are great, I will order two more. I've been waiting for this product for years, it's hopefully finally here.

nine_k 2y ago
As a side note, it always slightly puzzles me when I see "voice interface" and "private" used together. Maybe it takes living alone to issue voice commands and feel some privacy. (Yes, I do understand that "privacy" here is mostly about not sending it for processing to third parties.)
- staunton 2y ago
  > Yes, I do understand that "privacy" here is mostly about not sending it for processing to third parties. Then why does it puzzle you?
  - entropicdrifter 2y ago
    Because you wouldn't ask it deeply private questions in front of your mom, for instance
    - inquirerGeneral 2y ago
      [dead]
    - xandrius 2y ago
      There are levels of privacy. Because I'm not going to ask deeply private questions, it doesn't mean that I want everyone to be snooping into what I'm planning to eat tonight.
      - entropicdrifter 2y ago
        I personally agree with your stance, but was simply interpreting what nine_k expressed so everyone else could understand their perspective.
    - jorvi 2y ago
      You wouldn’t ask your partner deeply private questions in front of your mom either. Not sure how you think it’s a dig against voice assistant privacy.
      - entropicdrifter 2y ago
        I'm just translating why that person (nine_k) feels that any voice interface is non-private. It's not how I feel
    - mystified5016 2y ago
      In this scenario, you have intentionally and willfully done something private in public.
- iteria 2y ago
  I don't like these interaces because unless they are button activated or something, they must be always listening and sending sound from where you are to a 3rd party server. No thanks. Of course this could be happening with my phone, but at least it have to be a malicious action to record me 24/7
  - pimeys 2y ago
    How these ESP32-systems work is that you send a wake word to the device itself. It can detect the word without an internet connection, the device itself understands it and wakes up. After the device is woken up, it sends your speech to home assistant, which either - handles it locally, if you have fast enough computer - sends it to home assistant cloud, if you set it up - sends it to chatgpt, claude sonnet etc. if you set it up I'm planning on building a proxmox rack server next year, so I'm probably going to just handle all the discussions locally. The home assistant cloud is quite private too, at least that's what they say (and they're in EU, so I think there might be truth in what they say)...
  - deleted 2y ago
    [deleted]
  - horsawlarway 2y ago
    I mean... That's not true, though. The main pitch of a tool like this is that I can absolutely verify it's not true. I'm currently running a slightly different take of this (Esp 32 based devices, with whisper through Willow inference server, with Willow autocorrect, tied into home assistant). For context, it works completely offline. My modem can literally be unplugged and I can control my smart devices just fine, with my voice. Entirely on my local network, with a couple of cheap devices and a ten year old gaming PC as the server. My data
  - result2vino 2y ago
    [dead]
  - gregmac 2y ago
    FWIW that's not even how Alexa or Google Assistant work. Both listen locally for the wake word with onboard processing, and only when they recognize it do they send the audio stream to the server to fully interpret. You can test this in a couple ways: they'll respond to their wake word when the internet is down (but have an error response). You can also look at the outbound data and see they're not sending continuous traffic. Not to say with the proprietary products that they couldn't sneakily change this on the fly and record everything, maybe even turning it on for a specific device or account.
- pimeys 2y ago
  Private meaning that a big American corporation is not listening and using my voice to either track me or teach their own AI service with it.
- sangnoir 2y ago
  > Maybe it takes living alone to issue voice commands and feel some privacy Perhaps your definition of "private" is more stringent than most people's. Collective privacy exists, for example "The family would appreciate some privacy as they grieve". It is correct to term something "private" when it is shared with your entire household, but no one else.
ijidak 2y ago
I'm trying to understand. Is there an SDK I can use to enhance this? Or is this a package product? I'm really hoping it's the former. But I don't see any information about how to develop with this.
- pimeys 2y ago
  Yep, ESPHome SDK. It's all open source and well-documented: https://esphome.io/ https://esphome.io/ Some notable blog posts, docs and a video on the wake words and voice assistant usage: https://community.home-assistant.io/t/on-device-wake-word-on-esp32-s3-is-here-voice-chapter-6/693607 https://community.home-assistant.io/t/on-device-wake-word-on... https://esphome.io/components/voice_assistant.html https://esphome.io/components/voice_assistant.html https://www.home-assistant.io/voice_control/create_wake_word/ https://www.home-assistant.io/voice_control/create_wake_word... https://www.youtube.com/watch?v=oSKBWtBJyDE https://www.youtube.com/watch?v=oSKBWtBJyDE

by pimeys 2y ago

nine_k 2y ago

As a side note, it always slightly puzzles me when I see "voice interface" and "private" used together. Maybe it takes living alone to issue voice commands and feel some privacy. (Yes, I do understand that "privacy" here is mostly about not sending it for processing to third parties.)

staunton 2y ago
> Yes, I do understand that "privacy" here is mostly about not sending it for processing to third parties. Then why does it puzzle you?
- entropicdrifter 2y ago
  Because you wouldn't ask it deeply private questions in front of your mom, for instance
  - inquirerGeneral 2y ago
    [dead]
  - xandrius 2y ago
    There are levels of privacy. Because I'm not going to ask deeply private questions, it doesn't mean that I want everyone to be snooping into what I'm planning to eat tonight.
    - entropicdrifter 2y ago
      I personally agree with your stance, but was simply interpreting what nine_k expressed so everyone else could understand their perspective.
  - jorvi 2y ago
    You wouldn’t ask your partner deeply private questions in front of your mom either. Not sure how you think it’s a dig against voice assistant privacy.
    - entropicdrifter 2y ago
      I'm just translating why that person (nine_k) feels that any voice interface is non-private. It's not how I feel
  - mystified5016 2y ago
    In this scenario, you have intentionally and willfully done something private in public.
iteria 2y ago
I don't like these interaces because unless they are button activated or something, they must be always listening and sending sound from where you are to a 3rd party server. No thanks. Of course this could be happening with my phone, but at least it have to be a malicious action to record me 24/7
- pimeys 2y ago
  How these ESP32-systems work is that you send a wake word to the device itself. It can detect the word without an internet connection, the device itself understands it and wakes up. After the device is woken up, it sends your speech to home assistant, which either - handles it locally, if you have fast enough computer - sends it to home assistant cloud, if you set it up - sends it to chatgpt, claude sonnet etc. if you set it up I'm planning on building a proxmox rack server next year, so I'm probably going to just handle all the discussions locally. The home assistant cloud is quite private too, at least that's what they say (and they're in EU, so I think there might be truth in what they say)...
- deleted 2y ago
  [deleted]
- horsawlarway 2y ago
  I mean... That's not true, though. The main pitch of a tool like this is that I can absolutely verify it's not true. I'm currently running a slightly different take of this (Esp 32 based devices, with whisper through Willow inference server, with Willow autocorrect, tied into home assistant). For context, it works completely offline. My modem can literally be unplugged and I can control my smart devices just fine, with my voice. Entirely on my local network, with a couple of cheap devices and a ten year old gaming PC as the server. My data
- result2vino 2y ago
  [dead]
- gregmac 2y ago
  FWIW that's not even how Alexa or Google Assistant work. Both listen locally for the wake word with onboard processing, and only when they recognize it do they send the audio stream to the server to fully interpret. You can test this in a couple ways: they'll respond to their wake word when the internet is down (but have an error response). You can also look at the outbound data and see they're not sending continuous traffic. Not to say with the proprietary products that they couldn't sneakily change this on the fly and record everything, maybe even turning it on for a specific device or account.
pimeys 2y ago
Private meaning that a big American corporation is not listening and using my voice to either track me or teach their own AI service with it.
sangnoir 2y ago
> Maybe it takes living alone to issue voice commands and feel some privacy Perhaps your definition of "private" is more stringent than most people's. Collective privacy exists, for example "The family would appreciate some privacy as they grieve". It is correct to term something "private" when it is shared with your entire household, but no one else.

ijidak 2y ago

I'm trying to understand. Is there an SDK I can use to enhance this? Or is this a package product? I'm really hoping it's the former. But I don't see any information about how to develop with this.

pimeys 2y ago
Yep, ESPHome SDK. It's all open source and well-documented: https://esphome.io/ https://esphome.io/ Some notable blog posts, docs and a video on the wake words and voice assistant usage: https://community.home-assistant.io/t/on-device-wake-word-on-esp32-s3-is-here-voice-chapter-6/693607 https://community.home-assistant.io/t/on-device-wake-word-on... https://esphome.io/components/voice_assistant.html https://esphome.io/components/voice_assistant.html https://www.home-assistant.io/voice_control/create_wake_word/ https://www.home-assistant.io/voice_control/create_wake_word... https://www.youtube.com/watch?v=oSKBWtBJyDE https://www.youtube.com/watch?v=oSKBWtBJyDE