4 ms·

How do Android and iOS known when you say "OK Google" or "Hey Siri" if they are not listening all the time? Or, is it accepted that they are?

by _wldu 5y ago

angelzen 5y ago
They are listening all the time. Presumably 'just' for the wakeword. Better yet, Alexa listens to everything you do inside your house, and the Ring Network listens to everything you do outside your house. And both of them are connected to the power outlet, so the 'but battery depletion' excuse doesn't apply.
justusthane 5y ago
The listening for the initial keyword (e.g. “Hey Siri”) is done on hardware designed specifically for that purpose. That’s why you can’t change the keyword to whatever you want. Only when that keyword is heard does it actively listen to and process whatever follows. “Actively” listening all the time would simply consume too much CPU/bandwidth/battery.
- dylan604 5y ago
  >That’s why you can’t change the keyword to whatever you want. Which I feel is a huge security oversight. You should be able to choose from a list of activation keywords at least. I love being on video/voice chats with people that I know have a device and randomly say "Alexa, play Dancing Queen" or something equally obnoxious like "Alexa, turn off lights".
- angelzen 5y ago
  What exactly prevents the 'hardware designed specifically for that purpose' to include in its purpose the translation of your voice to text, with voice fingerprinting for good measure? Have you ran an electronic microscope atom layer slicing of the hardware to guarantee its complete set of functionalities?
  - midasuni 5y ago
    Power consumption
    - angelzen 5y ago
      The hardware runs an algorithm to detect the wakeword. What makes you confident that a different algorithm can't be run to translate voice to text? They will be in the same family of algorithms, just with different purposes. To make it more interesting, let's assume that 2021 algorithms can't do that. Will better low power algorithms be developed in the next 20 years? How is a democratic society supposed to work when would-be-snitches are ubiquitously deployed? To make it even more interesting, as more an more of our communications move to the internet (oooOOOooo, covid, cower in fear), what confidence you have that voice2text and voice fingerprinting does not already apply to all your phone/Zoom conversations? Do you run the phone/Zoom servers and monitor all potentially suspicious traffic? Have you heard of one Edward Snowden?
      - kempbellt 5y ago
        When you train your phone to understand "Hey Siri" it requires an internet connection for the training so that it can verify that you are actually saying, "Hey Siri" and not "Yo, phone spirit". After the initial training, it no longer requires (or uses) the internet to detect the keyphrase. You are asked to do a few iterations of training so that the phone has a model on your voiceprint for "Hey Siri". Which is why, when someone who does not have a similar voice to you says "Hey Siri" it does not trigger. But someone who does have a similar voice to you can sometimes trigger it. The model is super basic. The microphone is always passively passing input data to the "Hey Siri" detector (and a buffer), which acts as a sort of internal switch. If it detects the trigger phrase, it then sends the audio data to the cloud to be transcribed, and acted upon accordingly. It is not always sending voice data to the cloud. If it were, your battery life would be noticeably impacted - worse than being on a phone call.
        angelzen 5y ago
        Good explanation, thanks. The scenario is 'translate on device, steganograph the output on background traffic'. There is a huge push for 'on device ML', driven by noble intentions of hiding raw data from Skynet. Alas, the end of that road is devices that compress the raw sensorial input and report, possibly stealthily, the digest to Skynet.
- alpha_squared 5y ago
  Don't Android phones have passive listening? Pixel phones have always-on song detection (Shazam-like) and usually display song info on the lockscreen within ~10 seconds of the song being played nearby.
  - vitno 5y ago
    That works somewhat similarly to the "Ok Google". Dedicated chip and super small on-device ML model. Nothing goes back to the cloud.
    - stonewareslord 5y ago
      > Nothing goes back to the cloud What is your source on this? Every recognizable song is surely not stored on-device so something must be uploaded.
      - vadfa 5y ago
        It is actually stored in the device. A new database is downloaded from time to time
      - joestrong 5y ago
        I've tried putting my phone in airplane mode and the feature still works. You can easily try that yourself.
        foobiekr 5y ago
        That says the analysis is done locally. It does not say nothing goes to the cloud.
      - vitno 5y ago
        It's not every song, the model is built from a list of N (100k?) most popular songs. If you pick something a bit obscure it won't work. The model is updated on occasion and pushed in bulk to pixel devices.
    - 5y ago
cleverpebble 5y ago
In this case, the microphones are in fact on and processed by some relatively cheap (in terms of power consumption) algorithm whose sole purpose is detecting that phrase. Only after it detects that phrase does it start processing speech in any significant way
PragmaticPulp 5y ago
It’s less CPU intensive to monitor for a single trigger phrase (“OK Google”) after which the more general, CPU-intensive speech to text is performed (either on device or in the cloud). The devices aren’t translating all speech to text all of the time and then checking it against the phrase “OK Google”. They’re looking for the trigger phrase.

How do Android and iOS known when you say "OK Google" or "Hey Siri" if they are not listening all the time? Or, is it accepted that they are?

by _wldu 5y ago

How do Android and iOS known when you say "OK Google" or "Hey Siri" if they are not listening all the time? Or, is it accepted that they are?

angelzen 5y ago

They are listening all the time. Presumably 'just' for the wakeword. Better yet, Alexa listens to everything you do inside your house, and the Ring Network listens to everything you do outside your house. And both of them are connected to the power outlet, so the 'but battery depletion' excuse doesn't apply.

justusthane 5y ago

The listening for the initial keyword (e.g. “Hey Siri”) is done on hardware designed specifically for that purpose. That’s why you can’t change the keyword to whatever you want. Only when that keyword is heard does it actively listen to and process whatever follows. “Actively” listening all the time would simply consume too much CPU/bandwidth/battery.

dylan604 5y ago
>That’s why you can’t change the keyword to whatever you want. Which I feel is a huge security oversight. You should be able to choose from a list of activation keywords at least. I love being on video/voice chats with people that I know have a device and randomly say "Alexa, play Dancing Queen" or something equally obnoxious like "Alexa, turn off lights".
angelzen 5y ago
What exactly prevents the 'hardware designed specifically for that purpose' to include in its purpose the translation of your voice to text, with voice fingerprinting for good measure? Have you ran an electronic microscope atom layer slicing of the hardware to guarantee its complete set of functionalities?
- midasuni 5y ago
  Power consumption
  - angelzen 5y ago
    The hardware runs an algorithm to detect the wakeword. What makes you confident that a different algorithm can't be run to translate voice to text? They will be in the same family of algorithms, just with different purposes. To make it more interesting, let's assume that 2021 algorithms can't do that. Will better low power algorithms be developed in the next 20 years? How is a democratic society supposed to work when would-be-snitches are ubiquitously deployed? To make it even more interesting, as more an more of our communications move to the internet (oooOOOooo, covid, cower in fear), what confidence you have that voice2text and voice fingerprinting does not already apply to all your phone/Zoom conversations? Do you run the phone/Zoom servers and monitor all potentially suspicious traffic? Have you heard of one Edward Snowden?
    - kempbellt 5y ago
      When you train your phone to understand "Hey Siri" it requires an internet connection for the training so that it can verify that you are actually saying, "Hey Siri" and not "Yo, phone spirit". After the initial training, it no longer requires (or uses) the internet to detect the keyphrase. You are asked to do a few iterations of training so that the phone has a model on your voiceprint for "Hey Siri". Which is why, when someone who does not have a similar voice to you says "Hey Siri" it does not trigger. But someone who does have a similar voice to you can sometimes trigger it. The model is super basic. The microphone is always passively passing input data to the "Hey Siri" detector (and a buffer), which acts as a sort of internal switch. If it detects the trigger phrase, it then sends the audio data to the cloud to be transcribed, and acted upon accordingly. It is not always sending voice data to the cloud. If it were, your battery life would be noticeably impacted - worse than being on a phone call.
      - angelzen 5y ago
        Good explanation, thanks. The scenario is 'translate on device, steganograph the output on background traffic'. There is a huge push for 'on device ML', driven by noble intentions of hiding raw data from Skynet. Alas, the end of that road is devices that compress the raw sensorial input and report, possibly stealthily, the digest to Skynet.
alpha_squared 5y ago
Don't Android phones have passive listening? Pixel phones have always-on song detection (Shazam-like) and usually display song info on the lockscreen within ~10 seconds of the song being played nearby.
- vitno 5y ago
  That works somewhat similarly to the "Ok Google". Dedicated chip and super small on-device ML model. Nothing goes back to the cloud.
  - stonewareslord 5y ago
    > Nothing goes back to the cloud What is your source on this? Every recognizable song is surely not stored on-device so something must be uploaded.
    - vadfa 5y ago
      It is actually stored in the device. A new database is downloaded from time to time
    - joestrong 5y ago
      I've tried putting my phone in airplane mode and the feature still works. You can easily try that yourself.
      - foobiekr 5y ago
        That says the analysis is done locally. It does not say nothing goes to the cloud.
    - vitno 5y ago
      It's not every song, the model is built from a list of N (100k?) most popular songs. If you pick something a bit obscure it won't work. The model is updated on occasion and pushed in bulk to pixel devices.
  - 5y ago

cleverpebble 5y ago

In this case, the microphones are in fact on and processed by some relatively cheap (in terms of power consumption) algorithm whose sole purpose is detecting that phrase. Only after it detects that phrase does it start processing speech in any significant way

PragmaticPulp 5y ago

It’s less CPU intensive to monitor for a single trigger phrase (“OK Google”) after which the more general, CPU-intensive speech to text is performed (either on device or in the cloud). The devices aren’t translating all speech to text all of the time and then checking it against the phrase “OK Google”. They’re looking for the trigger phrase.