7 ms·

There is a big difference between smoke alarm detection built into the wake-up-word circuitry, versus every sound sent to Google's servers for analysis. I am no

by paul_f 6y ago

There is a big difference between smoke alarm detection built into the wake-up-word circuitry, versus every sound sent to Google's servers for analysis. I am not bothered by the former, but would be aghast at the latter. Does anyone know for sure?

Waterluvian 6y ago
What does it matter? The bottom line is that once there’s microphones in your home, Google just has to wait out the normalization of deviance until nobody’s surprised or upset that they transfer everything to their servers for analysis. If they don’t today, they will one day. They don’t have a choice on the matter because it’s a money maker.
- marricks 6y ago
  Google did this with “we don’t scan your emails for ads”. Initially, 2009 ish, that was a big line a lot of people cared about but years later here we are, Google scanning emails for targeted ads and that fight is over. EDIT: Thanks for the reply exittheone, I see GMail actually stopped this ad scanning practice in 2017[1], likely with google sign in on chrome and so many places it’s very possibly they just don’t that extra info. They still can let third party extensions read your email so I wouldn’t say we’re exactly in a better world...[2] 1. https://variety.com/2017/digital/news/google-gmail-ads-emails-1202477321/ https://variety.com/2017/digital/news/google-gmail-ads-email... 2. https://mashable.com/article/google-reading-your-emails-response/ https://mashable.com/article/google-reading-your-emails-resp...
  - exitheone 6y ago
    https://support.google.com/mail/answer/6603?hl=en https://support.google.com/mail/answer/6603?hl=en This is most definitely false. Google still states that they do not scan your emails for as targeting.
    - nkozyra 6y ago
      And why would they? The data we provide to Google in myriad ways across numerous devices and services is far more fungible.
      - TallGuyShort 6y ago
        This thread is like Chris Farley's lines from Billy Madison. "That Veronica Vaughn is one hot piece of ass. And I know from experience." "No you don't" "Well not me personally, but this guy I know. Him and her GOT IT ON." "No they didn't" "But you can imagine what it would be like if they did."
        jhardy54 6y ago
        No it isn't. Google didn't originally scan emails. Then they did. Then they didn't. Countering "Google scanned emails" with "well the don't right now" is the only ridiculous behavior in this thread.
        joshuamorton 6y ago
        But they did[0] from the beginning. And then later they stopped. There was no start then stop. There was only a stop. (I work on Gmail) [0]: https://privacyrights.org/resources/privacy-and-civil-liberties-organizations-urge-google-suspend-gmail https://privacyrights.org/resources/privacy-and-civil-libert... (from 2004, the year Gmail launched)
        jhardy54 6y ago
        Interesting, thanks for the correction! I could've sworn that Gmail provided some "we don't scan your email" privacy since the beginning but it sounds like I was confused.
    - tunap 6y ago
      Your words are as spurious as Goog's. Where you state Still means, they STOPPED scanning emails for targeted ads, supposedly in 2017(Did they "pinky-swear"?). So, they always did before, as stated it in plain words in 1st 2 paragraphs of their previous EULAs. Now that they claim they don't "scan emails for ads", they don't state they no longer scan emails and it does not state what they do, currently, scan emails for. Please don't retort "they don't sell PII", as P2 most definitely is what they use to sell X to 3rd parties in whatever form the ubiquitous NDAs prevent the public from knowing. https://en.wikipedia.org/wiki/Gmail#Automated_scanning_of_email_content https://en.wikipedia.org/wiki/Gmail#Automated_scanning_of_em... Edit: P2 > PII, am watching motogp & removed unnecessary personal jab.
  - exitheone 6y ago
    First of all, thanks for linking to the change in policy. I was not aware of the timeline. To address both links: 1) As a user I'm actually happy about the tight integration into calendar and other apps. These necessarily require reading my mail though. I'm already trusting Google with my mail so using it for better integration into other Google products is fine with me. 2) They are not randomly handing out access to your emails to third parties, they just have an API that lets third parties read your emails _after you gave explicit consent via the regular oAuth flow_. Which is also completely reasonable for me.
  - deleted 6y ago
    [deleted]
secabeen 6y ago
Now that the feature is disabled, it's hard to know. Once the feature is enabled, it's pretty easy to tell by looking at the network traffic the device generates. Sending every sound to google is going to be obvious.
- im3w1l 6y ago
  You could have covert wake-words.
  - secabeen 6y ago
    You'd still see the network traffic for the audio going back to Amazon's servers for analysis.
- MiroF 6y ago
  What about transcribing all the audio you get and then sending it as a chunk of stuff every time the Hone is activated. Don't think you could catch that.
  - secabeen 6y ago
    That would require a lot more computational power than is currently available in an Echo device. It's possible that Amazon could sneak a custom ASIC into the device to do something like this, but it would add quite a bit to the device's Bill of Materials, and would still probably be noticed by the iFixit crowd.
- Androider 6y ago
  The article makes it unclear, but this feature is available right now for all Nest Aware subscribers (if you have any type of Nest camera, you'll have a Nest Aware subscription). The article mentioned that this feature was "accidentally" available to non-Nest Aware subscribers as well, but it's the same existing feature and not some test of upcoming features. You can toggle glass break and fire alarm detection on any of your Google Home devices in the Home app, and this includes the old Google Home mini pucks and hubs, as well as new Nest branded minis and hubs.
pabe 6y ago
Most likely, an AI model that detects specific sounds is running directly on those devices. No need to transfer anything to Google's servers for analysis =)
- _visgean 6y ago
  From what I have heard you don't need any fancy AI models for detecting either of these sounds. Some of the older alarms were using classic signal processing to to this decades ago.
  - MiroF 6y ago
    But why wouldn't they have AI models? It's how I would do it.
    - _visgean 6y ago
      Well they might do it with AI, there are however some issues, constantly running sounds through ML models is power hungry which is as far as i know why devices such as these optimize a lot for commands prompts such as "OK google" which is when the ML loop actually starts. On the other smoke alarm sound is very easy to detect with classical methods so might be extremely cheap to run it without any ML.
      - MiroF 6y ago
        > constantly running sounds through ML models is power hungry which is as far as i know why devices such as these optimize a lot for commands prompts such as "OK google" which is when the ML loop actually starts. How power hungry is it actually? I'd only have to run it on the big model if a pretty dumb model thinks that the voice is even in the audio stream.
  - joshvm 6y ago
    Similarly glass-break detectors for home security systems have done this for years. The advantage of an ML model is that you can do multi-class prediction for a sound clip with somewhat arbitrary complexity and the cost of execution is more or less the same even if you add an extra class or two. It's just an extra row in your output prediction vector. By complexity I mean signals that don't have an obviously characteristic spectrum, like glass smashing. A typical CNN backbone is capable of classifying hundreds if not thousands of classes with high accuracy - even the edge architectures. Always-on detection tends to use very compact networks (kB in size) that will run on low-power ARM cores, or even specialist ASICs, but even so 20-30 common audio event types seems very feasible. For people worrying about sending data back, there's no reason why you'd do this off-device. The exception might be that you feedback to Google that there was a false alarm, so they can use your sound clip as a negative training example. Just a guess there, but Tesla does this extensively for Autopilot - they deploy models in your car and specifically ask it to capture images of rare events (Andrej Karpathy gave an example of tree-occluded stop signs).
chaboud 6y ago
It’s easy enough to test, but I don’t really need to. Alexa Guard has had this functionality for a while, and I’d expect the folks here at HN to be able to infer a few things from the support link and basic reasoning. Support link https://support.ring.com/hc/en-us/articles/360028205592-Using-Alexa-Guard-with-Alarm-to-Detect-Broken-Glass-Smoke-and-Carbon-Monoxide https://support.ring.com/hc/en-us/articles/360028205592-Usin... So: 1) if an event is detected, you can listen to a 10 second clip or drop in (2-way call) to listen in or look. 2) Echo devices have relatively small amounts of RAM 3) Echo devices aren’t constantly hammering WiFi connections From this, one should be able to deduce that the wakeword engine detects events and streams clips to servers only in situations that match events and settings to support these features. Why? Because processing, transit, and storage aren’t free, and one can’t store data in RAM that isn’t there or transmit data over WiFi without the physical layer showing signs of it. Furthermore, Amazon hasn’t cracked the code on hyper-efficient GB into KB lossless compression only to squirrel it away only for use in voice assistants. Take the number of Alexa devices sold and run the numbers for all of those devices sending audio data to AWS all the time. The costs would be astronomical. The same goes for Google (though not with AWS). They’re no doubt incorporating the detectors into their on-device models.
- econcon 6y ago
  > Furthermore, Amazon hasn’t cracked the code on hyper-efficient GB into KB lossless I thought kbg archiver solved it eons ago.
  - jdc 6y ago
    Nope, its high compression comes at the cost of similarly high CPU and memory usage.
    - nitrogen 6y ago
      Compressed audio at speech quality uses miniscule amounts of storage, especially with modern codecs like Opus.

by paul_f 6y ago

Waterluvian 6y ago

What does it matter? The bottom line is that once there’s microphones in your home, Google just has to wait out the normalization of deviance until nobody’s surprised or upset that they transfer everything to their servers for analysis. If they don’t today, they will one day. They don’t have a choice on the matter because it’s a money maker.

marricks 6y ago
Google did this with “we don’t scan your emails for ads”. Initially, 2009 ish, that was a big line a lot of people cared about but years later here we are, Google scanning emails for targeted ads and that fight is over. EDIT: Thanks for the reply exittheone, I see GMail actually stopped this ad scanning practice in 2017[1], likely with google sign in on chrome and so many places it’s very possibly they just don’t that extra info. They still can let third party extensions read your email so I wouldn’t say we’re exactly in a better world...[2] 1. https://variety.com/2017/digital/news/google-gmail-ads-emails-1202477321/ https://variety.com/2017/digital/news/google-gmail-ads-email... 2. https://mashable.com/article/google-reading-your-emails-response/ https://mashable.com/article/google-reading-your-emails-resp...
- exitheone 6y ago
  https://support.google.com/mail/answer/6603?hl=en https://support.google.com/mail/answer/6603?hl=en This is most definitely false. Google still states that they do not scan your emails for as targeting.
  - nkozyra 6y ago
    And why would they? The data we provide to Google in myriad ways across numerous devices and services is far more fungible.
    - TallGuyShort 6y ago
      This thread is like Chris Farley's lines from Billy Madison. "That Veronica Vaughn is one hot piece of ass. And I know from experience." "No you don't" "Well not me personally, but this guy I know. Him and her GOT IT ON." "No they didn't" "But you can imagine what it would be like if they did."
      - jhardy54 6y ago
        No it isn't. Google didn't originally scan emails. Then they did. Then they didn't. Countering "Google scanned emails" with "well the don't right now" is the only ridiculous behavior in this thread.
        joshuamorton 6y ago
        But they did[0] from the beginning. And then later they stopped. There was no start then stop. There was only a stop. (I work on Gmail) [0]: https://privacyrights.org/resources/privacy-and-civil-liberties-organizations-urge-google-suspend-gmail https://privacyrights.org/resources/privacy-and-civil-libert... (from 2004, the year Gmail launched)
        jhardy54 6y ago
        Interesting, thanks for the correction! I could've sworn that Gmail provided some "we don't scan your email" privacy since the beginning but it sounds like I was confused.
  - tunap 6y ago
    Your words are as spurious as Goog's. Where you state Still means, they STOPPED scanning emails for targeted ads, supposedly in 2017(Did they "pinky-swear"?). So, they always did before, as stated it in plain words in 1st 2 paragraphs of their previous EULAs. Now that they claim they don't "scan emails for ads", they don't state they no longer scan emails and it does not state what they do, currently, scan emails for. Please don't retort "they don't sell PII", as P2 most definitely is what they use to sell X to 3rd parties in whatever form the ubiquitous NDAs prevent the public from knowing. https://en.wikipedia.org/wiki/Gmail#Automated_scanning_of_email_content https://en.wikipedia.org/wiki/Gmail#Automated_scanning_of_em... Edit: P2 > PII, am watching motogp & removed unnecessary personal jab.
- exitheone 6y ago
  First of all, thanks for linking to the change in policy. I was not aware of the timeline. To address both links: 1) As a user I'm actually happy about the tight integration into calendar and other apps. These necessarily require reading my mail though. I'm already trusting Google with my mail so using it for better integration into other Google products is fine with me. 2) They are not randomly handing out access to your emails to third parties, they just have an API that lets third parties read your emails _after you gave explicit consent via the regular oAuth flow_. Which is also completely reasonable for me.
- deleted 6y ago
  [deleted]

secabeen 6y ago

Now that the feature is disabled, it's hard to know. Once the feature is enabled, it's pretty easy to tell by looking at the network traffic the device generates. Sending every sound to google is going to be obvious.

im3w1l 6y ago
You could have covert wake-words.
- secabeen 6y ago
  You'd still see the network traffic for the audio going back to Amazon's servers for analysis.
MiroF 6y ago
What about transcribing all the audio you get and then sending it as a chunk of stuff every time the Hone is activated. Don't think you could catch that.
- secabeen 6y ago
  That would require a lot more computational power than is currently available in an Echo device. It's possible that Amazon could sneak a custom ASIC into the device to do something like this, but it would add quite a bit to the device's Bill of Materials, and would still probably be noticed by the iFixit crowd.
Androider 6y ago
The article makes it unclear, but this feature is available right now for all Nest Aware subscribers (if you have any type of Nest camera, you'll have a Nest Aware subscription). The article mentioned that this feature was "accidentally" available to non-Nest Aware subscribers as well, but it's the same existing feature and not some test of upcoming features. You can toggle glass break and fire alarm detection on any of your Google Home devices in the Home app, and this includes the old Google Home mini pucks and hubs, as well as new Nest branded minis and hubs.

pabe 6y ago

Most likely, an AI model that detects specific sounds is running directly on those devices. No need to transfer anything to Google's servers for analysis =)

_visgean 6y ago
From what I have heard you don't need any fancy AI models for detecting either of these sounds. Some of the older alarms were using classic signal processing to to this decades ago.
- MiroF 6y ago
  But why wouldn't they have AI models? It's how I would do it.
  - _visgean 6y ago
    Well they might do it with AI, there are however some issues, constantly running sounds through ML models is power hungry which is as far as i know why devices such as these optimize a lot for commands prompts such as "OK google" which is when the ML loop actually starts. On the other smoke alarm sound is very easy to detect with classical methods so might be extremely cheap to run it without any ML.
    - MiroF 6y ago
      > constantly running sounds through ML models is power hungry which is as far as i know why devices such as these optimize a lot for commands prompts such as "OK google" which is when the ML loop actually starts. How power hungry is it actually? I'd only have to run it on the big model if a pretty dumb model thinks that the voice is even in the audio stream.
- joshvm 6y ago
  Similarly glass-break detectors for home security systems have done this for years. The advantage of an ML model is that you can do multi-class prediction for a sound clip with somewhat arbitrary complexity and the cost of execution is more or less the same even if you add an extra class or two. It's just an extra row in your output prediction vector. By complexity I mean signals that don't have an obviously characteristic spectrum, like glass smashing. A typical CNN backbone is capable of classifying hundreds if not thousands of classes with high accuracy - even the edge architectures. Always-on detection tends to use very compact networks (kB in size) that will run on low-power ARM cores, or even specialist ASICs, but even so 20-30 common audio event types seems very feasible. For people worrying about sending data back, there's no reason why you'd do this off-device. The exception might be that you feedback to Google that there was a false alarm, so they can use your sound clip as a negative training example. Just a guess there, but Tesla does this extensively for Autopilot - they deploy models in your car and specifically ask it to capture images of rare events (Andrej Karpathy gave an example of tree-occluded stop signs).

chaboud 6y ago

It’s easy enough to test, but I don’t really need to. Alexa Guard has had this functionality for a while, and I’d expect the folks here at HN to be able to infer a few things from the support link and basic reasoning. Support link https://support.ring.com/hc/en-us/articles/360028205592-Using-Alexa-Guard-with-Alarm-to-Detect-Broken-Glass-Smoke-and-Carbon-Monoxide https://support.ring.com/hc/en-us/articles/360028205592-Usin... So: 1) if an event is detected, you can listen to a 10 second clip or drop in (2-way call) to listen in or look. 2) Echo devices have relatively small amounts of RAM 3) Echo devices aren’t constantly hammering WiFi connections From this, one should be able to deduce that the wakeword engine detects events and streams clips to servers only in situations that match events and settings to support these features. Why? Because processing, transit, and storage aren’t free, and one can’t store data in RAM that isn’t there or transmit data over WiFi without the physical layer showing signs of it. Furthermore, Amazon hasn’t cracked the code on hyper-efficient GB into KB lossless compression only to squirrel it away only for use in voice assistants. Take the number of Alexa devices sold and run the numbers for all of those devices sending audio data to AWS all the time. The costs would be astronomical. The same goes for Google (though not with AWS). They’re no doubt incorporating the detectors into their on-device models.

econcon 6y ago
> Furthermore, Amazon hasn’t cracked the code on hyper-efficient GB into KB lossless I thought kbg archiver solved it eons ago.
- jdc 6y ago
  Nope, its high compression comes at the cost of similarly high CPU and memory usage.
  - nitrogen 6y ago
    Compressed audio at speech quality uses miniscule amounts of storage, especially with modern codecs like Opus.