11 ms·

While I appreciate the sentiment...unless you actually think Google and Amazon devices are recording irrelevant ambient sound deliberately (they aren’t), this d

by wrs 8y ago

While I appreciate the sentiment...unless you actually think Google and Amazon devices are recording irrelevant ambient sound deliberately (they aren’t), this doesn’t help anything. Unless the software here is better than theirs at recognizing the trigger word (very unlikely), there will be even more false positive activations on this device than there are on the originals. Edit: It’s very unlikely because Amazon and Google pay for false positives, so they have a strong incentive to develop really good trigger word detection.

heinrichhartman 8y ago
> (they aren’t) How do you know? And, how do you know they will not do this silently in the future? Also worse detection does not mean more false positives. Usually, you can get the false positive rate very low by allowing more false negatives. In this way you have a choice, how you want to trade-off. Without this device, you are stuck with the choice that Amazon/Google make for you.
- bussierem 8y ago
  > How do you know? And, how do you know they will not do this silently in the future? Because it's a literal hardware limitation. The device is built in a way that requires a wake word before any recording can possibly happen, thanks to it being built with 2 separate control boards. If they ended up maybe changing the wakeword to "the", then maybe they could "silently" listen to everything, but that would be caught pretty quick because the device would be "lit up" constantly (another _hardware_ thing), or someone would notice that it no longer responds to "Alexa" or "Google". Seriously, a lot of people on HN need to do their damn homework about these devices before declaring them to be something they have been proven not to be. Packet sniffing and hardware inspection both instantly disprove all these conspiracy-theory nonsense claims that these devices are recording your every word.
  - zdkl 8y ago
    There's a widespread sentiment that current evidence of compliance to "doing right by users" should be viewed with circumspection. And it's fair to say Google's past behaviour raises doubts about the level of trust users should extend them. What is a conspiracy theory about today's hardware, I have no trouble imagining is a planned or at least considered future iteration of their "service".
  - stronglikedan 8y ago
    > Packet sniffing and hardware inspection both instantly disprove... I'm under the impression that packet sniffing is useless with end-to-end encryption, but I could be wrong. I.e., you can tell that something is being sent, but you can't know what.
    - Arelius 8y ago
      The theory is that we can still estimate how much data goes over the network. So if it were sending all audio to the cloud, we'd see. It does however not exclude other information, like sending keyword flags, or storing audio fragments to send along with other messages later on.
      - dcbadacd 8y ago
        Neither does it exclude the possibility of any "time-bomb"-type of features or future OTA updates altering the software's behavior.
    - LeftTurnSignal 8y ago
      If OP can show what they use to packet sniff those boxes, then life will be good and we can put the conspiracy theories to bed. If they can't see all of the traffic, but just know where the traffic is going, then I don't think they did their homework.
    - UncleMeat 8y ago
      You own the client. You can do anything to it. There is no way for encryption on the client to prevent you from inspecting the content.
  - falcolas 8y ago
    How about the issue where Google’s devices were errantly recording everything due to a hardware issue - where the button override for the voice activation was stuck in the activated position. People who think that there’s no way that Google and Amazon could be recording everything need to realize that this is also not true. Most of these “limitations” are software enforced, and that software is updated constantly.
    - sailfast 8y ago
      This is the main problem that I see. Sure, I tested the packets, sniffed them, made sure it wasn't recording, etc, but then they push an update the next day. I don't think it's practical to monitor these devices all the time, and I haven't been asked to opt-in to an Echo update. I also don't necessarily assume mal-intent on the part of the companies, but that doesn't mean there won't _ever_ be that intent. Trusting that all of these assumptions hold over time is hard.
      - falcolas 8y ago
        > Trusting that all of these assumptions hold over time is hard. This is the big point. You are not only trusting that the company as it is today is doing the right thing, but that the company will continue to do the right thing for so long as the device is in your house - and that they will do the right thing in perpetuity with your data (including if/when they sell the company down the road).
      - mehrdadn 8y ago
        > This is the main problem that I see. Sure, I tested the packets, sniffed them, made sure it wasn't recording, etc, but then they push an update the next day. I don't think it's practical to monitor these devices all the time, and I haven't been asked to opt-in to an Echo update. This is a problem with forced updates in general (I'm also thinking of Windows, Chrome, Chrome extensions, etc. here) that security experts seem completely blind to. That said, note that even if the software didn't update, it doesn't mean it would have to send bad packets when you're actually observing. It could randomly start doing that once in a while after a few months.
      - 8y ago
  - Arelius 8y ago
    Citation strongly needed. I do understand that the wake word processing happens in a special kernel in a low power state, however.... The wake word is a trained kernel, it can be trained to listen for a huge set of things (as seen in the Pixel's passive song detection), so they would just train the kernel to detect a large targeted (marketing?) vocab. about being "lit up" constantly? I'm not saying you are wrong, but I'd really like to see a citation that this is true. Is it true for both echo and home? And while Packet Sniffing can disprove that it's listening and sending whole audio to the cloud, it can't disprove that it's listening for a huge set of "wake words" and toggling bits in other control messages to track users in more subtle ways.
  - teacpde 8y ago
    > If they ended up maybe changing the wakeword to "the", then maybe they could "silently" listen to everything, but that would be caught pretty quick because the device would be "lit up" constantly (another _hardware_ thing) From pure technical perspective, can the device not be programmed to be waked by wakewaord “the” with the light off?
    - darkpuma 8y ago
      > "can the device not be programmed to be waked by wakewaord “the” with the light off?" It absolutely can be.
  - mondoshawan 8y ago
    Those two separate control boards didn't stop my Amazon dot from acually recording ambient noise and uploading it to Amazon's systems. I know this because of the audio history they themselves provide! You can literally go back and play back all the audio recorded, and a great deal of it did not include questions. Further, there was also a report of being able to trigger audio recording without either activating the LED ring or using a wake word via a serial root console. While a third-party attacker is unlikely to use that method of access, nothing about the hardware actively prevents Amazon ftom triggering it that way. Likewise for Google. And yes, I work on this stuff. Neither Google nor Amazon have the hardware limitations you suggest.
    - jhall1468 8y ago
      You are making an enormous amount of assumptions based on a semantic argument. Echo devices only begin recording if they think they hear the wake word. Obviously this is less than straight-forward, hence the recordings that didn't follow the wake word (just examples of an Alexa device incorrectly thinking it heard it). To suggest that a serial root console is a point of attack for an Echo device is bordering on insanity. You'd need a breakout board connected via the USB interface (not port, mind you) in order for this work-around to be effective. So yes, if a hacker had physical access to your device, time enough to solder on a breakout board, said third-party could record a variety of things. But then, it's a whole hell of a lot easier to just install a mic in someones house and get the same effect, now wouldn't it?
      - monsieurbanana 8y ago
        > To suggest that a serial root console is a point of attack for an Echo device is bordering on insanity. That was not what he said. He argues that Amazon/Google could remotely use a similar exploit (without direct access to the hardware) to start recording without lighting up the LED.
        jhall1468 8y ago
        Nobody has EVER gotten root console access on an Echo device remotely, and the only successful "remote" exploit that didn't require soldering requires that the attacker and the victim are both on the same wifi network. Please, feel free to explain how Amazon and Google could exploit that vulnerability (that has since been patched)? More importantly, I'd love to hear how they are going to pull this off and hide it, given network traffic will be a dead give away? If what your suggesting is actually what he meant, that's even more absurd than attackers trying to do the same.
  - shittyadmin 8y ago
    > the device would be "lit up" constantly (another _hardware_ thing) Unless the mic and lights are somehow wired together (they aren't), then this is really just more software which can be trivially updated away.
    - eclipxe 8y ago
      They are.
      - shittyadmin 8y ago
        How so? Ultimately the mic is always on and listening for its keywords - if you look at the teardown of the Alexa on iFixIt, I don't even see any device other than the main CPU that would be capable of performing keyword recognition. Meaning the main CPU would have to be the thing then controlling the lights after the keywords are recognized... The Google Home at least has a separate board with a microcontroller on it which could be used for keyword recognition, but I'm pretty sure they allow that to be updated for the sake of improving keyword recognition and there's no reason that an update couldn't disable the LEDs in the listen state as far as I can see.
        a_t48 8y ago
        Not a hardware guy, but couldn’t you tie the LEDs to whatever bus that connects the mic and the main CPU?
        shittyadmin 8y ago
        Yes, I don't mean to say it's impossible - just that you'd need an entirely isolated system to detect when data was flowing over that link which is physically connected in all cases and cannot be updated. I don't believe we see that in either the Alexa or Google Home, but I'd be happy to be mistaken if anyone's done a more in depth teardown of these systems. And all of this is hinged on hoping you notice LEDs firing in the corner while you're having a conversation. Perhaps a more noticeable method should be used in cases like this. A forced "beep"/tone or something from an isolated circuit hardwired to the speakers.
  - glitchc 8y ago
    These are software companies. These devices support OTA updates, including changing the activation word. It's trivial to change the activation word to empty or something innocuous. Ergo, there's no hardware limitation. If your argument stems from "Google and Amazon would never do that," I do not trust any corporate entity to value my rights more than their ability to make a dollar.
  - rsync 8y ago
    "The device is built in a way that requires a wake word before any recording can possibly happen, thanks to it being built with 2 separate control boards." This is a very technically naive interpretation of their hardware/software solution. If the wake word were hard-coded into silicon then perhaps I would be charitable about your misunderstanding(s) - but of course it is not. The wake word is user-definable and can be changed to arbitrary sounds at any time. Whatever hardware limitation(s) may exist are trivially worked around with software, which can be updated over the top of you at any time.
  - withinrafael 8y ago
    > Because it's a literal hardware limitation. The device is built in a way that requires a wake word before any recording can possibly happen [...] Take note that Amazon Drop In [1] is a feature built around turning on the Echo mic remotely without a wake word. I don't think this feature could exist if there was a hardware limitation. [1] https://www.amazon.com/gp/help/customer/display.html?nodeId=202153130 https://www.amazon.com/gp/help/customer/display.html?nodeId=...
    - ehsankia 8y ago
      But doesn't the device light up when that happens?
      - withinrafael 8y ago
        I was just offering a counter to the hardware limitation claim. It's possible the device makes itself known during use, I haven't used this feature yet.
      - jedieaston 8y ago
        And it beeps, and the audio from the other end starts coming through the echo's speaker. There is just about no way to know someone dropped in on you.
        smnrchrds 8y ago
        But is this behaviour implemented in hardware or software?
        withinrafael 8y ago
        This feature is configured via its software.
        ehsankia 8y ago
        My understanding is that the light at the very least was hardware. Sound can be disabled.
  - tomn 8y ago
    Do you (or anyone else with the same claim) have a citation for this? I've spent some time reverse-engineering the echo microphone board, and while there is an interlock that prevents recording while the red mute button is lit (just the light under the button, not the ring), I didn't see anything that would prevent recording while the ring light was off.
  - trhway 8y ago
    > Packet sniffing The unauthorized recorded data can be sent encoded with/into the authorized data.
  - LinuxBender 8y ago
    > Because it's a literal hardware limitation. Unless there is a separate out of band board with a relay I can hear or see (meaning, code alone can't enable something), then it really isn't a hardware limitation. The security controls and operations are in the code. The code can change or may already have silent monitoring capabilities. Nobody on HN could really answer whether or not this is the case. All we can do is speculate. If someone were required to put lawful monitoring code in place, they would not be allowed to discuss it here. The best anyone could do is decompile the code or get the source code for the firmware. Even then, there could be non-volital space that allows for updates. Case in point, there have been malware packages that could enable your microphone and camera on the laptop without turning on the LED. This varied with camera model. Some power the LED when the camera has power. Microphones don't always activate an LED. There are a myriad of articles you can find providing examples of malware that can listen to cell phone microphones, laptop microphones without activating the LED.
  - darkerside 8y ago
    I get that you believe this, and I even understand you repeating it to other people on the Internet. What I don't get is that your tone indicates that you are offended people don't believe what you believe... which also just happens to have been incorrect in the past and many others seem to think is provably possible in the future.
  - TeMPOraL 8y ago
    > Seriously, a lot of people on HN need to do their damn homework about these devices before declaring them to be something they have been proven not to be. Based on what? Marketing copy? Eyeballing iFixit teardowns? > but that would be caught pretty quick because the device would be "lit up" constantly (another _hardware_ thing) Totally not buying it, unless you can show me the traces and discrete components that force power through the LED when signal from the microphone is allowed to reach the uC. If it's done in software, I'm not trusting it. If you're making an argument of "trust the vendor because economics", you have to recognize how weak it is.
  - amatecha 8y ago
    > Because it's a literal hardware limitation. Citation needed. Further, listening for a wake word and reacting to that is likely done completely in software: the fact it's even listening for a "wake word" means the hardware (microphone) is in fact always listening, it's just [presumably] not actually sending that audio to The Cloud (tm).
  - tokyodude 8y ago
    I don't own either of them but Siri and Google on my phone both require training when I first use them. Do these devices not? IF they do then isn't that proof they are re-programmable and could be programmed to respond to anything?
  - acct1771 8y ago
    Good luck packet sniffing an encrypted text blob of your conversations the device is transcribing.
- wrs 8y ago
  >How do you know? Because that would be pointless. The real problem right now is false triggering. If you’re actually worried about being spied on, why on earth would you have one of these in the first place?
- deleted 8y ago
  [deleted]
- pwython 8y ago
  Not that this helps anyone sleep easier, but imagine in today's age... a whistleblower -- perhaps one of the thousands of software devs working on one of these -- leaked proof that these devices are recording everything to re-market and profit, without permission... The resulting backlash and legal ramifications would be so huge it just wouldn't be worth it. It wouldn't just take an insane and stupid CEO to do that, but also thousands of other tech/adops employees who'd have to be like, "yea this is a great idea."
  - sp332 8y ago
    I just kinda doubt this. How much backlash was there when it came out that the NSA was recording the full content of every cell phone call in the Bahamas? Edit: codename SOMALGET, subproject of MYSTIC. https://en.wikipedia.org/wiki/MYSTIC_(surveillance_program)#Targets https://en.wikipedia.org/wiki/MYSTIC_(surveillance_program)#... http://www.documentcloud.org/documents/1164088-somalget.html http://www.documentcloud.org/documents/1164088-somalget.html
    - debacle 8y ago
      > was recording the full content of every cell phone call in the Bahamas? The what now?
      - 0xADEADBEE 8y ago
        I didn't realise this either but it looks pretty widely known: https://theintercept.com/2014/05/19/data-pirates-caribbean-nsa-recording-every-cell-phone-call-bahamas/ https://theintercept.com/2014/05/19/data-pirates-caribbean-n...
    - archgoon 8y ago
      Why would there be? The Bahamas is not inside the United States and thus is part of the NSA's mission of monitoring foreign communications. Spy agencies spy. It's their job description.
      - deleted 8y ago
- pacala 8y ago
  "Allow Google Maps to always access your location. Yes. Ask Me Later". Given the multiple precedents on the erosion of privacy path in the past 20 years, of which I quoted one example above, it's pretty obvious that they will turn "always on listening" in the future, using whatever dark patterns necessary to avoid a class action suit.
  - paulie_a 8y ago
    I am getting sick of yes, ask me later. Stop asking me completely
mohaine 8y ago
Actually, it doubles your area of risk. Now you have 2 companies to worry about per device.
- guelo 8y ago
  It's an open source project. There's no company to trust.
  - kawfey 8y ago
    Users without the skills to verify the code isn't nefarious have to trust good samaritan developers instead.
    - spion 8y ago
      Nothing is 100% guaranteed, but with an open source project, given enough users, its far less likely for someone to be able to bury nefarious stuff without many eyes looking at it and at least one person sounding an alert.
      - dymk 8y ago
        Yeah but really this isn't true. Popular open source that has tens of thousands of eyes on it still gets compromised all the time (see: npm). Even the Linux kernel has had rogue git commits injected into it.
        calcifer 8y ago
        > the Linux kernel has had rogue git commits injected into it What? Who "injected" what and when?
        spacedog11 8y ago
        I'm also interested to know more about this.
        8y ago
floatingatoll 8y ago
The sentiment this is countering is not "Google and Amazon _are_ recording 24/7/365". It is countering "Amazon and Google _would_ record 24/7/365 if they could get away with it socially". Arguing that the technology doesn't do this does not address the underlying root perception that Amazon and Google are not to be trusted. Google built its entire business on harvesting _all_ data on the web and building an infrastructure to process it efficiently. Purchasing Nest made clear that to us that they now wish to harvest data from the home. If they were able, socially and politically, to harvest _all_ data from the home, their history indicates that they would do so with all possible speed. Having a device you built yourself that learns locally and is under your control prevents Amazon and Google from changing their mind about what level of recording is acceptable without your informed opt-in consent at the time of the change. Both providers reserve the right to update their privacy policies without notification or consent, including granting themselves to increase data collection. TLDR: This device is the physical expression of mistrust of Amazon and Google. Their greed for your metadata is well-documented, and their policies let them increase data collection in your home at any time without your consent. Such increases would be prevented by this device.
Aunche 8y ago
Also, it would make no business sense to always be listening. A lot of people think that just because Alexa and Google Assistant are free to use, it means that these services are virtually free for the companies as well, but that's not the case. There is no way Amazon or Google would waste millions of dollars running the state of the art speech recognition algorithms on your house's background noise.
- reaperducer 8y ago
  There is no way Amazon or Google would waste millions of dollars running the state of the art speech recognition algorithms on your house's background noise. Unless they wanted to listen for a dog barking, then add "pet food buyer" to your profile data. There are a lot of very easy use cases for monitoring background noise.
guelo 8y ago
From a non-privacy perspective this adds the feature of being able to customize your wake word, which besides being a nice feature on its own also counters Google and Amazon's desire to inject their brands deeper into our psyche by making us say them out loud. From a privacy perspective, having user control of the wake word prevents Google and Amazon from adding future wake words that could be abused for other ways to track us. For example, Google might get the bright idea to track TV ads by listening for audio in the ads. Or tracking people in your house by making Android phones emmit non-audible chirps. These kind of "features" could be easily introduced at any point in the future by an update to privacy policies that nobody notices.
- reaperducer 8y ago
  Google might get the bright idea to track TV ads by listening for audio in the ads Considering that some TVs have this built-in, I'd be surprised if Google wasn't already doing the same. It's why my "smart" TV isn't allowed to connect to my wifi network. (There was a previous HN article about it, I believe the brand was Samsung.)
  - thinkingemote 8y ago
    I think it was the Facebook app, it listened to ambient background to determine music and TV shows. The user had to opt in or at least approve permissions.
crankylinuxuser 8y ago
> While I appreciate the sentiment...unless you actually think Google and Amazon devices are recording irrelevant ambient sound deliberately (they aren’t) Citation needed. Or if you have reference firmware I can load, that would be awesome... What do you mean its closed firmware and controlled by Amazon/Google? You mean they can change it whenever they wish, and we have no say??? Long story short; you rented a spy device and you trust some random person online it isn't spying... Even though there are credible stories of these devices doing precisely that. No, no, absolutely not, and hell no!
almost_usual 8y ago
Who cares. Whoever buys the hardware can do whatever they want with it. If someone feels better with this device on their Google / Alexa product let them do it. If the speech recognition is horrible I’m sure they’ll take it off.
- aylmao 8y ago
  +1 to who cares. > If the speech recognition is horrible I’m sure they’ll take it off. This is interesting and cool, cause it sounds to me like they only detect for the keyword to "unlock" the Google home, so in theory they don't even need speech recognition. In most cases, they could just do with telling if a sound you make seems to match the sound you defined to be their name ¯\_(ツ)_/¯
kakarot 8y ago
The point is establishing as much trust as you can with your devices. Only having external points of trust when absolutely necessary. This is generally a Good Thing, and something we should do by default, not as a reaction to some corporate leak.
aylmao 8y ago
> It’s very unlikely because Amazon and Google pay for false positives, so they have a strong incentive to develop really good trigger word detection. There's a feature in my Pixel to show what song is playing --like in the real world-- on the lock screen. A kind of always-on Shazam. They don't mind paying for always on.
- CraftThatBlock 8y ago
  That's fully on-device though, it keeps a database of the top songs fingerprints and doesn't use the network to recognize songs
- parineum 8y ago
  "Now Playing" is an offline feature https://venturebeat.com/2017/10/19/how-googles-pixel-2-now-playing-song-identification-works/ https://venturebeat.com/2017/10/19/how-googles-pixel-2-now-p...
  - aylmao 8y ago
    Go figure, I stand corrected.
- deleted 8y ago
  [deleted]
freeflight 8y ago
Afaik by now, there have been least two court cases where Echo recordings were handed over as evidence [0] [1]. At this rate, it's only a matter of time before evidence like that gets leaked/released, which would serve as a good probe on how much these devices really record. [0] https://techcrunch.com/2018/11/14/amazon-echo-recordings-judge-murder-case/ https://techcrunch.com/2018/11/14/amazon-echo-recordings-jud... [1] https://edition.cnn.com/2017/03/07/tech/amazon-echo-alexa-bentonville-arkansas-murder-case/index.html https://edition.cnn.com/2017/03/07/tech/amazon-echo-alexa-be...
onion2k 8y ago
Edit: It’s very unlikely because Amazon and Google pay for false positives, so they have a strong incentive to develop really good trigger word detection. If the false positives give them data that increases the value of your marketing profile by more than the cost of processing the interactions then they actively profit from false positives, and have no reason at all to stop them.

by wrs 8y ago

heinrichhartman 8y ago

> (they aren’t) How do you know? And, how do you know they will not do this silently in the future? Also worse detection does not mean more false positives. Usually, you can get the false positive rate very low by allowing more false negatives. In this way you have a choice, how you want to trade-off. Without this device, you are stuck with the choice that Amazon/Google make for you.

bussierem 8y ago
> How do you know? And, how do you know they will not do this silently in the future? Because it's a literal hardware limitation. The device is built in a way that requires a wake word before any recording can possibly happen, thanks to it being built with 2 separate control boards. If they ended up maybe changing the wakeword to "the", then maybe they could "silently" listen to everything, but that would be caught pretty quick because the device would be "lit up" constantly (another _hardware_ thing), or someone would notice that it no longer responds to "Alexa" or "Google". Seriously, a lot of people on HN need to do their damn homework about these devices before declaring them to be something they have been proven not to be. Packet sniffing and hardware inspection both instantly disprove all these conspiracy-theory nonsense claims that these devices are recording your every word.
- zdkl 8y ago
  There's a widespread sentiment that current evidence of compliance to "doing right by users" should be viewed with circumspection. And it's fair to say Google's past behaviour raises doubts about the level of trust users should extend them. What is a conspiracy theory about today's hardware, I have no trouble imagining is a planned or at least considered future iteration of their "service".
- stronglikedan 8y ago
  > Packet sniffing and hardware inspection both instantly disprove... I'm under the impression that packet sniffing is useless with end-to-end encryption, but I could be wrong. I.e., you can tell that something is being sent, but you can't know what.
  - Arelius 8y ago
    The theory is that we can still estimate how much data goes over the network. So if it were sending all audio to the cloud, we'd see. It does however not exclude other information, like sending keyword flags, or storing audio fragments to send along with other messages later on.
    - dcbadacd 8y ago
      Neither does it exclude the possibility of any "time-bomb"-type of features or future OTA updates altering the software's behavior.
  - LeftTurnSignal 8y ago
    If OP can show what they use to packet sniff those boxes, then life will be good and we can put the conspiracy theories to bed. If they can't see all of the traffic, but just know where the traffic is going, then I don't think they did their homework.
  - UncleMeat 8y ago
    You own the client. You can do anything to it. There is no way for encryption on the client to prevent you from inspecting the content.
- falcolas 8y ago
  How about the issue where Google’s devices were errantly recording everything due to a hardware issue - where the button override for the voice activation was stuck in the activated position. People who think that there’s no way that Google and Amazon could be recording everything need to realize that this is also not true. Most of these “limitations” are software enforced, and that software is updated constantly.
  - sailfast 8y ago
    This is the main problem that I see. Sure, I tested the packets, sniffed them, made sure it wasn't recording, etc, but then they push an update the next day. I don't think it's practical to monitor these devices all the time, and I haven't been asked to opt-in to an Echo update. I also don't necessarily assume mal-intent on the part of the companies, but that doesn't mean there won't _ever_ be that intent. Trusting that all of these assumptions hold over time is hard.
    - falcolas 8y ago
      > Trusting that all of these assumptions hold over time is hard. This is the big point. You are not only trusting that the company as it is today is doing the right thing, but that the company will continue to do the right thing for so long as the device is in your house - and that they will do the right thing in perpetuity with your data (including if/when they sell the company down the road).
    - mehrdadn 8y ago
      > This is the main problem that I see. Sure, I tested the packets, sniffed them, made sure it wasn't recording, etc, but then they push an update the next day. I don't think it's practical to monitor these devices all the time, and I haven't been asked to opt-in to an Echo update. This is a problem with forced updates in general (I'm also thinking of Windows, Chrome, Chrome extensions, etc. here) that security experts seem completely blind to. That said, note that even if the software didn't update, it doesn't mean it would have to send bad packets when you're actually observing. It could randomly start doing that once in a while after a few months.
    - 8y ago
- Arelius 8y ago
  Citation strongly needed. I do understand that the wake word processing happens in a special kernel in a low power state, however.... The wake word is a trained kernel, it can be trained to listen for a huge set of things (as seen in the Pixel's passive song detection), so they would just train the kernel to detect a large targeted (marketing?) vocab. about being "lit up" constantly? I'm not saying you are wrong, but I'd really like to see a citation that this is true. Is it true for both echo and home? And while Packet Sniffing can disprove that it's listening and sending whole audio to the cloud, it can't disprove that it's listening for a huge set of "wake words" and toggling bits in other control messages to track users in more subtle ways.
- teacpde 8y ago
  > If they ended up maybe changing the wakeword to "the", then maybe they could "silently" listen to everything, but that would be caught pretty quick because the device would be "lit up" constantly (another _hardware_ thing) From pure technical perspective, can the device not be programmed to be waked by wakewaord “the” with the light off?
  - darkpuma 8y ago
    > "can the device not be programmed to be waked by wakewaord “the” with the light off?" It absolutely can be.
- mondoshawan 8y ago
  Those two separate control boards didn't stop my Amazon dot from acually recording ambient noise and uploading it to Amazon's systems. I know this because of the audio history they themselves provide! You can literally go back and play back all the audio recorded, and a great deal of it did not include questions. Further, there was also a report of being able to trigger audio recording without either activating the LED ring or using a wake word via a serial root console. While a third-party attacker is unlikely to use that method of access, nothing about the hardware actively prevents Amazon ftom triggering it that way. Likewise for Google. And yes, I work on this stuff. Neither Google nor Amazon have the hardware limitations you suggest.
  - jhall1468 8y ago
    You are making an enormous amount of assumptions based on a semantic argument. Echo devices only begin recording if they think they hear the wake word. Obviously this is less than straight-forward, hence the recordings that didn't follow the wake word (just examples of an Alexa device incorrectly thinking it heard it). To suggest that a serial root console is a point of attack for an Echo device is bordering on insanity. You'd need a breakout board connected via the USB interface (not port, mind you) in order for this work-around to be effective. So yes, if a hacker had physical access to your device, time enough to solder on a breakout board, said third-party could record a variety of things. But then, it's a whole hell of a lot easier to just install a mic in someones house and get the same effect, now wouldn't it?
    - monsieurbanana 8y ago
      > To suggest that a serial root console is a point of attack for an Echo device is bordering on insanity. That was not what he said. He argues that Amazon/Google could remotely use a similar exploit (without direct access to the hardware) to start recording without lighting up the LED.
      - jhall1468 8y ago
        Nobody has EVER gotten root console access on an Echo device remotely, and the only successful "remote" exploit that didn't require soldering requires that the attacker and the victim are both on the same wifi network. Please, feel free to explain how Amazon and Google could exploit that vulnerability (that has since been patched)? More importantly, I'd love to hear how they are going to pull this off and hide it, given network traffic will be a dead give away? If what your suggesting is actually what he meant, that's even more absurd than attackers trying to do the same.
- shittyadmin 8y ago
  > the device would be "lit up" constantly (another _hardware_ thing) Unless the mic and lights are somehow wired together (they aren't), then this is really just more software which can be trivially updated away.
  - eclipxe 8y ago
    They are.
    - shittyadmin 8y ago
      How so? Ultimately the mic is always on and listening for its keywords - if you look at the teardown of the Alexa on iFixIt, I don't even see any device other than the main CPU that would be capable of performing keyword recognition. Meaning the main CPU would have to be the thing then controlling the lights after the keywords are recognized... The Google Home at least has a separate board with a microcontroller on it which could be used for keyword recognition, but I'm pretty sure they allow that to be updated for the sake of improving keyword recognition and there's no reason that an update couldn't disable the LEDs in the listen state as far as I can see.
      - a_t48 8y ago
        Not a hardware guy, but couldn’t you tie the LEDs to whatever bus that connects the mic and the main CPU?
        shittyadmin 8y ago
        Yes, I don't mean to say it's impossible - just that you'd need an entirely isolated system to detect when data was flowing over that link which is physically connected in all cases and cannot be updated. I don't believe we see that in either the Alexa or Google Home, but I'd be happy to be mistaken if anyone's done a more in depth teardown of these systems. And all of this is hinged on hoping you notice LEDs firing in the corner while you're having a conversation. Perhaps a more noticeable method should be used in cases like this. A forced "beep"/tone or something from an isolated circuit hardwired to the speakers.
- glitchc 8y ago
  These are software companies. These devices support OTA updates, including changing the activation word. It's trivial to change the activation word to empty or something innocuous. Ergo, there's no hardware limitation. If your argument stems from "Google and Amazon would never do that," I do not trust any corporate entity to value my rights more than their ability to make a dollar.
- rsync 8y ago
  "The device is built in a way that requires a wake word before any recording can possibly happen, thanks to it being built with 2 separate control boards." This is a very technically naive interpretation of their hardware/software solution. If the wake word were hard-coded into silicon then perhaps I would be charitable about your misunderstanding(s) - but of course it is not. The wake word is user-definable and can be changed to arbitrary sounds at any time. Whatever hardware limitation(s) may exist are trivially worked around with software, which can be updated over the top of you at any time.
- withinrafael 8y ago
  > Because it's a literal hardware limitation. The device is built in a way that requires a wake word before any recording can possibly happen [...] Take note that Amazon Drop In [1] is a feature built around turning on the Echo mic remotely without a wake word. I don't think this feature could exist if there was a hardware limitation. [1] https://www.amazon.com/gp/help/customer/display.html?nodeId=202153130 https://www.amazon.com/gp/help/customer/display.html?nodeId=...
  - ehsankia 8y ago
    But doesn't the device light up when that happens?
    - withinrafael 8y ago
      I was just offering a counter to the hardware limitation claim. It's possible the device makes itself known during use, I haven't used this feature yet.
    - jedieaston 8y ago
      And it beeps, and the audio from the other end starts coming through the echo's speaker. There is just about no way to know someone dropped in on you.
      - smnrchrds 8y ago
        But is this behaviour implemented in hardware or software?
        withinrafael 8y ago
        This feature is configured via its software.
        ehsankia 8y ago
        My understanding is that the light at the very least was hardware. Sound can be disabled.
- tomn 8y ago
  Do you (or anyone else with the same claim) have a citation for this? I've spent some time reverse-engineering the echo microphone board, and while there is an interlock that prevents recording while the red mute button is lit (just the light under the button, not the ring), I didn't see anything that would prevent recording while the ring light was off.
- trhway 8y ago
  > Packet sniffing The unauthorized recorded data can be sent encoded with/into the authorized data.
- LinuxBender 8y ago
  > Because it's a literal hardware limitation. Unless there is a separate out of band board with a relay I can hear or see (meaning, code alone can't enable something), then it really isn't a hardware limitation. The security controls and operations are in the code. The code can change or may already have silent monitoring capabilities. Nobody on HN could really answer whether or not this is the case. All we can do is speculate. If someone were required to put lawful monitoring code in place, they would not be allowed to discuss it here. The best anyone could do is decompile the code or get the source code for the firmware. Even then, there could be non-volital space that allows for updates. Case in point, there have been malware packages that could enable your microphone and camera on the laptop without turning on the LED. This varied with camera model. Some power the LED when the camera has power. Microphones don't always activate an LED. There are a myriad of articles you can find providing examples of malware that can listen to cell phone microphones, laptop microphones without activating the LED.
- darkerside 8y ago
  I get that you believe this, and I even understand you repeating it to other people on the Internet. What I don't get is that your tone indicates that you are offended people don't believe what you believe... which also just happens to have been incorrect in the past and many others seem to think is provably possible in the future.
- TeMPOraL 8y ago
  > Seriously, a lot of people on HN need to do their damn homework about these devices before declaring them to be something they have been proven not to be. Based on what? Marketing copy? Eyeballing iFixit teardowns? > but that would be caught pretty quick because the device would be "lit up" constantly (another _hardware_ thing) Totally not buying it, unless you can show me the traces and discrete components that force power through the LED when signal from the microphone is allowed to reach the uC. If it's done in software, I'm not trusting it. If you're making an argument of "trust the vendor because economics", you have to recognize how weak it is.
- amatecha 8y ago
  > Because it's a literal hardware limitation. Citation needed. Further, listening for a wake word and reacting to that is likely done completely in software: the fact it's even listening for a "wake word" means the hardware (microphone) is in fact always listening, it's just [presumably] not actually sending that audio to The Cloud (tm).
- tokyodude 8y ago
  I don't own either of them but Siri and Google on my phone both require training when I first use them. Do these devices not? IF they do then isn't that proof they are re-programmable and could be programmed to respond to anything?
- acct1771 8y ago
  Good luck packet sniffing an encrypted text blob of your conversations the device is transcribing.
wrs 8y ago
>How do you know? Because that would be pointless. The real problem right now is false triggering. If you’re actually worried about being spied on, why on earth would you have one of these in the first place?
deleted 8y ago
[deleted]
pwython 8y ago
Not that this helps anyone sleep easier, but imagine in today's age... a whistleblower -- perhaps one of the thousands of software devs working on one of these -- leaked proof that these devices are recording everything to re-market and profit, without permission... The resulting backlash and legal ramifications would be so huge it just wouldn't be worth it. It wouldn't just take an insane and stupid CEO to do that, but also thousands of other tech/adops employees who'd have to be like, "yea this is a great idea."
- sp332 8y ago
  I just kinda doubt this. How much backlash was there when it came out that the NSA was recording the full content of every cell phone call in the Bahamas? Edit: codename SOMALGET, subproject of MYSTIC. https://en.wikipedia.org/wiki/MYSTIC_(surveillance_program)#Targets https://en.wikipedia.org/wiki/MYSTIC_(surveillance_program)#... http://www.documentcloud.org/documents/1164088-somalget.html http://www.documentcloud.org/documents/1164088-somalget.html
  - debacle 8y ago
    > was recording the full content of every cell phone call in the Bahamas? The what now?
    - 0xADEADBEE 8y ago
      I didn't realise this either but it looks pretty widely known: https://theintercept.com/2014/05/19/data-pirates-caribbean-nsa-recording-every-cell-phone-call-bahamas/ https://theintercept.com/2014/05/19/data-pirates-caribbean-n...
  - archgoon 8y ago
    Why would there be? The Bahamas is not inside the United States and thus is part of the NSA's mission of monitoring foreign communications. Spy agencies spy. It's their job description.
    - deleted 8y ago
pacala 8y ago
"Allow Google Maps to always access your location. Yes. Ask Me Later". Given the multiple precedents on the erosion of privacy path in the past 20 years, of which I quoted one example above, it's pretty obvious that they will turn "always on listening" in the future, using whatever dark patterns necessary to avoid a class action suit.
- paulie_a 8y ago
  I am getting sick of yes, ask me later. Stop asking me completely

mohaine 8y ago

Actually, it doubles your area of risk. Now you have 2 companies to worry about per device.

guelo 8y ago
It's an open source project. There's no company to trust.
- kawfey 8y ago
  Users without the skills to verify the code isn't nefarious have to trust good samaritan developers instead.
  - spion 8y ago
    Nothing is 100% guaranteed, but with an open source project, given enough users, its far less likely for someone to be able to bury nefarious stuff without many eyes looking at it and at least one person sounding an alert.
    - dymk 8y ago
      Yeah but really this isn't true. Popular open source that has tens of thousands of eyes on it still gets compromised all the time (see: npm). Even the Linux kernel has had rogue git commits injected into it.
      - calcifer 8y ago
        > the Linux kernel has had rogue git commits injected into it What? Who "injected" what and when?
        spacedog11 8y ago
        I'm also interested to know more about this.
      - 8y ago

floatingatoll 8y ago

The sentiment this is countering is not "Google and Amazon _are_ recording 24/7/365". It is countering "Amazon and Google _would_ record 24/7/365 if they could get away with it socially". Arguing that the technology doesn't do this does not address the underlying root perception that Amazon and Google are not to be trusted. Google built its entire business on harvesting _all_ data on the web and building an infrastructure to process it efficiently. Purchasing Nest made clear that to us that they now wish to harvest data from the home. If they were able, socially and politically, to harvest _all_ data from the home, their history indicates that they would do so with all possible speed. Having a device you built yourself that learns locally and is under your control prevents Amazon and Google from changing their mind about what level of recording is acceptable without your informed opt-in consent at the time of the change. Both providers reserve the right to update their privacy policies without notification or consent, including granting themselves to increase data collection. TLDR: This device is the physical expression of mistrust of Amazon and Google. Their greed for your metadata is well-documented, and their policies let them increase data collection in your home at any time without your consent. Such increases would be prevented by this device.

Aunche 8y ago

Also, it would make no business sense to always be listening. A lot of people think that just because Alexa and Google Assistant are free to use, it means that these services are virtually free for the companies as well, but that's not the case. There is no way Amazon or Google would waste millions of dollars running the state of the art speech recognition algorithms on your house's background noise.

reaperducer 8y ago
There is no way Amazon or Google would waste millions of dollars running the state of the art speech recognition algorithms on your house's background noise. Unless they wanted to listen for a dog barking, then add "pet food buyer" to your profile data. There are a lot of very easy use cases for monitoring background noise.

guelo 8y ago

From a non-privacy perspective this adds the feature of being able to customize your wake word, which besides being a nice feature on its own also counters Google and Amazon's desire to inject their brands deeper into our psyche by making us say them out loud. From a privacy perspective, having user control of the wake word prevents Google and Amazon from adding future wake words that could be abused for other ways to track us. For example, Google might get the bright idea to track TV ads by listening for audio in the ads. Or tracking people in your house by making Android phones emmit non-audible chirps. These kind of "features" could be easily introduced at any point in the future by an update to privacy policies that nobody notices.

reaperducer 8y ago
Google might get the bright idea to track TV ads by listening for audio in the ads Considering that some TVs have this built-in, I'd be surprised if Google wasn't already doing the same. It's why my "smart" TV isn't allowed to connect to my wifi network. (There was a previous HN article about it, I believe the brand was Samsung.)
- thinkingemote 8y ago
  I think it was the Facebook app, it listened to ambient background to determine music and TV shows. The user had to opt in or at least approve permissions.

crankylinuxuser 8y ago

> While I appreciate the sentiment...unless you actually think Google and Amazon devices are recording irrelevant ambient sound deliberately (they aren’t) Citation needed. Or if you have reference firmware I can load, that would be awesome... What do you mean its closed firmware and controlled by Amazon/Google? You mean they can change it whenever they wish, and we have no say??? Long story short; you rented a spy device and you trust some random person online it isn't spying... Even though there are credible stories of these devices doing precisely that. No, no, absolutely not, and hell no!

almost_usual 8y ago

Who cares. Whoever buys the hardware can do whatever they want with it. If someone feels better with this device on their Google / Alexa product let them do it. If the speech recognition is horrible I’m sure they’ll take it off.

aylmao 8y ago
+1 to who cares. > If the speech recognition is horrible I’m sure they’ll take it off. This is interesting and cool, cause it sounds to me like they only detect for the keyword to "unlock" the Google home, so in theory they don't even need speech recognition. In most cases, they could just do with telling if a sound you make seems to match the sound you defined to be their name ¯\_(ツ)_/¯

kakarot 8y ago

The point is establishing as much trust as you can with your devices. Only having external points of trust when absolutely necessary. This is generally a Good Thing, and something we should do by default, not as a reaction to some corporate leak.

aylmao 8y ago

> It’s very unlikely because Amazon and Google pay for false positives, so they have a strong incentive to develop really good trigger word detection. There's a feature in my Pixel to show what song is playing --like in the real world-- on the lock screen. A kind of always-on Shazam. They don't mind paying for always on.

CraftThatBlock 8y ago
That's fully on-device though, it keeps a database of the top songs fingerprints and doesn't use the network to recognize songs
parineum 8y ago
"Now Playing" is an offline feature https://venturebeat.com/2017/10/19/how-googles-pixel-2-now-playing-song-identification-works/ https://venturebeat.com/2017/10/19/how-googles-pixel-2-now-p...
- aylmao 8y ago
  Go figure, I stand corrected.
deleted 8y ago
[deleted]

freeflight 8y ago

Afaik by now, there have been least two court cases where Echo recordings were handed over as evidence [0] [1]. At this rate, it's only a matter of time before evidence like that gets leaked/released, which would serve as a good probe on how much these devices really record. [0] https://techcrunch.com/2018/11/14/amazon-echo-recordings-judge-murder-case/ https://techcrunch.com/2018/11/14/amazon-echo-recordings-jud... [1] https://edition.cnn.com/2017/03/07/tech/amazon-echo-alexa-bentonville-arkansas-murder-case/index.html https://edition.cnn.com/2017/03/07/tech/amazon-echo-alexa-be...

onion2k 8y ago

Edit: It’s very unlikely because Amazon and Google pay for false positives, so they have a strong incentive to develop really good trigger word detection. If the false positives give them data that increases the value of your marketing profile by more than the cost of processing the interactions then they actively profit from false positives, and have no reason at all to stop them.