5 ms·

In the end, we don't know what Alexa is listening to. If we do dead-listing style analysis with teardowns and eeprom dumps, it can only tell us what it was lis

In the end, we don't know what Alexa is listening to. If we do dead-listing style analysis with teardowns and eeprom dumps, it can only tell us what it was listening to at that time.. and sometimes not even that. How do we know what [.0076162328 3.27617819 91817.111121123 -65.2129] is listening to, especially when combined with complicated neural network weights? We can only check and verify every possible input, and that's effectively infeasible. We chose not to buy these types of devices for our house, because of the great deal of unknowns.

nathanaldensr 9y ago
Exactly. Wouldn't we have to have access to all the relevant source code, both for the devices and the server-side components, to tell what Alexa and other home assistant devices are actually doing and storing?
- eckza 9y ago
  And rolling hashes of every build. And a device to constantly cross-check the hashes. And a device to constantly verify that the cross-checker was working. And a workflow to show diffs of each commit. And time to read through them, grok them, and sandbox-test that they weren't doing anything nasty in an obfuscated way. At the end of the day, you really can't trust anything completely, and the only real solution is to 1) trust Amazon (which doesn't seem like a great bet right now), or 2) don't own one.
  - GhostVII 9y ago
    Or 3) just don't worry about it
    - macintux 9y ago
      Isn't 3 the same as 1?
majormajor 9y ago
I considered the "don't buy" strategy, but I'm not willing to go all the way to the "don't buy a phone" point, where we've got the same situation, which leaves me with little other choices. :|
- baxtr 9y ago
  Siri can be turned off on iPhones. I don’t see how buying a phone gets you in the same situation as buying a device that has a voice-only interface.
  - jasonkostempski 9y ago
    Just because some UI element labeled "Enable Siri" can be toggled doesn't mean it works exactly as you, or even Apple devs, expect it to. And even phones without digital assitant technology can be turned into remote listening devices.
    - majormajor 9y ago
      Yeah, exactly. You can tell the software to disable the feature, but the hardware is still wired up, and it could still be doing everything but the user-facing bits, or a future update could flip that on even if it's not presently doing it. I have an Android phone and I trust Google somewhat less in terms of personal-info-collecting than Amazon. :|
      - jasonkostempski 9y ago
        I have an Android too but I lost all trust in Google when I realized the default keyboard sends my keystrokes to Google by default. And the number of times ive had to re-disable highly undesired defaults after an update is infuriating.
    - baxtr 9y ago
      That is of course correct from a technical perspective, but not from a trust perspective. I can’t remember that Apple has done anything like that on purpose. Sure, devs get wild, people hack, but I’d never expect them to do this systematically. With amazon, I’m not so sure. But maybe that’s just me
r00fus 9y ago
That's my big concern as well. Hidden trigger phrases and self-incrimination issues (i.e., valid trigger phrase + sending potentially incriminating speech) - either separately (or worse, combined) are something I'd rather not worry about in my own home.
- djsumdog 9y ago
  I'm surprised we haven't seen any warrants from police/FBI that would require wire tapping using one of these devices.
  - CharlesW 9y ago
    https://www.nbcnews.com/tech/internet/prosecutors-get-warrant-amazon-echo-data-arkansas-murder-case-n700776 https://www.nbcnews.com/tech/internet/prosecutors-get-warran...
  - marnett 9y ago
    Not for wire tapping, but just in case you weren't aware, there was a case where law enforcement wanted all recordings from an Alexa in a homicide case[1] https://www.cnn.com/2017/03/07/tech/amazon-echo-alexa-bentonville-arkansas-murder-case/index.html https://www.cnn.com/2017/03/07/tech/amazon-echo-alexa-benton...
  - IntronExon 9y ago
    http://5newsonline.com/2016/12/28/bentonville-police-use-smart-water-meters-as-evidence-in-murder-investigation/ http://5newsonline.com/2016/12/28/bentonville-police-use-sma...
- crankylinuxuser 9y ago
  I've played around with microphone arrays with a Kinect and http://hark.jp http://hark.jp and it's amazing what you can do even with 4 microphones. ----------------- I can figure out where people are talking spatially. I can figure out how people are moving around when speaking. I can voiceprint people. I can cancel voices individually, or mute all noise past a certain distance from the center of the mic array. I can 3d map rooms by recording over time how people talk and how they move. I can connect multiple nodes together to create wider areas of coverage of above. I can convert all sounds to text if appropriate. I can send tiny messages that include voiceprint, position, and content. I can tell how many people are in your house at any one time. I can tell if you have friends over, and when. If they have similar system, can collaborate backend data across all of above ------------ So yeah, there's plenty of things people aren't thinking of. I doubt they even have the experience to know how possible it is with open source tooling and cheap hardware. But all of these things are possible, and my guess is being done. The distinction with phones is hard. Apple has stayed within the "we support privacy" aspect. And with things like limeSDR being cheap and common, wouldn't be as hard to attack and prove. Android is tricky, but with some of the anti-crapware systems and basebuild efforts going on, can be considered possibly trustworthy (if edited). And in general, phones move a lot. Amazon tunacan/tomatocan doesn't. Just add a $1 accelerometer, and detection of movement is trivial. Phones can have this, but doing the above would be a magnitude harder.
Digital-Citizen 9y ago
You have good reason not to trust it or any device like it; proprietary software (which is always untrustworthy) + mic and/or camera + Internet connection equals spy device that doesn't respect your software freedom, pure and simple. You deserve to have as much control over your computers as you wish and no device like that will respect your privacy. The alleged convenience is highly overrated (everyone who has one has lived most of their lives without one and apparently been able to do just fine without) and, more importantly than any alleged convenience, not a good trade-off in light of paying with your freedom and privacy. But the same can be said of the tracker devices residing in many pockets. Some call it a cell phone or mobile phone, but its primary job and the job it does most of the time is to geolocate or track itself. And a tracker's mic has no indicator when it's listening, or a way for the user to vet the captured data, or control where the captured data goes. Those trackers that also have cameras are even worse in this regard. And they all run non-free, user-subjugating, proprietary software. So none of them respect a user's software freedom or privacy. I suspect that those with user-unmodifiable batteries aren't truly ever off except for when all of the batteries run down. But most users never turn their trackers off anyhow, so that's a bit of a moot point. What other freedoms can proprietors convince you to do away with in exchange for some minor convenience or possibly a little innumerate fear-baiting (such as "you'll want this tracker in the case of an emergency!").
- deleted 9y ago
  [deleted]
secabeen 9y ago
We don't know what alexa is listening to, but we do know that the data it transmits to the cloud is roughly proportional to the amount of time that you are talking to it. https://www.iot-tests.org/2017/06/careless-whisper-does-amazon-echo-send-data-in-silent-mode/ https://www.iot-tests.org/2017/06/careless-whisper-does-amaz...

nathanaldensr 9y ago

Exactly. Wouldn't we have to have access to all the relevant source code, both for the devices and the server-side components, to tell what Alexa and other home assistant devices are actually doing and storing?

eckza 9y ago
And rolling hashes of every build. And a device to constantly cross-check the hashes. And a device to constantly verify that the cross-checker was working. And a workflow to show diffs of each commit. And time to read through them, grok them, and sandbox-test that they weren't doing anything nasty in an obfuscated way. At the end of the day, you really can't trust anything completely, and the only real solution is to 1) trust Amazon (which doesn't seem like a great bet right now), or 2) don't own one.
- GhostVII 9y ago
  Or 3) just don't worry about it
  - macintux 9y ago
    Isn't 3 the same as 1?

majormajor 9y ago

I considered the "don't buy" strategy, but I'm not willing to go all the way to the "don't buy a phone" point, where we've got the same situation, which leaves me with little other choices. :|

baxtr 9y ago
Siri can be turned off on iPhones. I don’t see how buying a phone gets you in the same situation as buying a device that has a voice-only interface.
- jasonkostempski 9y ago
  Just because some UI element labeled "Enable Siri" can be toggled doesn't mean it works exactly as you, or even Apple devs, expect it to. And even phones without digital assitant technology can be turned into remote listening devices.
  - majormajor 9y ago
    Yeah, exactly. You can tell the software to disable the feature, but the hardware is still wired up, and it could still be doing everything but the user-facing bits, or a future update could flip that on even if it's not presently doing it. I have an Android phone and I trust Google somewhat less in terms of personal-info-collecting than Amazon. :|
    - jasonkostempski 9y ago
      I have an Android too but I lost all trust in Google when I realized the default keyboard sends my keystrokes to Google by default. And the number of times ive had to re-disable highly undesired defaults after an update is infuriating.
  - baxtr 9y ago
    That is of course correct from a technical perspective, but not from a trust perspective. I can’t remember that Apple has done anything like that on purpose. Sure, devs get wild, people hack, but I’d never expect them to do this systematically. With amazon, I’m not so sure. But maybe that’s just me

r00fus 9y ago

That's my big concern as well. Hidden trigger phrases and self-incrimination issues (i.e., valid trigger phrase + sending potentially incriminating speech) - either separately (or worse, combined) are something I'd rather not worry about in my own home.

djsumdog 9y ago
I'm surprised we haven't seen any warrants from police/FBI that would require wire tapping using one of these devices.
- CharlesW 9y ago
  https://www.nbcnews.com/tech/internet/prosecutors-get-warrant-amazon-echo-data-arkansas-murder-case-n700776 https://www.nbcnews.com/tech/internet/prosecutors-get-warran...
- marnett 9y ago
  Not for wire tapping, but just in case you weren't aware, there was a case where law enforcement wanted all recordings from an Alexa in a homicide case[1] https://www.cnn.com/2017/03/07/tech/amazon-echo-alexa-bentonville-arkansas-murder-case/index.html https://www.cnn.com/2017/03/07/tech/amazon-echo-alexa-benton...
- IntronExon 9y ago
  http://5newsonline.com/2016/12/28/bentonville-police-use-smart-water-meters-as-evidence-in-murder-investigation/ http://5newsonline.com/2016/12/28/bentonville-police-use-sma...
crankylinuxuser 9y ago
I've played around with microphone arrays with a Kinect and http://hark.jp http://hark.jp and it's amazing what you can do even with 4 microphones. ----------------- I can figure out where people are talking spatially. I can figure out how people are moving around when speaking. I can voiceprint people. I can cancel voices individually, or mute all noise past a certain distance from the center of the mic array. I can 3d map rooms by recording over time how people talk and how they move. I can connect multiple nodes together to create wider areas of coverage of above. I can convert all sounds to text if appropriate. I can send tiny messages that include voiceprint, position, and content. I can tell how many people are in your house at any one time. I can tell if you have friends over, and when. If they have similar system, can collaborate backend data across all of above ------------ So yeah, there's plenty of things people aren't thinking of. I doubt they even have the experience to know how possible it is with open source tooling and cheap hardware. But all of these things are possible, and my guess is being done. The distinction with phones is hard. Apple has stayed within the "we support privacy" aspect. And with things like limeSDR being cheap and common, wouldn't be as hard to attack and prove. Android is tricky, but with some of the anti-crapware systems and basebuild efforts going on, can be considered possibly trustworthy (if edited). And in general, phones move a lot. Amazon tunacan/tomatocan doesn't. Just add a $1 accelerometer, and detection of movement is trivial. Phones can have this, but doing the above would be a magnitude harder.

Digital-Citizen 9y ago

You have good reason not to trust it or any device like it; proprietary software (which is always untrustworthy) + mic and/or camera + Internet connection equals spy device that doesn't respect your software freedom, pure and simple. You deserve to have as much control over your computers as you wish and no device like that will respect your privacy. The alleged convenience is highly overrated (everyone who has one has lived most of their lives without one and apparently been able to do just fine without) and, more importantly than any alleged convenience, not a good trade-off in light of paying with your freedom and privacy. But the same can be said of the tracker devices residing in many pockets. Some call it a cell phone or mobile phone, but its primary job and the job it does most of the time is to geolocate or track itself. And a tracker's mic has no indicator when it's listening, or a way for the user to vet the captured data, or control where the captured data goes. Those trackers that also have cameras are even worse in this regard. And they all run non-free, user-subjugating, proprietary software. So none of them respect a user's software freedom or privacy. I suspect that those with user-unmodifiable batteries aren't truly ever off except for when all of the batteries run down. But most users never turn their trackers off anyhow, so that's a bit of a moot point. What other freedoms can proprietors convince you to do away with in exchange for some minor convenience or possibly a little innumerate fear-baiting (such as "you'll want this tracker in the case of an emergency!").

deleted 9y ago
[deleted]

secabeen 9y ago

We don't know what alexa is listening to, but we do know that the data it transmits to the cloud is roughly proportional to the amount of time that you are talking to it. https://www.iot-tests.org/2017/06/careless-whisper-does-amazon-echo-send-data-in-silent-mode/ https://www.iot-tests.org/2017/06/careless-whisper-does-amaz...