3 ms·

Yeah, they're half-on listening for the wakeword, but they don't send the audio anywhere until you utter it.

by kennu 7y ago

AlexandrB 7y ago
This is not only something you largely have to trust Amazon's word on but a limitation that can be removed at any time via OTA update.
- kennu 7y ago
  I don't think that's entirely true. People have analyzed the network traffic and found it to match the wakewords. The hardware and software architecture has also been analyzed and my understanding is that the wakeword functionality is implemented by secondary low-power hardware, which turns on the main system only after hearing the wakeword. Of course, anything is possible in theory and I wouldn't be surprised to see some targeted attacks happening occasionally, but that's pretty different from massive always-on 24/7 audio collection from all consumers. Some related links: https://www.iot-tests.org/2017/06/careless-whisper-does-amazon-echo-send-data-in-silent-mode/ https://www.iot-tests.org/2017/06/careless-whisper-does-amaz... https://arxiv.org/abs/1707.08696 https://arxiv.org/abs/1707.08696
  - teucris 7y ago
    Do you think that people care about the difference between only detecting wakewords and actual continual listening? My understanding from people I’ve talked to with smart speakers in their homes is that they’ve accepted that it’s listening all the time.
    - kennu 7y ago
      I'm sure people have different feelings about it. My personal view is that you can listen to anyone using the microphones in their computers and smartphones anyway (with a targeted attack) and nobody is unhackable. At the same time I think people have become more privacy-aware in general and started to demand less bulk data collection. So they might accept certain risks of getting hacked but not necessarily submit to terms of service that allow bulk collection.
wlesieutre 7y ago
Minor but important correction - until it thinks you utter it. If it misheard you it can do bad things: https://www.theguardian.com/technology/2018/may/24/amazon-alexa-recorded-conversation https://www.theguardian.com/technology/2018/may/24/amazon-al...
SlowRobotAhead 7y ago
That’s such a short sighted cop out to a legitimate threat. When you make the argument that “no one said the keyword so it isn’t uploading!”, you aren’t considering that it’s listening for voices, your voice, your SO’s voice, how many unique voices, what time they are talking, what temperature it is, if there is a TV or music on, what station it’s on via pattern recognition, all manner of meta data that DOES NOT require a keyword or a large outgoing data stream. It could store meta data for days and dump it with a single MQTT packet which it’s sending all the time, and they’re all encrypted. We flat out have no idea what these things are doing on the data collection end that doesn’t have to do with someone saying “Alexa” first.
- kennu 7y ago
  I think you're right, I was only thinking about raw audio being uploaded to the cloud. In the future, there might be more and more local machine learning happening on the devices and the extracted metadata being uploaded unnoticed. On the other hand, I still think people can analyze the hardware and software running on the devices. If there are "black boxes" doing something unknown, we should be worried, particularly if they are always powered on and have the processing capability to analyze audio content. Still, the original claim in the article was that there is a lot of this metadata being uploaded right now. I don't think anybody has verified that claim, even if it could be true.

Yeah, they're half-on listening for the wakeword, but they don't send the audio anywhere until you utter it.

by kennu 7y ago

Yeah, they're half-on listening for the wakeword, but they don't send the audio anywhere until you utter it.

This is not only something you largely have to trust Amazon's word on but a limitation that can be removed at any time via OTA update.

kennu 7y ago
I don't think that's entirely true. People have analyzed the network traffic and found it to match the wakewords. The hardware and software architecture has also been analyzed and my understanding is that the wakeword functionality is implemented by secondary low-power hardware, which turns on the main system only after hearing the wakeword. Of course, anything is possible in theory and I wouldn't be surprised to see some targeted attacks happening occasionally, but that's pretty different from massive always-on 24/7 audio collection from all consumers. Some related links: https://www.iot-tests.org/2017/06/careless-whisper-does-amazon-echo-send-data-in-silent-mode/ https://www.iot-tests.org/2017/06/careless-whisper-does-amaz... https://arxiv.org/abs/1707.08696 https://arxiv.org/abs/1707.08696
- teucris 7y ago
  Do you think that people care about the difference between only detecting wakewords and actual continual listening? My understanding from people I’ve talked to with smart speakers in their homes is that they’ve accepted that it’s listening all the time.
  - kennu 7y ago
    I'm sure people have different feelings about it. My personal view is that you can listen to anyone using the microphones in their computers and smartphones anyway (with a targeted attack) and nobody is unhackable. At the same time I think people have become more privacy-aware in general and started to demand less bulk data collection. So they might accept certain risks of getting hacked but not necessarily submit to terms of service that allow bulk collection.

wlesieutre 7y ago

Minor but important correction - until it thinks you utter it. If it misheard you it can do bad things: https://www.theguardian.com/technology/2018/may/24/amazon-alexa-recorded-conversation https://www.theguardian.com/technology/2018/may/24/amazon-al...

SlowRobotAhead 7y ago

That’s such a short sighted cop out to a legitimate threat. When you make the argument that “no one said the keyword so it isn’t uploading!”, you aren’t considering that it’s listening for voices, your voice, your SO’s voice, how many unique voices, what time they are talking, what temperature it is, if there is a TV or music on, what station it’s on via pattern recognition, all manner of meta data that DOES NOT require a keyword or a large outgoing data stream. It could store meta data for days and dump it with a single MQTT packet which it’s sending all the time, and they’re all encrypted. We flat out have no idea what these things are doing on the data collection end that doesn’t have to do with someone saying “Alexa” first.

kennu 7y ago
I think you're right, I was only thinking about raw audio being uploaded to the cloud. In the future, there might be more and more local machine learning happening on the devices and the extracted metadata being uploaded unnoticed. On the other hand, I still think people can analyze the hardware and software running on the devices. If there are "black boxes" doing something unknown, we should be worried, particularly if they are always powered on and have the processing capability to analyze audio content. Still, the original claim in the article was that there is a lot of this metadata being uploaded right now. I don't think anybody has verified that claim, even if it could be true.