20 ms·
Amazon Sends 1,700 Alexa Voice Recordings to a Random Person
- dixie_land 8y agoDoes Amazon keep the raw recordings for GDPR compliance reasons? It seems to me keeping them just creates liability? (Yes data analysis for targeting and such would be valuable, but those can be done with, say transcripts instead of the raw recordings)
- samangan 8y agoThe actual recordings are the most valuable data for modeling purposes so I actually assume they try to store it to improve the system
- dixie_land 8y agothat makes sense. though I wonder if for training purposes they can simply remove the user identifier. (like anonymizing them for instance)
- nolok 8y agoNothing in the article suggests they were linked to any kind of user identifier, all we know is that each recording is linked to the transcription alexa made of it (which makes sense, training and log wise). The user were easy to identify because if I listen to the last twenty queries you made to alexa between what you ask, what other people say in the background, what you talk about and what noise is in the background that gives me a lot of information; that's exactly how they were able to identify some of the users from their recordings.
- dixie_land 8y ago> Nothing in the article suggests they were linked to any kind of user identifier They were able to provide the customer a bundle of his recordings upon request so they must've maintained such a lookup mapping. Of course that led to the mix up reported here. But I'd argue it's safer if on Amazons side they simply can't fulfill such request themselves due to anonymization.
- closetohome 8y agoI suppose that does raise a question of where you draw the line on a company's obligation to anonymize information. Is disassociating a user ID from the data enough? What about data that makes the user identifiable through patterns? If I say "Alexa, my name is Bob Jones and I have chlamydia," they're not really helping me out by just not associating that audio clip with my username.
- fiftyacorn 8y agoDoesn't using live data for testing create GDPR issues?
- thecatspaw 8y agoI would assume that if they made a voice print for you personally it would be fine. It should be still recognized as your own personal data, and underly the same laws
- jonas21 8y agoI'd imagine that a competent company would keep access to user audio recordings tightly locked down, either because they actually care about privacy or because they care about the bad PR of recordings getting leaked. But if the GDPR requires them to send the recordings to users upon request, they need to have a way to do that -- either by exposing a service on the web, or by providing access to customer service employees to handle the request. Not sure how I feel about that.
- roywiggins 8y agoThe Alexa app lets you play back your own recordings.
- SlowRobotAhead 8y ago>"Using these files, it was fairly easy to identify the person involved and his female companion; weather queries, first names, and even someone’s last name enabled us to quickly zero in on his circle of friends,” according to the report. “Public data from Facebook and Twitter rounded out the picture.” Please keep in mind the next time you or a friend says "well, what do I care if Amazon knows when I ask to turn the heat up". I read about a cool Echo Dot hardware hack where it was made into an old phone with mute on the speaker and microphone until picked up. As a tech people it boggles my mind that other tech people accept an always on microphone in their homes. I truly don't get it!
- sciurus 8y agoSpeaking of phones, aren't most people's smartphones already "an always on microphone in their homes"?
- berbec 8y agoand gps and video camera and still camera and network sniffer
- tjoff 8y agoOn the vast majority of devices it is not always on. It is not a nitpick. Accidentally recording or even intentionally recording is many orders of magnitude simpler when the device is supposed to always be listening. The fact that a single employee even could make the mistake mentioned in the articles is one example of why. Eventhough it also really does hint that the backend isn't up to par at amazon.
- dlubarov 8y ago> On the vast majority of devices it is not always on. Would you consider the device "listening" if a DSP is listening for a particular hotword, but no audio data is being uploaded? I'm pretty sure that's what happens in either case, whether you're using Alexa, Siri, or Google Assistant, and whether it's a "home" device or a phone with hotword detection. I used to work at Google on some related hardware projects. If you're suspicious because the home devices seem much more accurate, that's because they have microphone arrays which can facilitate beamforming.
- danso 8y agoI don’t understand how this problem of “human error” happened at a company like Amazon/AWS. Perhaps requests for Alexa info are rare enough that the internal interface for servicing such requests isn’t fully automated. But I’d be shocked if the process involved someone (low-level data entry person, or engineer) manually typing in a customer ID number.
- testplzignore 8y agoThere is obviously more to this story than Amazon is telling. https://www.heise.de/downloads/18/2/5/6/5/3/9/6/ct.0119.016-018_engl.pdf https://www.heise.de/downloads/18/2/5/6/5/3/9/6/ct.0119.016-... says that this was a "one-time error" and that "Amazon also claimed that they had discovered the error themselves". It is highly unlikely that either of these are true. The one time they made this error just happens to involve someone who is savvy enough to contact the right journalists to investigate it? And Amazon coincidently discovered the error themselves after being contacted by c't? Amazon is flat out lying. If there is ever a case for the max GDPR fine to be imposed, this is it.
- madrox 8y agoI wouldn't be shocked at all. Internal tools are never as streamlined as you would think. On top of that, the tools for doing this are probably not that mature given that GDPR hasn't been around that long, nor do a majority of users take advantage of it.
- bduerst 8y agoDoes this qualify as a GDPR violation?
- spullara 8y agoThe irony is that they were answering a GDPR request.
- whoisjuan 8y agoVery unlikely. In fact, the error came from an actual GDPR Right of Access request. If they inform the affected person and take measures to prevent it from happening again, then they're in compliance of GDPR. GDPR was created to prevent organizations from hiding data breaches, trading or transfering customer information, avoiding privacy compliance, etc. This is making the news because is Amazon, but you will be surprised how many of these breaches happen in a daily basis in all types of organizations.
- qznc 8y agoThe first GDPR fine was against knuddels.de because they accidentally stored passwords in plain text [0]: “By storing the passwords in clear text, the company knowingly violated its duty to ensure data security in the processing of personal data in accordance with GDPR Article 32(1)(a),” [0] https://www.welivesecurity.com/2018/11/27/german-chat-site-faces-fine-gdpr/ https://www.welivesecurity.com/2018/11/27/german-chat-site-f...
- accnumnplus1 8y agoThis is just facebook all over again. Why can't people see trouble-tech? I never used facebook and won't ever use an alexa, nor whatever google's thing is.
- Zelphyr 8y agoI don’t know why this is getting downvoted. I feel the same way. Amazon and Google have yet to demonstrate to me that they value my privacy enough to take necessary steps to protect it. They, like Facebook, know most people foolishly don’t care enough to hold these companies feet to the fire on it. At the risk of stating the obvious; Facebook has more than demonstrated the opposite.
- lykr0n 8y agoThe only company I've seen take demonstrable action to protect their customer's privacy is Apple. Google, Facebook, Microsoft, Amazon have never given me a single reason to trust that they have my privacy concerns on their mind.
- tfha 8y agoBut with Apple's entire ecosystem being closed and intentionally locking you in, is Apple really a better choice. All it takes is one decision from the board that it would be more profitable to start violating privacy. The board's legal mandate is to make as much money as possible. As soon as privacy is no longer a fad that makes more money and growth than the opportunity cost to not violating it, Apple will flip a switch. That's their duty as a for profit company.
- ceejayoz 8y ago> The board's legal mandate is to make as much money as possible. Why do people keep repeating this falsehood? https://www.nytimes.com/roomfordebate/2015/04/16/what-are-corporations-obligations-to-shareholders/corporations-dont-have-to-maximize-profits https://www.nytimes.com/roomfordebate/2015/04/16/what-are-co... Per SCOTUS in US v. Hobby Lobby: > While it is certainly true that a central objective of for-profit corporations is to make money, modern corporate law does not require for-profit corporations to pursue profit at the expense of everything else, and many do not do so. For-profit corporations, with ownership approval, support a wide variety of charitable causes, and it is not at all uncommon for such corporations to further humanitarian and other altruistic objectives. Many examples come readily to mind. So long as its owners agree, a for-profit corporation may take costly pollution-control and energy-conservation measures that go beyond what the law requires. A for-profit corporation that operates facilities in other countries may exceed the requirements of local law regarding working conditions and benefits. If for-profit corporations may pursue such worthy objectives, there is no apparent reason why they may not further religious objectives as well.
- miles 8y agoNot the first time: Amazon Echo records private conversation and sends to co-worker https://www.youtube.com/watch?v=IQrv5GbOKhc https://www.youtube.com/watch?v=IQrv5GbOKhc
- samplesize 8y agoUser who doesn't understand how device works accidentally sends voice message to coworker.
- ndnxhs 8y agoThe original article stated that they do know how it works but the device malfunctioned and detected a command where there was none without the owner even being aware that it was activated.
- danso 8y agoIIRC, that was a case of someone inadvertently activating Alexa, and being misinterpreted. This is a completely different situation than what’s discussed in the posted article, which implies that an Amazon employee sent someone another user’s Alexa data.
- rdtsc 8y agoI am baffled by how many people have no problems adding always listening and recording Google and Facebook devices to their living rooms. Some are considered tech people who should know stuff like this could happen, but they just don't care or don't think it will happen to them. Somewhere there is an ex-Stasi officer who is thinking "I wish we were that good and had people fooled to voluntarily install listening devices and didn't have to tap phone lines and crawl around basements and rooftops".
- lykr0n 8y agoThis. You're trusting everything you say & do to a company who says they're not going to share your data with others and that have the proper control in place to prevent other people from getting it.
- carapace 8y agoYeah, it's like volunteering to become a serf.
- isostatic 8y ago"He thought of Alexa with its never-sleeping ear. They could spy upon you night and day, but if you kept your head you could still outwit them." The irony is the ability for mass survelence (by unelected corporations) is far more than they had in 1984
- sevensor 8y agoWe've settled the 1984 versus Brave New World debate with the answer "yes, please."
- eclipxe 8y agoBecause it doesn't matter. Human error. Will be fixed.
- jakeogh 8y ago
- randyrand 8y agoIs there a good non-internet connected device for turning on lights with your voice?
- isostatic 8y agoChildren
- rpeden 8y agoI've heard they're expensive to maintain and eventually stop responding to commands.
- jtbayly 8y agoI thought most children had been updated with internet connections in the last couple of years...
- matwood 8y agoI was the TV "remote control" for many years growing up. If you ask my dad about the lights, he would say I had a bug when it came to turning them off.
- dopamean 8y agoI asked this the last time one of these posts came up and everyone suggested I check out https://www.home-assistant.io/ https://www.home-assistant.io/ I haven't started building anything yet but I will in the new year (fingers crossed).
- synesthesiam 8y agoIf you do use Home Assistant, you may be interested in Rhasspy: https://github.com/synesthesiam/rhasspy-hassio-addon https://github.com/synesthesiam/rhasspy-hassio-addon This is an external service I wrote that does offline speech/intent recognition with pocketsphinx, and forwards structured JSON events into Home Assistant for use in automation scripts.
- rpeden 8y ago
- closetohome 8y agoI'm having trouble getting worked up about this one. Yeah a privacy breach happened, but it was only one person's data exposed, and only to one other person. The only reason it made the news is because people are already paranoid about voice assistants.
- pmontra 8y agoThe interesting fact is how much data they store with no apparent reason. They can perform speech to text on the fly and throw away the recording. Machine learning? Do it with people paid to talk to Alexa.
- madrox 8y agoI'm not necessarily disagreeing with the thrust of your argument (do you really need to store all that?), but constraining your sample to people paid to talk to Alexa can create huge swathes of bias. You'd need to make sure the people you pay also reflect all the accents and languages of the people who use Alexa. On top of that, without some amount of voice data, how are you to even know what that accent breakdown looks like? That's a near-impossible task.
- Analog24 8y ago1.) It's not actually scalable to pay enough people to talk to Alexa to train these models. 2.) That data would not be representative of what real users are doing so it would bias the models.
- madrox 8y agoThe only other kind of data I could imagine this kind of reaction to would be NEST cameras or similar. I'm pretty accepting of voice assistants now, but even I would be pretty put out by the thought of video of me in my jammies going to a stranger without my permission.
- justtopost 8y agoI was unxer the impression this was already somewhat the case. Don't many doorbell cameras offer onfettered access to law enforcement?
- pasbesoin 8y agoAmong other things, this makes me suspect that there is no formal internal audit record/trail with regard to accessing these records. We are left trusting individual Amazon employees to do the right thing. Most will, anyway. But hand in hand with "privacy" comes "accountability", on the part of records holders.
- ivolimmen 8y agoHe probably asked for it.... xD
- vufufuffu 8y agoI'm just waiting until people become accustomed to these devices and the tech giants reveal their new real estate play with these devices always on and built into the walls of homes. EDIT: I am definitely feeling what Yanis Varoufakis was talking about when he said that if something isn't done soon, The Matrix will be a documentary.
- madrox 8y agoI've never seen a demographic breakdown of HN, but I'm 37 years old. I'm definitely not the oldest guy here, but I see history repeat itself regularly. I remember when people used location tracking as a reason to not own a cell phone. I'm talking about flip phones, mind you, not even iPhones with GPS receivers. I recall the bulk of complaints in that vein died down around the time Google Maps for phones came out. If this happened today but with Verizon and someone's location via cell tower records, no one would use that as evidence that no one should own a cell phone. That probably tells you what you need to know about the trajectory of voice assistants.
- noobiemcfoob 8y agoAll it ever takes is the utility outweighing the risk for enough people to support a business.
- rchaud 8y agoI am in my 30s too, and I remember reading articles saying that DHS had hundreds of thousands of hours of wiretapped conversations they would never get to because they all needed to be manually reviewed. When the company selling a suspiciously cheap home speaker ('stocking stuffer') is the same company that boasts of having infinitely scalable computing ability and is the only cloud vendor that meets the Pentagon's procurement requirements, people are justified in thinking that Amazon's ambitions go beyond 'making it easier to order online'.
- closetohome 8y agoJust so I'm clear here, you're proposing a conspiracy theory where Amazon, the DHS, and The Pentagon are in collusion to collect and transcribe audio from Amazon Echos -- in such a way that none of the many people who have hacked and extensively reverse-engineered them would be able to tell? I can't find any holes in that theory.
- lghh 8y agoWhat hacking and reverse engineering have been done on the servers that back the Alexa network?
- onetimemanytime 8y agoI feel for average, clueless Joe that has limited knowledge, but anyone that has average and above average knowledge of tech, deserves what he /she gets in these cases. How stupid or tech ignorant would you have to be to add a device that records you in your home? Imagine, fights /sex with wife, lover, arguments with kids, talk about drug deals, tax evasion, leaving job, stealing neighbors home, cheating on husband /wife...WTF would you want to take the chance that a recording device is even within 10 000 feet of your home?
- Kiro 8y agoStill worth it for me. And if they actually recorded all that and used it against me it would be like living in a real dystopian science-fiction movie, which would be an awesome experience in itself. Similar like how you want to experience a zombie apocalypse. Yes, I think it's that unlikely.
- burtonator 8y agoWhen I was in uni root send me the unshadowed /etc/passwd file... no idea what that was about. Just logged in one day with an email of 'passwd' and everyone's passwords were there salted... almost seemed like they were daring me!
- zmix 8y agoDidn't Amazon tell it's customers, that Alexa constantly records locally, but only sends the words after the "Alexa" magic, including 6 seconds before and 6 seconds after it, to the servers?
- deleted 8y ago[deleted]
- archgoon 8y agoThere is nothing in this article that disproves that; and it likely would be featuring prominently if it were more. Also, it seems quite silly for Amazon to lie about this if it could be disproved with a GDPR request. The mistake that was made here is that the wrong customer's data was sent in a GDPR request.
- zmix 8y agoYou are right. But I remember, that I had read a similar story, a week, or two, ago and the topic was, that the police found out about the killer in a murder case, by listening to Echo recordings from the kitchen. I doubt, that this would be possible, if there is no constant stream of data on Amazon's servers. That gave me the impression, that they store it all. At least for a while.
- withinrafael 8y agoI'm a generally happy Echo owner but am getting increasingly worried these mistakes are only going to ramp up, especially with the new Drop In feature, allowing users to enable the microphone on any Echo -- not limited to their own -- given the appropriate permissions are in place. (It's not clear if these are enforced at all on-device or simply cloud-based permissions.) [1] https://www.amazon.com/gp/help/customer/display.html?nodeId=202153130 https://www.amazon.com/gp/help/customer/display.html?nodeId=...
- noobiemcfoob 8y agoIf civilization keeps increasing data production around ambivalent use cases, perhaps we can successfully hide >99% of the lethal data siting around out there until its utility expires.
- amai 8y agoThere is an alternative to the spying devices of Amazon, Apple, Google and Microsoft: https://snips.ai/technology/ https://snips.ai/technology/ "Our technology runs on-device, works offline and guarantees Privacy by Design"
- sambroner 8y agoIs it good? I’m excited for the day when it is!
- saagarjha 8y agoHow does this device grab the current weather and buy things offline?
- binomialxenon 8y agoIt can get data from the internet, I think the point is that it "lives" on-device and hence can be interacted with offline, unlike Siri/Alexa/Cortana which live entirely on the company's servers.
- saagarjha 8y agoSiri/Alexa/Cortana are assistants, while the part you are talking about is speech recognition. Sure, it’s great that this is done offline, but I don’t think it’s fair to compare it to more services because at that point you do need to get data from the internet just like any other service.
- pard68 8y agoI would say your comment was the unfair one. Sure, it cannot tell the weather without internet. But it can still do plenty of more important home automation functions, namely automation. No one owns one of these home assistants for it to tell the weather. You get it for controlling other devices with your voice and this can do that without access to thr world wide web.
- Kiro 8y agoI'm all against unwarranted surveillance but for voice assistants you basically sell the risk of privacy intrusion for utility. You're making an active choice. I see nothing wrong with that.
- tjpnz 8y agoIt wasn't long ago that people were up in arms about the Xbox One sending audio back to Microsoft in order to implement voice commands. In the time since this kind of thing seems to have become acceptable or even mundane. What is it that changed? It's not as if big tech has become more trustworthy in recent years.
- bitrrrate 8y agoThere’s just no end to the abuse of our data is there? Sorry, no Alexa, Siri, Google Assistant for me.
- gigatexal 8y agoI paid full retail for my HomePod (BestBuy in the states has them for 150 usd off) and love it. But I love it more because of the company behind it: Apple is not in the business to sell my data to the highest bidder. And their stance on security and customer privacy allowed me to put a talking tube in my home. There was no way in hell I was going to add a listening device from Google or Amazon.
- dbg31415 8y agoWhat happened to all the layers of security Alexa is supposed to have to prevent data from being sent upstream, isn't it supposed to process everything locally? Eww. Come on guys, Rule 1: Don't make creepy tech.
- _Codemonkeyism 8y agoAfter two years of Alexa it will move out 2019. Experience was underwhelming, especially play lists from Spotify were hit and miss, one day no problem, the next day sometimes an excuse, sometimes just a beep, sometimes silence. I would need to turn on the light by myself in the future though.
- purplezooey 8y agoGlad we have a GDPR-like thing in California and wish some of the lesser states would get their house in order. :)