12 ms·
I am oversimplifying here but: * install device that is designed to listen to speech in the house * the device is connected to internet * the device is capab
by yason 8y ago
I am oversimplifying here but:
* install device that is designed to listen to speech in the house
* the device is connected to internet
* the device is capable of contacting other internet peers/services/hosts
* the device knows a bit about its owner's internet presence such as contacts
* the device is equipped with simple conversational user interface based on fuzzy human speech-command detection
These basically bound the failure modes of the complete system. I am not surprised that a glitch like that happened. I can certainly attest to how shocking it might be to discover first hand that it really did happen, but given the context I can't say it's hard to not foresee something like this.
- tj-teej 8y agoIf you replace "fuzzy human speech-command detection" with "buttons which can accidentally be pressed" - how is this different from butt-dialing?
- zymhan 8y agoI don't think there is much of a difference. People took a while to understand butt-dialing, and not do it as much. It'll take a while for people to get used to voice assistants that trigger seemingly randomly.
- sdenton4 8y agoAFAIK, pocket dials have mainly been reduced by improvements in phones, rather than changes in user behavior; if the camera just sees inside of a pocket, it's unlikely to be an intentional button press.
- zymhan 8y agoI mean way back in the day when people had candy-bar phones and had to remember to enable the keypad lock :D
- EvanAnderson 8y agoI made a number of emergency calls from my pocket on my Nokia 1100 even with key lock enabled. It was infuriating. It happened once when I was having a heated argument with somebody and it took some work to convince the dispatcher not to send a law enforcement officer. (There was nothing physical happening and no threats of violence, but we were speaking very sharply to each other.) I do not miss that about my Nokia 1100 at all.
- modzu 8y agowhen my phone is locked (which it always is in a pocket) it can't butt dial. whats the analogous behaviour for alexa? also remember these are shared devices in a home, not a personal device in your hand. could i shout into somebody's window, "hey alexa send my browsing history to bob@hotmail.com"!
- lopmotr 8y agoThey can still call the police while locked, which is something that happened to a friend with an old feature phone while were were in the process of illegally hiding/dumping an old car.
- reaperducer 8y agoMike, is that you?
- notatoad 8y agoThe "lock" analogue on an Alexa device would be the mute button on the top of the unit. https://www.safety.com/wp-content/uploads/2017/06/mute-button-cropped-1.jpg https://www.safety.com/wp-content/uploads/2017/06/mute-butto...
- coatmatter 8y agoThe analogous behaviour would be you being aware of Alexa's flaws and not triggering it accidentally. While you may have never pocket dialled, plenty of people have. I've received more than a few accidental calls in the past. I equate this to some phone users placing their devices hanging over the edge of tables - as if they don't care about them hitting the floor. Should phone makers toughen their phones and should Amazon improve Alexa anyway? Sure.
- edwhitesell 8y agoMost phone now have some kind of lock screen, which makes it pretty difficult to get to the butt dialing stage. Will speech command recognition get to that stage? The main reason I won't have any of those products in my house is because of that. I'd much rather have a confirmation of some kind before the system takes action. "...random talking..." Assistant "I am recording" Me: "Stop recording"
- modzu 8y agosome kind of acoustic biometrics would be helpful here (ie respond only to account holder, or disable some actions for others) along with better heuristic recognition of directives thats not foolproof but much better than what we have now, and i think we are pretty close
- edwhitesell 8y agoAgreed, but I think vocal biometrics is a significantly more difficult problem to solve when even the best speech recognition still has issues like this. Phonetics is hard. Especially with ambient noise, echo and such. I had a conversation with one of the speech engineers when I worked at a speech recognition company and the level of detailed problems to solve was impressive. Totally made sense after talking about it, but things I would not have thought about before. I'd imagine the next thing to come in this area that would really make an improvement is an "on person" microphone. Maybe it's a pen in your pocket, or some kind of vibration detection (that could pick up the wearers voice), that would then allow some improvemnts in the domain of "who is talking" and how well the voice is processed.
- jwandborg 8y agoAnd once you've managed to achieve perfect biometrics your next task is to prevent replay attacks. An alternative, "wireless" approach to near-perfect security would be to invent-plement some kind of vocal, human-executable GPG/TLS.
- 8y ago
- cookiecaper 8y agoa) Your phone has a microphone that picks up information spoke more-or-less directly into it. An Echo is specifically equipped with a microphone array designed to extract audio from anywhere in the room. b) There is a great deal less ambiguity around "pressing buttons" than there is around interpreting speech. While it is unlikely that your phone will incorrectly detect button presses, it's very common for voice-activated devices to a) incorrectly detect a wake word (either negative or positively); and b) misunderstand some particular word used in the command. Your phone is not going to think you pressed the "Clothes" button when you actually pressed "Close". c) The entire functionality of the device is accessible from behind that big ambiguous interface. On a phone, there are many distinct steps and screens to step through when you want to do something (this complex interface, by the way, is a non-trivial part of why butt-dials are quite rare these days). On a "smart speaker", most things are just one misheard statement/command away from occurring.
- TheSpiceIsLife 8y agoIt's interesting that the pattern of human communication where things are just one misheard statement/command away from occurring has been well modeled in these devices. Does having the device programmed to make mistakes make it more comfortable to use, because we know humans are infallible too.
- spdustin 8y agoSimply: no one else is dialing via my butt.
- dolessdrugs 8y agohold my beer
- hunter2_ 8y agoJust as nobody but the primary user(s) triggered the failure in this case!
- BonesJustice 8y agoI can't pick up my coworker's phone and manually dial a number, because years of learning from our mistakes have led us to implement lock screens and such. I can, however, say "Hey Siri, call 867-5309." Or "Hey Siri, Facebook status: my anus is bleeding. SEND!" Because we are apparently incapable of applying past lessons to new technologies.
- rdiddly 8y agoIf they operate anything like the way I operate, they know there should be some kind of lock on there, they just haven't gotten around to building that part yet. And if their management is anything like my management, they go ahead and release it anyway.
- pishpash 8y agoIt really shouldn't be hard to do a quick safe filter and ask "You sure you want to send that?", again, like a real human. My point is they're really half-assed products right now that are behaving badly/immaturely.
- collyw 8y agoApparently it did have a confirm question.
- edm0nd 8y agoIf the iPhone has a lock screen, Siri will not do any of those actions without it being unlocked first FYI.
- BonesJustice 8y agoThat was definitely not the case when I tried it a couple years ago. If they’ve since locked it down, then I’m glad to hear that.
- arcticbull 8y agoHm. I can't "Hey Siri" someone else' phone, can I? It's certainly never worked for me.
- wlll 8y agoI haven't butt-dialled anyone since I got swipe to unlock on my iPhone. A 6 digit unlock code/fingerprint would make it seem even less likely to happen again.
- robotrout 8y agoI do not want to set up this feature on my Alexa for precisely this reason and I am finding Amazon to be really aggressive about forcing me to enable this feature and harvest my contact information. The Alexa phone app is almost unusable for me, as it constantly goes into the telecom setup screen whenever I try to do something else in the app. I am forced to exit the app and come back into it, whenever this happens, to escape that screen.
- mirimir 8y agoWait! So you're saying that Alexa actually includes the capability to record voice, and send recordings to contacts? I suppose that some would love that. But, as this demonstrates, it's prone to fail spectacularly. You'd think that there'd be protection. Maybe a restricted set of contacts who could get such messages. And a confirmation step, such as "Do you want to send this recording to 'my mother'?" Or "... to '911'?" Edit: Damn, didn't see Clubber's comment.
- coatmatter 8y agoWhat do you actually use Alexa for?
- robotrout 8y agoI use Alexa for listening to music. I rarely go into the app, but occasionally attempt to use it to set up a new music service, control the volume or track (if I am unwilling to shout down the music to tell it by voice). When I open the app, I get immediately thrown into the setup screen for the voip feature they're pushing.
- duxup 8y agoI remember the first voice command stuff on google. Buddy of mine was trying it out and I couldn't help myself but shout: "NEW TEXT MESSAGE TO JENNY! FUCK YOU! SEND!" Fortunately the phone was too slow to take it all (even if it had Jenny would have found it amusing, she was a cube or two away). But yeah first thing I thought of was that if it's listening how easy is it for it to mistake intent or someone else or etc. Humans talk funny, computers don't get it, they're gonna mistake commands all the time. As it is I hate how I can't talk ABOUT "ok google" around my phone... because it always goes off.
- monk_e_boy 8y agoMy friend installed IBM OS/2 WARP beta when it came out, it too was voice controlled. We would shout into his room, "select all, delete" or "format c" or other childish fun commands.
- duxup 8y agoHumans are terrible. When the computers take over.... I won't be happy about it, but I'll kinda understand.
- eXpl0it3r 8y agoJust shout "self-destruct" all the time and we'll be fine.
- vidarh 8y agoAlexa has 4-5 joke responses to self destruct. E.g. "I'll start the countdown but only on the understanding you'll dramatically cancel it at the last second", and "3-2-1 Boom. Hmm that didn't quite work did it?" or something like that, and a few others.
- teddyh 8y agoLike the countdown at the end of Monsters vs. Aliens: https://www.youtube.com/watch?v=FJ5DA22tBOY#t=4m27s https://www.youtube.com/watch?v=FJ5DA22tBOY#t=4m27s “8… 7… 6… 5… 4… 3… 2… 1… … Hmm, nothing happened, maybe my count w… [BOOM]”
- Clubber 8y agoSounds like the equivalent of a confirm popup would be a solution. Are you sure you want to send this to so and so?
- mtgx 8y ago> "I felt invaded," she said. "A total privacy invasion. Immediately I said, 'I'm never plugging that device in again, because I can't trust it.'" Let this lady's reaction be a lesson to all of those who say "people don't care about privacy." She wired her home with Alexa devices, and yet when she found out how her privacy could practically be invaded through such devices, she seems like couldn't get rid of them fast enough. People care deeply about their privacy. They just don't understand the true implications of even a device that's supposed to "listen to you" and the possibilities of what someone else could do to their internet-connected devices. This is why governments must intervene with laws such as GDPR or even much stronger ones in the future to protect people's privacy, because the companies themselves have no incentive to "self-regulate" other than a negligible amount. The only problem is some governments will not intervene unless enough people go through similar situations such as this lady. In the US even that may not help much because the vast majority of politicians care most about continuing to receive (however modest) donations (also called bribe money elsewhere) from the industry.
- ubi_now 8y ago> This is why governments must intervene with laws such as GDPR or even much stronger ones in the future to protect people's privacy, because the companies themselves have no incentive to "self-regulate" other than a negligible amount. Market failure does not imply government solution. You're going to need a much stronger argument than "companies aren't doing a good enough job." Amazon invaded her privacy, she realized it, and now she doesn't use the product. If you want others to do the same, start raising awareness yourself instead of enacting nanny state laws with other people's money.
- Proziam 8y agoI fail to see how market failures are not a reasonable point to look at whether governmental solutions are applicable. Case and point - 2008 Market Crash, everyone agrees the dishonest practices that led to the crash should have been made absolutely illegal (in the cases where they already weren't) In this case, we have MANY significant and potentially extraordinarily damaging failures in the markets to effectively protect people's privacy occurring fairly regularly. GDPR is not the ideal solution my any metric, but that's a failure of government to create effective legislation - not a lack of need for effective rules.
- AnaniasAnanas 8y ago* the device is composed of proprietary blobs Amazon might as well record everything that you say and send it to the NSA.
- JepZ 8y agoYou are right, but your argument is beside the point. The point is, that those devices are not designed to work 100% accurate. So even if Amazon has no malicious intent, such privacy-invading events will happen! They are just unlikely, not impossible.
- reaperducer 8y agoEvery time my wife says to me, "Are you serious?" Her iPhone chirps up with something like, "Yes, I am Siri." Somehow it hasn't broken her (my wife) of the habit yet.
- rz2k 8y agoTo be fair it hasn't taught you not to say such astonishing things either!
- foolfoolz 8y agoi think there’s a market for non cloud home automation. i will never buy alexa
- kevsim 8y agoOthers agree. Silk Labs[1] (founded by former Mozilla CTO Andreas Gal) seems to be active in this space [1] https://silklabs.com https://silklabs.com
- Digital-Citizen 8y agoIndeed, "if you apply General Curtis Le May to a situation and you get havoc, well, that s what you called General LeMay in for" (to use an Eben Moglen quote out of the context in which he said it); if you host a device in your home running proprietary software you ought not be surprised that it is spying on you and the proprietors determine what to do with that data, not you. We must not forget another critical component of all this: these always-on listening devices run on proprietary (nonfree, user-subjugating) software. Therefore the user has no permission to: inspect what it does, modify it to do only what the user wants (or, since most computer users aren't programmers, get someone technical they trust to perform such modifications for them), run the modified software in their device, and distribute copies of the improved software to help their community. If these devices ran on free software (software that respected the user's freedom to run, inspect, modify, and share) the more technical among us could help them. But as it is even the most technically-minded willing person cannot legally do this work to help them. As Eben Moglen reminded us after the Snowden revelations came out: It's critical that we don't fall into the trap of saying something akin to 'those kids take too many darn pictures' like concluding that we just can't have these devices or their services at all. We can have all of their alleged conveniences but only if we have free software implementations.
- harryh 8y agoThe idea that if these devices ran free software they would be more humane than the proprietary software they currently run is ridiculous. What are you giving your mom? A Mac running OSX or a PC running Linux?
- AnaniasAnanas 8y ago> What are you giving your mom? A Mac running OSX or a PC running Linux? I gave my 60 year old mother my old PC running Debian with dwm as its WM. She likes it. > The idea that if these devices ran free software they would be more humane than the proprietary software they currently run is ridiculous. Is it though? I am pretty sure that Amazon already gathers free data from its users, such a thing would not exist in a free as in freedom device.
- tantalor 8y agoThey key feature you missed, * devices can operate without user's knowledge These types of devices are easy to accidentally trigger; it must be vastly obvious when that happens.
- bshastry 8y agoyou just described a smartphone
- amelius 8y agoThat's almost the description of a smartphone.
- willortiz 8y agoHello everyone! I had seen so many recommendations on trusthackerslounge2018@gmail.com, so I contacted him to help me Clone my ex cell phone and WhatsApp. Just like Magic, I got the files to get it done and I have access to my ex phone. He was really efficient and I have access to everything including phone calls, logs, sms, surrounding and location. What I like about the job is that it cannot be traced back to me. I have this working for 3 months now. I am just another satisfied customers. Thanks to trusthackerslounge2018@gmail.com.....You can check his webpage it's still on construction..