12 ms·
Amazon’s Alexa assistant told a child to do a potentially lethal challenge
- mccorrinall 5y agoSave a click: https://twitter.com/klivdahl/status/1475220450598924297 https://twitter.com/klivdahl/status/1475220450598924297
- annoyingnoob 5y agoCan Alexa tell the age of the person behind a voice? The internet is full of questionable things, like the penny challenge, and Alexa is just voice search. Teaching your kids to use Alexa and expecting good results is the real issue here. As a parent, this looks like a parenting problem to me.
- omnicognate 5y agoYou don't teach children to use Alexa. If Alexa is there they just talk to it. Putting a device into homes that is voice activated, unlocked and usable by anybody capable of speech carries with it some duty of care not to, you know, tell kids to play with mains power. Amazon don't disagree with this.
- gruez 5y ago>Putting a device into homes that is voice activated, unlocked and usable by anybody capable of speech carries with it some duty of care not to, you know, tell kids to play with mains power. I agree. That's why we should have mandatory faceID on smartphones/tablets/computers/books, so we can positively identify the user's age before allowing access to it. If you fail the age test, your access will be restricted to kid-friendly websites with curated content.
- jdmichal 5y ago> You don't teach children to use Alexa. If Alexa is there they just talk to it. I'll second this. My 5 year old talks to my father's Google device when we're visiting. We never taught him to -- he just learned from grandpa.
- annoyingnoob 5y agoYes, kids learn a lot by observing others. Your 5 year did not talk to a Google device before he saw someone else do it. Kids learn just from watching you, its still teaching and learning. Saying OK Google to electronic devices is not innate in humans.
- jdmichal 5y agoI guess "teaching" can be done passively, though I feel like the context of my comment made it clear that I meant "teach" as a purposeful action. Maybe you can read it as, "we never instructed him to", if being pedantic about the word "teach" is preventing this conversation from moving forward.
- cpuguy83 5y agoJust because they can figure out how to use it doesn't mean you don't teach them how to use it. In particular what to do with the information they get from it.
- encryptluks2 5y agoThis is the best answer and the media looks for opportunities to push an agenda. I'd rather Alexa sometimes get things wrong because it is just searching the web than to have it constantly updated with "approved" content.
- shkkmo 5y agoNot only did Alexa retrieve the wrong content (instead of a list of challenges to do, Alexa sourced from a list of dangerous challenges you shouldn't do), Alexa then stripped off the entire context of the warning about not doing the challenge because it was dangerous, and instead added a timer. So the issue here is less about needing to sanitize the internet for kids, and more about how AI struggles to categorize and summarize content and the dangerous effects that can have.
- encryptluks2 5y agoYou overestimate what AI means in terms of marketing. If a site owner doesn't markup their site properly, then that could also be the cause of issues like this. Do you really expect Alexa to be "smart" enough to understand human logic like an actual human, or do you not see that AI can basically mean Alexa goes to the web, does a search and parses markup from a website to the best of its ability and they call it AI? Some people care about the technicalities and others just want it to work perfectly, but I think the people that care about the technicalities are more realistic than just wanting something to be the way they think it should be.
- hakfoo 5y agoThe right answer is that Alexa (especially in a "Kids" mode) should only pull from a white-listed, manually vetted data pool. It wouldn't find the list of dangerous challenges in the first place. In the worst case, this means, oh no, they might need to find human moderators to approve content for the pool, or even custom create content. They can afford it.
- shkkmo 5y ago> You overestimate what AI means in terms of marketing. Not really, you are making huge assumptions about me here. I don't expect Alexa to be smart. I do expect that if a company is going to scrape other people's content to use in their service, it behooves them to do that scraping properly and to not use automation to do it if their automation isn't up to snuff.
- deleted 5y ago[deleted]
- shkkmo 5y agoThe source for the result was a page warning about a dangerous challenge that parents should watch out for. Alexa stripped that context away completely and instead added a timer that discouraged careful consideration. This is an algorithmic failure, regardless of the age of the user. Having this sort of failure in a product that has been explicitly advertised for use by kids looks pretty bad and thus it is unsurprising how quickly amazon acknowledged the error and promised to fix it. It might be unreasonable to expect Alexa to make the internet safe for kids, but if they want to be included in homes with children, they need to at least not directly make the internet more dangerous for children. I expect that managing children's safety online will become a key product feature of Alexa in years to come.
- mmastrac 5y agoAmazon's market cap is 1.72T. If anyone can afford to push the state-of-the-art in terms of safety, they could. They just don't care to because nobody pushes them to.
- systemvoltage 5y agoI personally don't want the toxic mix of Big Corp + Gov Regulations for something like a home assistant. Literal nanny state. Micromanaging is not the answer. We need Gov regulations to break up Big Tech in general. Just throw the goddamn device out of the window. Problem solved.
- salt-thrower 5y agoHow is that "literal nanny state?" We're talking about removing a home assistant's ability to tell you to do something lethally stupid. Basic consumer safety measures are a staple of a healthy society; just because a device is in your home doesn't exempt it from the government setting safety standards for it. If anything, the opposite is true. Though on your last point we agree. Throwing the thing out the window is the best solution overall.
- mmastrac 5y agoThere are plenty of government regulations that work. Regulation doesn't preclude breaking them up, which I'm also in favour of.
- systemvoltage 5y agoSure, I want natural gas lines leading to my apartment to be regulated. But, I see Alexa as an information source. I don't want books to be regulated just as much as I want Alexa to be regulated.
- salt-thrower 5y agoI for one wouldn't assume that "Alexa, give me a challenge to do" would be pulling from the open, uncensored internet. I would assume that Amazon had some kind of curation method that would remove the possibility of Alexa telling you to do something lethally stupid. Further, what Alexa did is actually even worse than just reading from the open internet, because it only took the "challenge" part and stripped away the context about how dangerous it is. This is akin to something like, "Alexa, tell me about civil rights" and it quoting Stormfront or some other hate site's take on race, without any other context about what it's reading from.
- annoyingnoob 5y agoWhen I ask for something really general, I assume that Alexa is just a voice controlled web browser. Since Alexa is really about advertising and selling you things, I would assume that any curation is just to drive you to Amazon products. Once you get outside of products that Amazon can sell you then Alexa breaks down to a basic web search - as evidenced here. We are calling out Alexa here but I think the same is true for other voice assistants too. Calling these products 'smart' is so wrong - there is nothing 'smart' about them. That 'smart' marketing has you and nearly everyone else assuming that they are a lot more capable and functional than they really are. Its just voice commands to drive things you would otherwise touch in some way - it has no where near human abilities.
- hakfoo 5y agoThe assistant platforms fall back on web search because they want to pretend to be omniscent. If it said in the documentation "here are the 1,602 commands explicitly supported, the data sources it pulls from, and anything else goes to an error message", it wouldn't be magical anymore. Less magical might be a far better bargain for the platform brands, though. It's literally the face (err, voice) of your brand, and you're fighting a completely adversarial war here. At best, you've got unrelated external data sources that could make it say something stupid, but there's an entire spectrum of bad actors with incentivization to get Alexa/Siri/Google Assistant to say something offensive, scandalous, or hazardous.
- blamazon 5y agoDiscussed here yesterday with 520 comments: https://news.ycombinator.com/item?id=29709369 https://news.ycombinator.com/item?id=29709369
- tasubotadas 5y agoI fail to see how is this responsibility of Alexa. It's parent's fault for letting children use devices like Alexa. Stupid chatbots will always give stupid answers.
- AlexandrB 5y agoSo if you have an Alexa in your house and a child says "Hey Alexa" will Alexa ignore that by default? Aren't you just saying that it's the parent's fault for having an Alexa at all?
- akersten 5y agoIf this answer from a search engine was actually persuasive enough to their impressionable child that it posed a real risk to them, then it is absolutely the parent's fault for exposing said child to the literal internet before they are ready and able to determine what is good and what is not. "That cup shouldn't have let my kid drink the bleach" is not a convincing argument that the cupmaker is at fault for leaving your cleaning equipment in an accessible location. It's the internet, it's dangerous, teach your kids about it before it teaches them. I am 0% concerned about hypothetical effects of search results and 100% concerned about things that are actually harming our children, like the LAPD[0]. [0]: https://edition.cnn.com/2021/12/28/us/lapd-teen-killed-valentina-orellana-peralta-family/index.html https://edition.cnn.com/2021/12/28/us/lapd-teen-killed-valen...
- omnicognate 5y agoSo how do you propose parents prevent their children from using Alexa other than simply not having it? Not having it is my solution, and I put a bunch of (specialised, technical) work into safely introducing my kids to the internet [1], but if that's the only safe solution Alexa should have a "not safe if you have kids" warning. But of course it's not the only solution. Amazon just need to make sure their device doesn't tell kids to stick things in power sockets, and they don't dispute this. [1] https://news.ycombinator.com/item?id=29177716 https://news.ycombinator.com/item?id=29177716
- deleted 5y ago
- api 5y agoSo I've got a new Fermi paradox solution. Once we build a large enough radio telescope we will start receiving a lot of chatter from around the universe. Buried in this will be a "high energy physics challenge" involving a powerful particle accelerator, a superconductor, and some synthetic elements with very large nuclei. After performing this challenge there will be another asteroid belt where Earth used to be, and someone somewhere will get some LULZ when they detect a certain kind of characteristic gamma ray pulse followed by silence.
- wingmanjd 5y agoSo, the celestial equivalent of "Press Alt+F4 for hacks"?
- ljm 5y agoInterdimensional Tik-Tok sounds interesting. I wonder how many alien civilisations are receiving our broadcasts millions of years in the future, and trying to reproduce them. Like an entire planet just received the first season of Jackass and their anthropologists have had to rethink their entire concept of this distant human civilisation.
- heavyarms 5y agoNot exactly the same, but "The Three-Body Problem" by Cixin Liu has a plot that starts a little like this.
- 0xdeadb00f 5y agoThat's actually a fantastic idea for a novel or a movie (are you referencing one?) If that's never been written before, I hope someone does.
- m1117 5y agoThe child asked for it lol. But maybe they can evaluate safety of advise.
- woodruffw 5y agoChildren ask for (and do) all kinds of dumb things without possessing the agency necessary to evaluate the consequences. That's why our society has the concept of "child-proofing" household objects and appliances. Why would Alexa be any different?
- coldcode 5y agoWhen I was around 5-6 I wanted to know what would happen if I put a hairpin into a light socket. It seemed a reasonable experiment beforehand; after I picked myself up after flying across the room I learned a valuable lesson. Around the same time I tested whether Santa existed by putting out food; when no one ate it I assumed Santa was fake. Children are not very experienced at knowledge and thinking and often do things that confound adults. Giving them access to something they do not yet have the ability to understand or discern can be both useful (learning) and dangerous and it's up to parents and other adults to make sure it's not going to kill them, yet at the same time not completely stifle learning. I could say I eventually became a programmer because I nearly fried myself doing a test...
- hiptobecubic 5y agoYour primary job is a parent is to make sure your kids don't die. The rest is kind of a wash. FWIW, I blew up a power outlet with a paperclip when I was 17. You don't have to be a small child to be dangerously ignorant.
- tsomctl 5y agoI stuck a table knife into the outlet when I was little. I only did it because everyone kept telling me not to. I have this memory of it jumping out at me, I'm genuinely curious if I actually touched the hot side or it's just my imagination.
- Cerium 5y agoI accidentally inserted a screw driver into a switch without a cover plate, making contact between the hot and the grounding frame on the switch. I still have that screwdriver somewhere, about half the tip was melted. I don't remember getting shocked, but it blew the fuse on the circuit and scared the hell out of me.
- gruez 5y ago>Around the same time I tested whether Santa existed by putting out food; when no one ate it I assumed Santa was fake. That doesn't seem like sound logic to me. Maybe he's just too busy to eat food that was put out at every house? You'd get pretty full by eating a cookie from every house in your neighborhood.
- boomboomsubban 5y agoWas there ever a lawsuit against GPS providers telling people to turn right into something like a lake? I couldn't immediately find one except a women who walked in the middle of a highway because that's what Google's route showed. It seems like the same ruling would apply to both, which could effect consumer GPS somewhat.
- micromacrofoot 5y agoehh an adult driving a car has the reasonable expectation of making safe decisions to avoid driving into a body of water… could I blame the GPS for slamming into the car in front of me or hitting a pedestrian? I think there’s a considerable difference from giving a child instructions to electrocute themselves… they literally don’t know better. They’re not licensed or educated.
- boomboomsubban 5y ago>an adult driving a car has the reasonable expectation of making safe decisions to avoid driving into a body of water The famous example of that happened in the middle of the night during a storm, but you can use the other common example of being told to turn the wrong way into a one way. The two scenarios seem similar, an ai giving dangerous directions and I suspect the difference in age and experience would not effect the precedent.
- micromacrofoot 5y agosure but we’re failing if we’re teaching adults to blindly follow ai
- cpuguy83 5y agoIn both situations I don't think this is "blindly following ai". If you are unfamiliar with an area, hence why you are using gps, then it is reasonable that someone does not know a street is a one way and it may not be immediately evident. Add time of day or inclimate weather to the situation and things can get bad quickly.
- Grazester 5y agoThe fact that you can do even do this should highlight how bad the 110v American sockets are. With the 220V UK style socket this never results in anything happening.
- deleted 5y ago[deleted]
- woodruffw 5y agoI wholly agree with you, although I think it's worth pointing out that the specific risk here (America's dangerous socket design) is purely incidental. The underlying problem (Amazon appears to be naively scraping information from the information and automatically applying truth judgments to it) needs to be resolved.
- dionidium 5y agoThe latest National Electric Code requires tamper-proof receptacles practically everywhere. See: https://diy.stackexchange.com/questions/13414/are-there-locations-in-a-residence-where-a-non-tamper-resistant-receptacle-can-s https://diy.stackexchange.com/questions/13414/are-there-loca... I guess they wouldn't entirely eliminate this specific problem -- if you were determined to get a plug in halfway and touch one end with a penny, then you could probably pull it off -- but they do make unintentional errors of this variety much less likely. I replaced all my receptacles with TR versions when I moved in to this house. It takes an afternoon, but it's a good opportunity to make sure they're wired correctly -- you may be surprised to learn that many won't be -- and to replace old, worn-out outlets that present hazards. It's a very easy job for a homeowner, given some basic knowledge about residential wiring that's easily obtained.
- Grazester 5y agoThe receptacle is only part of the problem. On 220uk plugs the part of the pins that is partially exposed as you insert the plug is insulted to prevent people doing stupid things like this coin thing.
- JetSetWilly 5y ago
- jrmg 5y agoMan, the number of people here arguing that parents should not let their children use Alexa, or that Amazon bears no responsibility for it presenting factually incorrect answers to questions really makes me worried about the future of our industry… Have the people commenting used a voice assistant? There’s a very clear difference to even (or, given the weirdly unintuitive opinions here, maybe especially) a casual user between returning search results and using information gathered from them to directly answer a question. The difference is there even when interacting with humans - it’s the difference between “I read an article about that once, let me find it so you can read it too” and “The answer to your question is X”.
- annoyingnoob 5y agoSure Alexa can be improved but we still need to teach our kids about safety, online, voice assistant, and otherwise. My kids want to do all kinds of things I don't let them do. My son wanted to run around outside in freezing cold rain, in shorts with no shoes, I didn't let him but Alexa would have. I don't expect Alexa to be a parent or to supplant me as a parent.
- qiqitori 5y agoMaybe they could add a feature that disables commands from child-like voices, or let you register voice profiles to create whitelists or blacklists.
- corobo 5y agoI'd love a whitelist of voices It's been a while since I've had a gathering at my place resulting in 4 or 5 dildos ordered that need cancelling out but it is still irritating when something on TV or maybe a Twitch stream wakes the Alexa up.. especially if they've woken it up by telling their own Alexa to do something like play a song and mine joins in
- JabavuAdams 5y agoI like your friends already.
- vl 5y agoTo be fair, if it really was “lethal” half of eighties kids wouldn’t be alive today.
- ultrasounder 5y agoI did this when I was 9-10 back in Chennai, India(240V LOL). I picked up my Divider from my camel Geometry box(comes with divider, compass, protractor) and shorted it right across the terminals. All I knew was I was lying 2 feet outing the relay had tripped. Got a mouthful from my parents and went on to become an EE(I still am employed as an EE to this day!). I remember the experience as if it had happened just last month. Fast forward 30 years I caught my 1 year old son trying to put his fingers into a socket. I went ahead and child proofed the entire house. Fast forward, 11 years he wants to learn to solder now. I guess its in the genes.
- kragen 5y agoPrevious discussion of the toot: https://news.ycombinator.com/item?id=29709369 https://news.ycombinator.com/item?id=29709369
- codr7 5y agoI still don't get why people buy this crap, it's so obviously useless and potientially harmful. I once tried putting a plug with cut off cables into a socket, must have been around 5-6yo, it threw me quite some distance backwards and was very uncomfortable but nothing more came of it besides completely killing the urge to try again.
- JabavuAdams 5y agoThis just highlights that we need our agents to work for us, not for some other entity.
- zaxbeast 5y agoWhen I was a kid, I did something a lot safer... I wrapped a metal paper clip around a wooden pencil with the ends pointing the same way and then stuck it in the outlet... it probably tripped the breaker but I never got shocked (while doing that). Another thing we liked to do was mix milk and pool chlorine in a plastic bottle... it makes a nice pop (can get you hurt if you don't get away fast enough though, after you put the cap back on).
- deleted 5y ago[deleted]
- throwaway34511 5y agoWhen I was maybe 8-9 I found a dual plug extension for the Christmas lights. It looked like two of those cheap green plugs with a little socket on the back, wired to each other. I think some people call it a widowmaker, and to this day I can't imagine why something so dangerous would have been manufactured. Anyway, I wanted to know what would happen if I plugged both plugs into the wall at the same time. I got paralyzed right there in front of mom, unable to even talk or shout, for at least 15 seconds, but it seemed like forever. It's weird because I knew to only touch insulator. I somehow fell off it and then unplugged it. I slunk away without her finding out, but my arm had a strange feeling for a while.
- boring_twenties 5y agoDo you mean one of these? https://i.imgur.com/SM60upY.jpeg https://i.imgur.com/SM60upY.jpeg
- throwaway34511 5y agoIt was like that, but with light string style piggyback plugs on both ends. So the non-wall side could have been in the middle of a bunch of piggybacked plugs, but there would still be two exposed, energized prongs on whatever it was plugged into.
- cyckl 5y agoSuicide cables!
- morpheuskafka 5y agoIn theory, plugging both of them into the same circuit (such as two plugs in one receptacle) shouldn't damage anything as far as I know. Of course, it won't do anything useful, and if the plug is not polarized then there's a good chance of plugging it in backwards and shorting out the circuit. And, as you learned then, there is a great danger of accidentally touching the live end. "Proper" use of this, if there is such a thing, would dictate plugging the load in first, then the supply, but in this case there was supply on both ends.
- aaron695 5y ago