7 ms·
The day my smart vacuum turned against me
- codetiger 1y agoThought I was just buying a smart vacuum. Turns out, it was a little spy on wheels. Here’s the story of how my vacuum stopped working after I blocked its data uploads — and how I uncovered a hidden remote “kill switch.”
- altairprime 1y agoHi, thanks for describing what you’ve found — but the details shared aren’t enough for the community to reproduce your findings. What hostname/s did you block? What filename prevents auto-reboot? What firmware version is your device? Were any credentials necessary to access your robot’s internal syslogs? Was the remote always precisely 8*86400 seconds after you powered on the repaired model? The repository contains only the barest “how to repurpose this device” details with no supporting material evident for your post’s topic, “what the OEM OS was doing”, which makes the final paragraph either wrong or misleading. Do you have a timeline in mind for when that will be published to GitHub? The story is marginally interesting, but without the technical details, it’s more “this is completely unsurprising, see also nearly all in-home smart devices” and less “this is novel and interesting”. (I concur with the outrage, but outrage alone does not satisfy.)
- taylodl 1y agoThis reveals a whole new channel of modern warfare. Imagine a nation state getting control of an adversaries' smart devices? You don't need to destroy capital-intensive infrastructure such as an electric grid if you can disable their ability to store and cook their food (internet connected ovens and refrigerators). That's morbidly fascinating, though I now realize I'm potentially open to such an attack.
- mft_ 1y agoIt struck me recently how vulnerable we are to small disrutpive attacks of the sort you mention, and more. For example, several major European airports were closed recently due to unidentified drone activity around them. I don't know if the authorities have figured it out, but in theory someone could cripple air travel for the cost of a few anonymous drones.
- no_no_no_no 1y ago[dead]
- wrs 1y agoCheck out Mr. Robot S2E1, where someone is terrorized by their hacked smart house. Or 1977's Demon Seed, a film where an AI house does worse things. I really thought the latter would remain fictional, but I'm less sure now.
- dylan604 1y agoLots of movie plots with hacking critical infrastructure. Hackers controlled the ballast of a ship. Some other movie(s??) have taken over the flow of oil/gas at refineries. The entire US electrical grid is nothing but a soft target. A Ukranian style sneak drone attack on enough substations would knock the entire country into a blackout without ever touching the generation plants.
- RajT88 1y agoDefinitely stick with Mr. Robot. Unless it's Saturday Night and you're drunk. Then go for Demon Seed.
- reaperducer 1y agoUnless it's Saturday Night and you're drunk. Then go for Demon Seed. Don't give Svengoolie any ideas!
- RajT88 1y agoFellow Illinoisan I see. Recording Svengoolie is one of the main functions of my JellyFin box. Alas, lately it's mostly been 60's and earlier, when at one point it was a lot of 70's-90's stuff.
- Loughla 1y agoYeah but Young Frankenstein? Come on. That's going to be good. Sven is one of the few things I miss about living in Chicago.
- 1y ago
- loloquwowndueo 1y agoGo for the phones. Watch “Leave the World Behind”.
- SchemaLoad 1y agoPhones are extremely hard to hack and getting harder. The phone hacking companies only bother targeting an extremely limited set of people because once it becomes widespread, the whole exploit chain gets patched and you have to spend millions of dollars developing a new one. With ARM Memory Tagging Extension becoming common on phones now it's getting borderline impossible to hack them.
- loloquwowndueo 1y agoThank god the movie is just a movie :)
- xnzakg 1y ago...or you can just give someone a "smart" device that requires then to install an app with lots of unnecessary permissions to use it.
- Terr_ 1y ago> This reveals a whole new channel of modern warfare. Imagine a nation state getting control of an adversaries' smart devices? I expect there are oodles of sci-fi references to this already, but in particular I remember an audio-log from the game Horizon: Forbidden West, where a hacker is critiquing some shady corporate research he found, implying a project to weaponize domestic robots: > There's the Moldova brain-hack of course, but also up and coming little devils like the know-it-all MEMEr, or my personal favorite, Sovereign 7482. > Now that's an apex predator. Assuming control of them Ti-D-O bots and arming 'em with household appliances? Imagine tidying up after that! Gotta admit, it'd be fun to see 'em hunt in the wild.
- Rooster61 1y agoMy wife thought I was being crazy for not connecting our Roomba to wifi when we bought it. I feel quite vindicated.
- codetiger 1y agoLesson learnt: Best thing to do for a smart device is not connect them to internet from day one. Though it beats the purpose to some extend, we don’t have an option of buying dump devices anymore.
- theamk 1y agoWell, one of the big upside of the modern smart vacuums is you can see the map it built, set up rooms or zones, do a virtual cleaning, etc... This (1) definitely requires some connectivity, and (2) has to go via central server if the phone is not on same subnet. Sadly, because of (2), most (all?) companies don't bother with local connectivity at all. Much easier to debug one codepath (via remote server) rather than two (remote server and direct connection). So yeah, if you are worried about device being remote controlled by its manufacturer, don't buy devices which say "Can be remote controlled" right on the box. But of course then you are back to ancient tech, setting physical virtual wall devices or bounding the clean area with overturned chairs.
- dylan604 1y agowhat exactly is a virtual cleaning? i don't think it would be received well if I said I virtually cleaned something when asked
- theamk 1y ago:) virtual area cleaning, I've missed a word. This is when you only want to clean only left half of the room, for example. In the old robotics vacuums, your choices were either enough physical "virtual wall" units (those little stubby boxes you place on the floor) or physically fencing it, with overturned chairs or boxes or something. With the smart vacuums, you get the nifty map and you can draw the lines/boxes around the area you want cleaned. Very convenient, but naturally requires some sort of device with a screen (to show map) and position input (to draw the boundaries), like a phone.
- theamk 1y agoVery dramatic presentation for something very mundane. Every computer has "remote control" of some sort - if anything, to install security updates. Without security updates, there is a good chance your devices will turn into huge botnet at some point. I believe that EU CRA even requires such backchannel. I can agree, however, that refusing to work without internet is be too much for the device which can support offline operation.
- codetiger 1y agoRemotely triggered security updates are very common. But my experience in seeing remote command execution to disable a device is bit concerning. Having rtty software installed is another nightmare. Not sure if you call all these mentioned in the article mundane
- theamk 1y agoYou are letting it connect to manufacturer's servers, and allow it to execute unknown commands. You know that one command, "501", disables vacuum. There is a very good chance that there is some other remote command with "remote exec random command" functionality, you just didn't see it. There is also a good chance that there are already commands for any creepy things you might want to be worried about (like send camera video). So, given that, why are you worried about rtty specifically? It's likely a redundant debugging channel in case the main app crashes. It does not add any special functionality that main app does not have. Now re "disabling the device" - I wonder what command means? Could it be something like "local logs buffer full, pausing operation until upload is done"? Thinking about this more, your blog basically says: 1. vacuum works fine 2. you disable half of the ports on the firewall 3. vacuum stops working 4. you send it for warranty repair I was very surprised to see that 4 was "send it to warranty repair", instead of "re-open ports on firewall and see if it starts to work now". Did you try this? If not, then it's pretty likely the vacuum was not "bricked" in any sense, but rather was waiting forever for its logs to get uploaded.
- worik 1y ago> Very dramatic presentation for something very mundane In what way is this mundane? The writer purchased a device, and after purchase the device was remotely disabled. Terrifying - that it happened is alarming but that it is now "mundane" is utterly chilling
- blutack 1y agoLuckily it's supported by Valetudo so it can go back to work. https://valetudo.cloud/pages/general/supported-robots.html#ikohs https://valetudo.cloud/pages/general/supported-robots.html#i...
- altairprime 1y agoI initially skipped this comment as sarcasm; it’s not! For other readers, the context: Valetudo is a custom firmware project. > Cloud replacement for vacuum robots enabling local-only operation
- deleted 1y ago[deleted]
- laulis 1y agoNot a custom firmware.
- boomskats 1y agoCustom man-in-the-middleware?
- hypfer 1y agoCloud replacement. Middle would imply there being another end still.
- mcv 1y agoLocal cloud running on your vacuum cleaner.
- yencabulator 1y agoA dust cloud?
- 1y ago
- wrs 1y agoThe home office literally sshing into your appliance is a little surprising, but years ago when it came out that Tesla was doing this regularly to cars, I remember a lot of people saying this was common practice. (Maybe for things bigger than vacuums.)
- stevage 1y agoThe article is very vague on what actually caused the shutdown. Does he think a human triggered the kill command? Or the remote servers do this when they haven't heard from the device in a while? Or the device shuts itself down if it can't reach the servers?
- mindcrash 1y agoSeemingly remote kill command, very likely punishment due to not receiving telemetry because that particular address was blocked through firewall: "2024/02/29, 14:06:55.852622 [LogKimbo][CAppSystemState] Handle message! cmd_id 501 RS_CTRL_REMOTE_EVENT, len 8 serialno 0" Note something being named RS_CTRL_REMOTE_EVENT
- stevage 1y agoYes, but automatic or manual? If automatic, what exactly was the trigger? I'd have been tempted to explore this further - does sending fake or repeated telemetry satisfy it?
- stronglikedan 1y agoOT, but anything automatic was manually and intentionally implemented at some point
- hirsto 1y ago[dead]
- conductr 1y agoBut is it malicious or innocuous? I could see just the assumption being made that if it hasn’t phoned home it must be malfunctioning and ask risk mitigation then force it to brick. It’s not super unreasonable considering very few people will ever block the comms.
- 1y ago
- Zee2 1y agoThis is AI-written. - Ten em-dashes - "not just A, but B" - wasn’t just a vacuum cleaner; it was a small computer on wheels - they didn’t merely create a backdoor; they utilized it - they hadn’t merely incorporated a remote control feature. They had used it to permanently disable my device - incessant bullet points/markdown-style formatting - And an overly dramatic/promotional tone Obviously the image is AI as well, but /shrug
- hyperhello 1y agoI don’t see why you’re being downvoted.
- adastra22 1y agoBecause using good typography and editing is not a unique fingerprint of AI generated content.
- astrange 1y agoAI's habit of writing every story as a bullet-point list really isn't a good writing style.
- adastra22 1y agoIt was until AI started doing it everywhere. When I took technical writing course in college, many years ago, breaking up paragraphs with bullet-point lists was one of the core techniques taught for writing clear, effective documentation.
- schrijver 1y agoI don’t think there’s any single way to be sure, but it sure reads like ChatGPT to me. Which I’m not sure is such a bad thing—I presume the author used an AI to help them write the story, but the story is real. Or maybe they edited it themselves to make it sound more generic. Whatever the reason, the style takes away from my reading experience. It’s a blog post, I expect some personality!
- hypfer 1y ago> That was the moment my vacuum ceased functioning. The timestamp matched precisely with when it had stopped working, even though I hadn’t touched the app. > 2024/02/29, 14:06:55.852622 [LogKimbo][CAppSystemState] Handle message! cmd_id 501 RS_CTRL_REMOTE_EVENT, len 8 serialno 0 > Someone—or something—had remotely issued a kill command. Uuuuh are you sure that you're not reading a bit too much into the word "REMOTE" in that logline? These are some very strong accusations and opinions that to me don't feel like they're being backed up with equally strong evidence. At least not evidence that is part of that post. What even is a RS_CTRL_REMOTE_EVENT? Did you maybe check with e.g. Ghidra?
- hypfer 1y agoHaving thrown one version of everest-server of _a_ CRL-200S firmware (which might not be the one OP's firmware is running) into Ghidra and having found the string, this "REMOTE CONTROL" to me really does not look like it's executing remote commands. I mean it does, but not like shell commands but probably IR remote? The CRL-200S can be controlled via an IR remote, so it is possible that it saw something. The sun, perhaps? Feel free to prove me wrong on this of course.
- theamk 1y agoThat vaccum comes with cell phone app. It'd totally be reasonable to have "remote control event" for things like "start vacuuming" button pressed on an app.
- muppetman 1y agoI love the whole article hinges on a debug message he can't possibly know the real meaning it's worded like that.
- kazinator 1y agoYour room dust now is being sucked up without swirling up a cloud, which is well and good---yet somehow you can't write the story about it without AI, or host it without the cloud.
- stronglikedan 1y agoLouis Rossman would have a field day with this.
- manithree 1y agoUnauthorized Bread
- curtisblaine 1y agoThe "kill switch" thing ridiculous dramatization, but I wonder if these telemetry endpoints are open and, if they are, how damaging would be flooding them with plausibile but incorrect data for a sustained amount of time.
- RHSeeger 1y agoI'm surprised they didn't push back when they said it was out of warranty. They send it in for repair, and it was never fixed. They can either continue the process of trying to repair it, or refund the original cost of the device.
- Vaslo 1y agoI had a similar problem with Twinkly Xmas tree lights. It was even more frustrating than this because instead of just having me send the lights back, they just kept me trying all these different power plugs/checking with a voltmeter, etc. I should have asked myself “why in the f$&@ am I doing this” when clearly the lights were just dead. After the last correspondence I was busy and waited a month to respond and when I wrote them back they basically said it’s out of warranty, good luck. I pushed back and they couldn’t have cared less. All you can do is tell your friends and write a bad review. Good luck making it slip past all the positive fake reviews!
- mcv 1y agoThis kind of intentional remote bricking should be super illegal. I would really like to see a law that would allow the customer a full refund of the original purchasing price if the manufacturer remotely disables advertised functionality of a device for whatever reason. Because this kind of deceptive behaviour needs to be slapped down hard.
- missingcolours 1y agoMost "smart" devices simply don't function without connectivity back to the manufacturer's cloud, and this is basically just the same thing with extra steps.
- webstrand 1y agoIt clearly doesn't need the cloud, it intentionally bricks itself if it can't exfiltrate it's logs. It's not like it's sending data necessary for its immediate operation.
- theamk 1y agoSure it does need the cloud - you might have notice that the kill command was delivered _over cloud connection_. And author carefully blocked not entire connectivity, but only the part that they considered "logs". They wanted to keep cloud control, just not the whole thing. Given the complete lack of relevant technical details, it could be something as simple as "internal log storage full, refusing to start up until logs uploaded". We'd never know.
- thfuran 1y agoI think the extra step should be enough to technically constitute hacking and destruction of property.
- floren 1y agoOh, your XOURR smart vacuum got remotely disabled? Unfortunately the XOURR company has ceased operations so no refund is possible, but luckily there's a sale on IWEEET robot vacuums now, and I hear they're very reliable!
- ianferrel 1y agoThe author of this article: 1. Has the technical skills to disassemble this device, trace circuit boards, design his own boards and custom software to interface with components to substantially reverse engineer this device. 2. Is totally mystified when his internet connected device stops working after he blocks its communication, and rather than try unblocking it and seeing if it works again, sends it out for repair repeatedly. Something here doesn't add up. Tastes like bullshit to me.
- thfuran 1y ago>I began to feel like I was losing my mind. How could a simple IP block disable a vacuum cleaner that is supposed to work offline as well? It sure sounds like they were aware of the relation, just not how or why one thing led to the other.
- BoredPositron 1y agoYou are right the logical conclusion would be to send it for repair repeatedly.
- akerl_ 1y agoYea, I feel like this is the kind of thing that makes manufacturers resistant to open/hackable devices. I've done restrictive or invasive things to a variety of devices I own. But if something isn't working the way it should, "reset back to a clean default state and test again" always comes before trying to engage a warranty service process.
- hakfoo 1y agoThere's two possibilities there: The device might not be designed with a publicized tool to restore to a "clean state", and there's also the business signaling factor of "the device stopped working, so I will make sure sure it costs them money to handle a warranty claim".
- akerl_ 1y ago
- Gabrys1 1y agoMy guess would be that the manufacturer didn't remotely block the device, but rather the device itself did. If last connection time < N days ago and last M tries connecting were unsuccessful, then: brick myself. Still shitty, no doubt (and very similar to planned obsolescence), but the customer can un-brick by resetting to factory like they did in the service center.
- ivanjermakov 1y agoWhat is the benefit to anyone of it bricking itself?
- cptroot 1y agoFun article, but the repeated use of what I'd call "AI punch-up language" is a little distracting.
- codetiger 1y agoAuthor here: Out of all AI hater comments, yours is definitely worth considering. Ill try by best next time. :) Thanks
- crazygringo 1y agoThis makes no sense. He's insisting that they repeatedly remotely disabled his device in retaliation for blocking their data collection... ...yet they paid for the device to be shipped back and forth and inspected several times under warranty, presumably costing them $$$$? It makes zero business sense to break your customer's products intentionally, which will lead to 1-star reviews and expensive support. Plus, I hate to be the "this sounds like it was written by ChatGPT" guy, but this does. People don't write like this: > Deep within the robot’s startup scripts, I discovered the smoking gun. > It came back to life instantly. They hadn’t merely incorporated a remote control feature. They had used it to permanently disable my device. > I may have lost my warranty, but I won back my autonomy. Also, the idea that someone would waste months (?!) of their life on some broken vacuum cleaner until they "had a complete understanding of how the hardware was designed, down to each chip and wire connector" is just not real life. This is more like someone with a mental illness related to obsession, unless they're starting their own smart vacuum company. I'm guessing this is 100% fiction from ChatGPT. Complete with the AI-generated image.
- muppetman 1y ago100% agree. The whole is written in such a gushingly dramatic tone. No real technical details, just lots of alluded information. I don't believe a word of it.
- ivanjermakov 1y agoLucky you, my Roborock won't even load the map without mothership connection, not to mention full cleaning. Seems like all SLAM and pathfinding is done remotely, crazy.
- codetiger 1y ago[Update #1] Author here: I made some further progress. 1. Like a few comments mentioned here, the remote event happens for multiple reasons. Especially triggered by their app when you start or stop cleaning etc. I missed mentioning it in the article. The one I showed was suspicious as the device got bricked just after that event. 2. I fixed it by reseting the firmware and it worked for 2 days now without any issues. After second day cleaning it went back to charge as usual and never turned on. The bricking happened again this time, and I see a similar remote event again. This time am 100% sure there was no action from my side on the App or remote control. 3. What I found so far, after cleaning completes, the device uploads the map (in PNG) and some more data (in binary) to their server (There are very clear logs for this). After this upload, it receives a remote event and stops working. 4. How did I fix the device? I've the backup of all files in the device. Rebooting the device after replacing the files do the trick. 5. Now it reports the device is not on flat surface. Probably a loose connection with sensors which am yet to figure out. This time I suspect the bricking technique has changed, we shall be updating after more research Note: I am not sure if I can publicly share all techniques used to get access to the device. But it is very straightforward as mentioned in the article and very easy if you have some knowledge on adb (Android Debug Bridge) tools and a USB to micro usb wire.
- hulitu 1y ago> The day my smart vacuum turned against me A vacuum cleaner that doesn't suck ? Is this from Microsoft ?