7 ms·
> What happened — The incorrect camera being shown was once owned by the same customer. After the customer deleted this camera, there was a bug where we did not
by Reelin 7y ago
> What happened — The incorrect camera being shown was once owned by the same customer. After the customer deleted this camera, there was a bug where we did not completely clean up the user-device association for Alexa viewing. This resulted in the customer being able to view the camera from their Alexa device even though he does not own this camera anymore. Because the bug is related to device deletion from Alexa’s system, this bug ONLY impacts cameras that were Alexa enabled and then transferred to another account.
- davidhyde 7y agoWhat a PR disaster. It only takes one incident like this to tank a product. Testing is so important for IOT stuff but it usually gets shipped out way before it’s ready.
- mcbutterbunz 7y agoI would bet that reselling a camera was treated as a corner case and the ticket was probably sitting on the backlog somewhere.
- reaperducer 7y agoOr just dismissed outright. It's very common see FAANG employees on HN labeling user concerns "edge cases" and blowing them off.
- justinjlynn 7y agoThis angers me... edge cases are where people get cut... and badly! If anything, they should have more attention paid to them than less. Incentive structures inside these orgs are poorly designed to account for this, however. Trouble is, I don't know a good way to do modify them to do so without introducing even worse perverse incentives.
- chairmanwow 7y agoThis is definitely a corner case and the amount of such corner cases balloon into numbers difficult to boggle quickly. I agree this is an egregious miss, but absolutely understandable from an engineers perspective. I’m sure no one is happy with this outcome.
- justinjlynn 7y agoUpper/executive management certainly were happy... of course, until it bit the company in the ass. Engineering time not spent handling what would happen when a customer becomes not a customer is development/engineering time spent making features for sales to tout instead of making sure the product was safe for people they no longer thought they had a reason to care about. Reputational risk is really poorly handled because it's such a nebulous problem - and now, hopefully they'll see a massive drop in new customer acquisition and existing customer usage or they'll have absolutely no reason to care at all.
- burgerboy 7y agoWow dude your projects must have unlimited resources and just be perfect eh?
- bartread 7y agoIs it really that common? FAANG have millions to billions of users/customers. Can you at least cite some examples, please? If you're FAANG-like and have 100M users an edge case that only affects 1% of customers still affects 1M people. Something that affects 0.1% still hits 100,000. Even at 0.01% you've still infuriated a small town's worth of people. I'd hope that the average FAANG employee is smart/aware enough to realise that even edge and corner cases represent significant problems, even if all you care about is PR. (Although worth thinking about what might happen if you're Netflix and all these people decide to cancel subscriptions.) (Incidentally, there's one plausible and recent example I'm aware of: issues with Macbook Pro butterfly keyboards.)
- TeMPOraL 7y ago> Although worth thinking about what might happen if you're Netflix and all these people decide to cancel subscriptions. Probably not much. The thing is, from business POV, deprioritizing edge cases can be a rational choice. It depends on how much the impact of an edge case is amplified. At the very base, if you piss off 0.01% of your users, you lose 0.01% of your revenue, which is not much and can be ignored. However, if that 0.01% can get a Twitter shitstorm rolling to the point it reaches mainstream media, suddenly the damage may be far greater. WRT. examples - any time myself or someone else mentions that a piece of software is designed to prevent productive/efficient use of it, or loses such a feature, you can see responses that amount to "power users are small minority, why would one care about them". The annoying thing is (besides showing the disrespect to other people's time), this is a self-fulfilling prophecy. If you don't leave room for user to get better at using your software, they will never get better at using it and you won't have power users.
- bartread 7y agoThat's true w.r.t. power user examples, and I agree that it's infuriating, but OTOH I tend to assume the people making such comments are just parroting ideas. I'm certainly not convinced they're all FAANG employees, as suggested by the comment I was responding to.
- Scoundreller 7y agoHad a similar experience with an old set-top-box: a Boxee. The new owner couldn’t add Netflix to it until the old owner removed their “sync”, even though the old owner wasn’t a subscriber anymore!
- supergirl 7y agoreselling might not be common but returning is, at least in europe. you could install the camera but then decide to return it.
- jcims 7y agoEh, I think most people are desensitized. These Wyze cams are dirt cheap, small, look nice, I dont think its going to impact them at all. Higher end products or things like IoT door locks, maybe.
- Springtime 7y agoThis happened to someone with a Netgear Arlo [1] a while ago. Netgear responded to the user that resale isn't 'supposed' to occur and so didn't consider it an issue previously. Nevertheless they claim to have been working on a fix. Outside of various geeks though I'm not sure that word of most vulnerabilities with internet connected cameras reaches a wider audience. [1] https://www.reddit.com/r/privacy/comments/4ortwb/i_bought_and_returned_a_set_of_wifi_connected/ https://www.reddit.com/r/privacy/comments/4ortwb/i_bought_an...
- aw3c2 7y agoAnd rightfully so! Privacy is a right of utmost importance and for a surveillance product should be a cornerstone of its design.
- tgsovlerkhgsel 7y agoIt should, but I doubt it will have any significant impact, unfortunately. It pays to not give a shit about security to get the product out faster. See e.g. WhatsApp. They had monthly breaches but still got big, and then cleaned up their act. Had they tried to do it right from the start, a more aggressive and reckless competitor might have eaten their lunch.
- tinus_hn 7y agoPeople have been trained to accept ‘perfect security doesn’t exist’. Most people think they have nothing to hide anyway. They don’t care.
- bitttttten 7y agoSo let's say an account at Wyze "owned" your device, now they can see your private cams. This sounds like a backdoor.
- nexuist 7y agoThis is an odd accusation. How else are they supposed to implement access control? It's more of a front door than a back door. It's like being shocked that Facebook servers have access to your private albums. Where else are they supposed to keep that data?
- tialaramex 7y agoEverything involved could be encrypted with the keys simply re-randomised when somebody else takes ownership. The manufacturer doesn't need those keys, only the owner. The way they did it was easier, but as we see also more prone to screw-ups. But it was not the only, or best, way to do it.
- TaylorAlexander 7y agoI’m all for strong encryption, but I want to point out that user experience with strong encryption also leads to customer dissatisfaction (from what I have heard). It seems to me most companies give themselves a golden key in part so they can solve user issues. Most people aren’t demanding strong encryption but they do want easy app setup. I mostly avoid those devices because they aren’t from privacy focused companies. If I do webcams I’d put them on an isolated WiFi network and use a VPN to view data remotely, but that’s not something any normal consumer wants to do.
- skinnymuch 7y agoI doubt the vast majority of users want strong enough security and encryption that it stops companies from being able to debug and fix issues.
- ssss11 7y agoSo someone deleted their data and surprise surprise it wasn’t really deleted? And only got noticed because the tool stuffed up by using it. What’s the bigger issue the stuff up or the fact Google didn’t honour his deletion properly?
- ajross 7y ago> or the fact Google didn’t honour his deletion properly? Alexa is an Amazon product. And in this case it's not clear that Amazon could have known the device was transferred. They provide the authentication layer, but the authorization is, as I understand it, in the app.
- mdip 7y ago> And in this case it's not clear that Amazon could have known the device was transferred. Very fair statement. It's been a while but I've done a couple of these[0]. One comes to mind. The product was an internet connected home automation platform that predated the term "IoT". They had APIs with OAuth and their own system for managing layers of device pairing. It's a "hub"-like product; it could pair with devices for control and be controlled by things it is paired with. In our case, we didn't have to think about this specific problem because of the fact their product handled that (and there was no alternative but to let it), so the service we wrote did little more than pass the data from point A(lexa) to point C(uster API) and back, through point B(loat) which did nothing but alter the inputs/outputs according to what "A" or "C" insisted on. It wasn't that the problem "couldn't happen", it's that it couldn't happen in the code I was responsible for[1]. "C", as you stated, provided authorization -- it was the authority and the only component in the chain with enough information to cause/detect/remediate this sort of problem. With the additional context of there being a former/current ownership link at play here, it's a crazy common kind of bug -- everyone who's been on the "user" end of technology has experienced it[2]. I'm not surprised it happened. Quite disappointed. I mean, this isn't hindsight. Do designers of IoT cameras[3] think that their customers are going to be anything less than mobs-and-pitchforks enraged if unauthorized users are able to view it? Sure, it matters a little bit if it's broadcasting the interior of a bedroom versus a rarely changing picture of the doormat, but that's 10% knocked off the mob. It's still a mob. Handling ownership transfer correctly for privacy-sensitive devices (pretty much anything, anymore) should be the thought shortly after "let's connect a camera to the malware-net". Further, it's not an entirely unsolved problem (if imperfectly). You're always at the peril of your users, but you can make the process active rather than passive. I purchased a product that had the user side of this handled pretty well (easily the more difficult of the two problems -- "they'll make a better idiot"). It was nothing exotic -- simply applied "default deny" in a manner that fit with the problem. IIRC, it was a SmartThings Hub that I purchased this from my brother-in-law, who had connected in once, used it for a month, disconnected it and tossed it in a box. I did the hardware factory reset (he couldn't remember the password) and expected the usual routine you'd get with a router. No dice -- after factory reset, it defaulted to denying access until a device-specific code, which is printed on a card, stuffed in the manual, neither of which I had. I had to call their support line, provide information about the previous registered owner as well as a few values from the bottom of the device (guessing it's used to calculate the activation code). It was pretty solid as far as minimizing accidental disclosures like this -- even if the user forgets to factory-reset -- as a hub, it's going to reveal a password-protected interface that's protecting PII about the previous owner and configuration about a bunch of devices they owned that likely wouldn't work even if they were all sold together to the same person. The previous owner could control the hub with their password which would allow them to ... change their PII and see error messages that occur when he/she tries to control the irrelevant devices it's now configured for? That process isn't good enough for a camera (if it's wired, or previous owner/current owner used open wireless), it could be plugged in, DHCP itself an address and become available to the previous owner through their cloud account. Unfortunately, that extra step is costly. If the activation code is generated in a cryptographically sound manner (some combination of a hash of the values I had to provide signed with a private key by an HSM?), that's going to be added software development/infrastructure costs. Then there's handling idiot customers (like me) which will require some way to authenticate me -- and you're probably going to end up needing a call center to handle all of them, because all they see is that the damn thing they bought isn't working. But that's just it -- (other than us) at no point had it crossed the customer's mind that the camera could have a privacy "surprise". They see "Camera" and "For my home" and assumes any product sold at a major retailer must get the basic feature of "don't accidentally expose the customer's genitals" right[5] (joking, though Amazon's Echo Show is often marketed sitting where your bedroom alarm clock would be). Even more fun is that the customer is far more likely to encounter that activation problem than they are a "genital disclosure" problem and are unlikely to understand or appreciate that the little bit of grief is for a "good reason". The business side will be able to calculate the costs of implementing activation but will struggle with the cost of not implementing it. It also leaves room for a gamble. Many things we aim to prevent are things that never happen -- the software isn't interesting enough to attract research into attacking it or the right set of "bad things" didn't happen to discover a vulnerability accidentally. The circumstances are then: (1) Pay a lot of money for something that might not happen or (2) Save the money, cross our fingers. Sometimes the money's not there and (2) is the only option. But for a product like this (1) is the only viable choice -- a single high-enough profile incident for a small enough company is enough to eliminate the company and if the company is successful, it sells more devices, which exceeds the preventative capability of crossed fingers. [0] I inherited the voice applications that my company developed for others' commercial products - all but one extremely large deployments. [1] That's not passing the buck. We didn't have their code, access to their code, or any expectation that we could get it if we asked nicely. They gave us documentation to cover the calls that needed to be made to make it work. [2] And when it's presets on the used car you just bought, it's no problem. [3] While writing this, I looked over at my desk. Apparently I am ... well ... my parents are/will be Wyze owners. I purchased them a camera to provide a view of Lake Huron from the second floor bay window. Haven't used it, but it had the features I wanted (security wasn't one of them; it's meant for sharing and I'm isolating it with the rest of the IoT gremlins), they're inexpensive as heck but don't look tacky or cheap. [4] I'm going from memory - I purchased this years ago. [5] Two Mandatory Features for IoT cameras: (1) Records video customer can watch on any internet-connected device. (2) Ensures customers genitals appear only on authorized devices. The importance of that second feature cannot be underestimated. Might want to do (2) first -- I'd, personally, rather own a camera that took no video at all than one that was good at sharing videos all by itself.
- Too 7y agoThis is why you should never buy/sell electronics used for sensitive stuff like this second hand. Cases like this might seem bad but are still quite isolated and unlikely, imagine if the seller deliberately implanted malware, or buyer trying to get past your 2FA.
- pbhjpbhj 7y agoOr companies should be required to have a "factory reset" and warrant its removal of all customer data and associations (with heavy minimum compensations for failure).