4 ms·
Somewhat related to this ongoing Mozilla plugin saga, are there any infamous stories I should look up that involve huge mistakes leading to unfixable clients? I
by Waterluvian 8y ago
Somewhat related to this ongoing Mozilla plugin saga, are there any infamous stories I should look up that involve huge mistakes leading to unfixable clients? Ie. Imagine bricking your customer's devices with an irreversible buggy update. In Mozilla's case they were able to deploy a hotfix for many, and an update for others. But I imagine there's got to be some great stories about completely bricking countless devices.
- theclaw 8y agoTwo immediately spring to mind, Oculus in 2018 [0] and Zune in 2008 [1] [0] https://www.neowin.net/news/every-oculus-rift-vr-headset-bricked-due-to-expired-certificate https://www.neowin.net/news/every-oculus-rift-vr-headset-bri... [1] https://gizmodo.com/30gb-zunes-failing-everywhere-all-at-once-5121311 https://gizmodo.com/30gb-zunes-failing-everywhere-all-at-onc...
- deleted 8y ago[deleted]
- mraison 8y ago“My left shoe won’t even reboot” https://arstechnica.com/gadgets/2019/02/my-left-shoe-wont-even-reboot-faulty-app-bricks-nike-smart-sneakers/ https://arstechnica.com/gadgets/2019/02/my-left-shoe-wont-ev... I don’t know if they found a fix in the end
- tgtweak 8y agoIt's one of my favorite interview questions. I only know the answer because it happened to us once with 5M users on the affected version before it was discovered. It's a very good question to see how people react in a mostly hopeless situation. The only candidate out of 100s to logically get to the solution (or essentially it) was one of the best engineesr I've hired and worked with. "So you and your team just released a tested and reviewed build, did gradual rollout and numbers looked good - so out to general public it goes. All clients update over the following days. QA discovers, upon testing the next release, a latent bug that prevents the current version from connecting to the update service and running the updater. How do you resolve this?"
- willj 8y agoI'm super interested to know the answer to this question. I imagine it's more interesting than just uninstall/reinstall.
- judge2020 8y agoAs said in the other answer, hopefully the interviewee asks how it's unable to connect. If it's an Oculus situation [0] where the signing certificate is expired and you don't have a backdoor like Firefox studies, the only way is a reinstall or a "fix utility" [1]. 0: https://news.ycombinator.com/item?id=16541235 https://news.ycombinator.com/item?id=16541235 1: https://www.theverge.com/2018/3/8/17095414/oculus-rift-software-fix-certificate-expiry https://www.theverge.com/2018/3/8/17095414/oculus-rift-softw...
- xfitm3 8y agoI’d love to hear how this was resolved.
- dhimes 8y agoMy first thought is to email a link to a fix, including a reason and an apology.
- dannyw 8y agoI would find any source of displaying a message to the end user. Most applications have some data that comes from a remote server, whether it's on the home screen, any kind of announcements screen, server-sourced error messages, etc. I would use that to communicate an update is available, with a handy link.
- westmeal 8y agoStart screaming and rollback?
- Waterluvian 8y ago"I would give my two weeks notice." So how'd I do in the interview? ;)
- 8y ago
- tass 8y agoNot an update, but expired certificate on Wink hubs: http://status.winkapp.com/incidents/4c47cdvgl6b4 http://status.winkapp.com/incidents/4c47cdvgl6b4
- celrod 8y agoBLU phones released an update that made it impossible to unlock the phones. I factory reset mine rather than muck with downloading another update for the sd card. https://www.pcmag.com/news/357649/blu-phone-update-is-bricking-the-low-cost-android-phone https://www.pcmag.com/news/357649/blu-phone-update-is-bricki...
- dev_dull 8y agoPretty amazing you could even reset it. If that happened on iOS you’d be out of luck.
- throwawayfuuu 8y agoMobile apps broadly fall under this, because update cycles of apps can be very long. Storytime: App devs built their app with the debug flag on. (Why would anyone use CI anyways?) The debug flag is used for the local developer builds. All API endpoints were pointing to dev/stagings. In order to make this fuckup go unnoticed(otherwise customer retention would be awful + possible media coverage), the dev/staging endpoints were redirected to production, so that they would take the real orders from the broken builds. But we could only do this for our own APIs, not for our payment providers. So we had to ship product without getting paid. :|
- kristianc 8y agoPretty much this with some versions of Samsung's One Connect Mini box - the box (which is the only way to connect HDMI to many Samsung sets) connects to an update service, but during the process triggers a reboot which cuts the update short, which then starts the update again, which then triggers a reboot and... There appears to be no way to get the box back to a working state without replacing it entirely. I will never buy another Samsung set again. Shipping a TV where HDMI access depends on a proprietary box is unconscionable.
- m-p-3 8y agoImagine if this affected something like FirefoxOS devices and the lack of working certificate was making it unable to fix remotely? I know I'd be annoyed.
- Legogris 8y agoNot exactly clients, but the Parity multisig "hack"[0] was pretty wild. All it took was one permissionless call to permanently wipe the shared library of a large bunch of multisig wallets, thereby freezing those wallets forever. Some of the impacted people, including Parity, are still pushing for overriding the wallets to be unlocked, despite no support from anyone else. 0: https://blog.zeppelin.solutions/on-the-parity-wallet-multisig-hack-405a8c12e8f7 https://blog.zeppelin.solutions/on-the-parity-wallet-multisi...
- missblit 8y agoNot unfixable, but there was a fun story about a game that shipped without the ability to be patched, so the developers later "patched" it by buffer overflowing their own EULA: https://www.gamasutra.com/view/feature/194772/dirty_game_development_tricks.php https://www.gamasutra.com/view/feature/194772/dirty_game_dev...