14 ms·

Software bugs that cause real-world harm

217 points by thepbone 3y ago

fennecfoxy 3y ago
It's not even software bugs, just bad design. As technology has gotten more complex we haven't really kept up with quality, imo. For example Google home devices that I have, such as the nest displays and nest minis are absolute crap. At first they worked fine, but bad antenna design means that their wifi connection is absolutely terrible. The nest displays after operating for some time just turn into laggy messes. Both my Macbook and my Windows machine (gaming) have random issues that pop up now and then. Firefox has a longstanding bug where using location causes wifi SSID scans to be spammed endlessly resulting in periodic 10s lag spikes for all networking on the entire machine (& I'm sure Chrome has similarly bad bugs), fixed by doing a Firefox "refresh" where it keeps data but reinstalls itself. Pretty much all software lacks the options we want, in the places we want them to be and requirements can differ completely from person to person. It's a really tough problem to solve to be honest. Sure bugs can be fixed, but some bugs only impact 0.5% of users and some "bugs" are just bad UI/UX and are bugs to some people but not others.
hlieberman 3y ago
I talked about this several years ago[1], but I strongly believe that we, as a profession, don't invest nearly enough into thinking through all the possible consequences of the things that we design. It's easy to write off that what you're doing "doesn't matter" or "can't hurt", but the world is far too interconnected for us to be so nonchalant about our work. [1]: https://blog.setec.io/2015/11/01/ethics.html https://blog.setec.io/2015/11/01/ethics.html
- renewiltord 3y ago
  I think we do actually. In terms of economic value / accidental death this is probably the most rigorous engineering discipline. The big difference with software engineering that other engineering disciplines fail at is asking the question "Is it worth the risk?" and then answering it well. For instance, historically and presently, bridge builders accept a much higher risk of killing someone than software engineers. Given the risk that most bridge engineers take, most software engineers would instead just do something else.
- giantrobot 3y ago
  There's also the issue that code designed for one purpose often gets used for different purposes. You literally can't think through all of the consequences for a design because the total number of combinations is enormous. Even when you do think through all of the possible combinations you can't know the runtime state of all of those combinations. Formal verification of software exists but can only really be trusted if running on hardware with some multiple redundancy and formal verification of its own.
- keyringlight 3y ago
  Mike Monteiro did a talk [1] along these lines - "How Designers Destroyed the World". [1] https://www.youtube.com/watch?v=qIcM21l61TE https://www.youtube.com/watch?v=qIcM21l61TE
Zealotux 3y ago
Sometimes, even perfectly functional code can do harm as well: https://www.freecodecamp.org/news/the-code-im-still-ashamed-of-e4c021dff55e https://www.freecodecamp.org/news/the-code-im-still-ashamed-... https://news.ycombinator.com/item?id=12965589 https://news.ycombinator.com/item?id=12965589
- lostlogin 3y ago
  Philips sell MRI scanners. I used to do angiograms with them. My workflow was to start a cheap and cheerful scan that showed the location of the injected contrast. As it got close to the vessel of interest I’d click and hold the button that started the high resolution angio run. When it was at the vessel of interest I’d lift my finger off the button and the scan would start. This all happened over about 10-20 seconds after the injection. Blood moves quickly. Start the scan too early and you miss everything, too late and you get veins rather than arteries. In a point release update with no mention in the release notes they changed the behaviour of the scanner - it then started the scan on clicking down in the button, not on lifting off. A time critical scan on a patient due for theatre was messed up by me. Their renal function was poor and the scan couldn’t be repeated for a few days.
  - renewiltord 3y ago
    Great story. It's interesting that regulatory barriers create these large companies that have no user interest at heart. Bundled software and hardware from a hardware company: guaranteed disaster. Perhaps only Apple has succeeded at this.
    - klysm 3y ago
      It’s unfortunate because the potential value in vertical ingestion is quite large, as apple have been demonstrating for some time.
  - sitkack 3y ago
    That sounds like a Boeing playbook. Changing action on button down vs up is a HUGE deal.
hoosieree 3y ago
I teach engineering ethics and while the example NSPE cases[1] skew more toward civil engineering, there are some cases relevant to software and computer engineers. But I find it helpful to draw examples from current events. For example, use of facial recognition tech in policing. TV and movies give the impression that a satellite can identify the perp with 99.4% accuracy if you yell "enhance" enough times. Meanwhile, in reality we're happy when our classifier can tell a "3" from a "B" in a real image. [1] https://www.nspe.org/resources/ethics/ethics-resources/board-ethical-review-cases https://www.nspe.org/resources/ethics/ethics-resources/board...
- JdeBP 3y ago
  There's also the decades-long tale of the software bug that put some U.K. sub-postmasters in prison, bankrupted others, and caused a few to commit suicide.
  - dm_me_dogs 3y ago
    For anyone wanting more context: https://en.wikipedia.org/wiki/British_Post_Office_scandal https://en.wikipedia.org/wiki/British_Post_Office_scandal
- bigbacaloa 3y ago
  There are engineering ethics courses?
  - WalterBright 3y ago
    There was an article 3 days ago about rampant cheating in an ethics class. https://news.ycombinator.com/item?id=36082650 https://news.ycombinator.com/item?id=36082650
  - 0xffff2 3y ago
    It's a single one-term course in most universities I believe, but engineering ethics is a required subject for ABET-accredited (the standard in the US) engineering degrees.
superice 3y ago
Story time! When I was a much more junior programmer I once received a call from a client, who asked me if I could double check the code sending the container weights from a container terminal to the ship planning system. She explained a captain of one of the ships had called the planners and remarked he was both deeper in the water and tilted quite a bit more than he anticipated. So I do my research, and it turns out we were sending weights including the metal of the container itself from the one system, but interpreting them as net weights, so excluding container weight itself. So we were off by about 3000kg per container, which is bad enough if the container is 25000kg total, but even worse when transporting empties, where we were off by 100%. Thank goodness it is drilled into captains that there are strict limits when it comes to stability, and I have never met a captain who will depart if they are not absolutely positive about the stability of their ship, but man I have spent a good few nights lying awake thinking about what could have happened. I thought I was just working on some boring logistics software at the time, where the worst that could happen was losing a container for a day or so. It was a rude awakening.
- SoftTalker 3y ago
  > I have never met a captain who will depart if they are not absolutely positive about the stability of their ship Guess you have not dealt with air freight pilots in the third world or bush country?
  - 6D794163636F756 3y ago
    Or that's why they wanted the check. They felt like it was a noticable problem so they got someone to look
  - BalinKing 3y ago
    I had interpreted the GP as specifically talking about maritime captains, which I imagine could be a very different boat (pardon the pun).
    - Aerbil313 3y ago
      Can someone please explain why native English speakers always feel the need to indicate and express the pun? Literally every single time I see “pun intended” or “no pun intended” I stop for a while and try to understand why the author felt the need to wrie that.
SoftTalker 3y ago
Good post, but the second story about the Android phone was more a complaint about desired features that the phone didn't provide, more than a bug. I'd put that in a different category. Software can't be everything to everyone.
- TeMPOraL 3y ago
  On the other hand, this technology has been around for 30 years. The feature in question reaches back to the earliest mobile phones. Not smartphones - think earlier than Nokia 3310. On the gripping hand, "silence.mp3" has been the solution for the author's "mom problem" for a good 20 years too. I hope they don't mean to imply you can't assign individual ringtones to contacts on stock Android? Even the K800i I had as a teenager supported that. On the yet another hand (we're moving fast into octopod territory here), for a moment here I was wondering what the author was smoking with the "ringtone & notifications" complaint. I don't remember seeing an Android phone that did not have separate volume slider for each. Then I realized, I never actually owned a stock Android phone. Still, to support the author's point against Google specifically, Samsung managed to not have this problem for at least 10 years now.
  - SoftTalker 3y ago
    My silence.mp3 story on Android was this: I was using an android device to play background music through a PA during breaks in activity. Every time I'd start a song there was a "pop" I assume by the audio output amplifier switching on or waking up. My solution to avoid this was to play "silence.mp3" in a repeat loop in another app, and then my music app would play over that and there were no pops when I started a song. IDK if the pop problem has been fixed, or whether it was a software or hardware issue.
  - maxime_cb 3y ago
    > I was wondering what the author was smoking with the "ringtone & notifications" complaint. I don't remember seeing an Android phone that did not have separate volume slider for each. This is why I included a screenshot of the volume sliders my Google Pixel displays. Samsung doesn't use stock android OS, and some Samsung users tend to assume every android phone works the same.
RobotToaster 3y ago
The infamous bug(s) in the Fujitsu system provided to the British post office caused several suicides. https://en.wikipedia.org/wiki/British_Post_Office_scandal#Horizon_computer_system https://en.wikipedia.org/wiki/British_Post_Office_scandal#Ho...
- goda90 3y ago
  >The Post Office resisted the SPMs' reports of faults in the system, insisted that the SPMs make up any shortfall of money and, when asked by an SPM, denied that other SPMs had reported problems. Seems more like a very toxic culture of shifting blame led to those suicides, with the bugs just being a catalyst.
  - yifanl 3y ago
    The bugs were causing very real financial distress it sounds like, if the software was partially responsible for determining their continued employment
    - nitwit005 3y ago
      Normal people would see dozens of staff members reporting the same issues as a reason to get the bugs fixed. Some of the stuff that came to light in that scandal was so extreme, that it's clear the leadership was a bit deranged. Crazy people produce crazy results.
- stephen_g 3y ago
  Crazy how they just refused to believe (or properly assess whether it was true) that the software could have any issues, despite having reports of errors starting very soon after it went in. I find it extraordinary that they secured convictions despite the fact there wouldn’t have been any unexplained wealth or any money popping up elsewhere - although it seems in a lot of cases solicitors basically convinced the postmasters to make false confessions (kind of like plea bargains I guess).
- GoblinSlayer 3y ago
jmclnx 3y ago
>The screen on my phone showed that the alarm I had set was in the process of ringing, but for some reason, the phone wasn’t vibrating or making any sound. And this is kind of an example of why it will be impossible to mitigate Climate Change. When I was young, clocks did not requite any power from the grid. It wound them, set the alarm and you heard it. Now we have faucets, toilets that use power to turn on/off and flush. Almost everything that was manual (mechanical) is now powered. That power needs to come from somewhere. In these examples, there is no real reason to have faucets, toilets and alarm clocks connected to the grid. Yes, I know in the case of the Cell Phone Alarm, it is probably a wash, but a mechanical clock in your house is probably a good thing to have.
- jrockway 3y ago
  It's most likely to me that many more people have overslept because they forgot to wind their clock than have overslept because their iPhone hit a weird bug where the alarm sound didn't play. Simply because mechanical clocks have been around much longer.
  - SoftTalker 3y ago
    Also at a certain time in my teenage years I could sleep through a ringing alarm clock with absolutely no awareness of it. The spring for the bell-ringing mechanism would unwind after maybe 30 seconds or so, and the ringing would stop, and I'd sleep completely through it. Not sure the powered alarms would have done any better, but maybe after 10 minutes of ringing it might have woken me up.
- alpaca128 3y ago
  > a mechanical clock in your house is probably a good thing to have. There are much better and more reliable options. Solar powered watches that can run months without recharging, some digital watches run a decade with a single battery. A mechanical watch, on the other hand, has to be wound regularly and is usually far less accurate. I like automatic mechanical watches but I don't think they can compete in any technical aspect, that's not the reason one would use them nowadays.
- 3y ago
kernal 3y ago
>Unfortunately, Android [9.0] suffers from poor user interface design in a few areas, and one of the most annoying flaws in its user interface is simply that the stock Android OS doesn’t have flexible enough options when it comes to controlling when the phone rings, which is one of the most important aspects of a phone. Do Not Disturb, at the time of Android 9, did support controlling when the phone rings. >Bedtime mode is quite useful, but I still have the other problem that my mom could decide to randomly call me in the daytime as well, and unfortunately I rarely want to take her phone calls. However, I also don’t want her to end up homeless or in jail (which has happened before, but that’s a story for another time), and so I don’t want to block her and completely lose the ability to receive her calls. This results in me having to almost always have my phone set to “do not disturb”, so that I don’t have to be disturbed at random times by unwanted phone calls Could he not have used a custom ringtone for his mother?
- pw6hv 3y ago
  *she
grepLeigh 3y ago
Great name for this kind of blog (pointersgonewild) > Long story short: a software bug caused the machine to occasionally give radiation doses that were sometimes hundreds of times greater than normal Oh my god. I have 15+ years working with "mission critical (software) infrastructure" and thought my job was important/hard because I played shepherd for production database fleets. Certain kinds of mistakes could bankrupt the business, so I had to make systems resilient to human errors. Today, I run a 3D printer software startup. Thinking deeply about the safety mechanisms I need to control machines running at 250-300°C. Precision errors when shooting radiation at a person is a whole different level. Wow.
- masklinn 3y ago
  The Therac-25 story is pretty wild, especially as it was a reuse (with modification) of an existing codebase, but for cost savings the -25 was designed without hardware interlocks, under the assumption that the software would run fine, as it had run fine on the -20. Turns out the software had run fine on the -6 -20 because of the mechanical interlocks which prevented it from critically fucking up, but there was no reporting built into the software interlocks, so no way to know when they'd triggered. And that was before additional modifications were added for the -25.
  - anonymousiam 3y ago
    Not all that different from the Ariane-5 maiden launch, except for human lives being at stake vs. billions of Euros (adjusted for inflation). https://en.wikipedia.org/wiki/Ariane_flight_V88 https://en.wikipedia.org/wiki/Ariane_flight_V88
    - nirui 3y ago
      Well, for me, the level of anxiety is definitely different. I somehow have a few degrees of fear towards machines that could emit harmful level of ionizing radiation, and I can be nervous if the machine is operating on me (say doing X-Ray at the dentist). Plus, I also have some anxiety when handling concurrent programming, which according to the Wikipedia is what caused the error in Therac-25. So I guess I got double anxieties on that story.
OnlyMortal 3y ago
A story related to me from a friend… A torpedo system was designed not to hit the submarine that fired it. It would detonate if it was aiming itself back. So, testing came along, the torpedoes armed and… the sub turned itself around. Bang.
- meghan_rain 3y ago
  Sorry, I don't understand what happened?
  - isidor3 3y ago
    Sounds like the torpedoes were armed, but not yet fired. So the sub itself turning around in a circle triggered the self-destruct mechanism while the weapons were still in the tubes.
  - orbz 3y ago
    I think the implication is that the torpedos had not yet left their tubes, so when the sub turned around with them still in it they registered that as a situation where they should destruct.
awesome_dude 3y ago
I'm surprised that the Metric <-> Imperial bug that caused NASA to lose 125 million wasn't mentioned https://en.wikipedia.org/wiki/Mars_Climate_Orbiter https://en.wikipedia.org/wiki/Mars_Climate_Orbiter
- contingencies 3y ago
  It's certain there is >$10B loss per annum on metric-imperial. Think about it: human error, conversion process losses, training materials, advertising materials, two physical versions of everything being produced, two physical versions in the supply chain... I for one plan to operate a metric shop in the US. Using imperial outside of supplier interface will be a formal warning. Will see how that goes...
soult 3y ago
It's interesting that the author picked the home depot delivery as an example, because I don't think it was caused by software bugs at all. The delivery notifications right at the end of the delivery slot happened because the driver has an unrealistic schedule, and to hit their metrics, they will mark deliveries as "delivered" rather than missing an assigned delivery slot. And that the expensive two-man-handling and delivery to the apartment service got "lost" between seller and subcontractor is more likely cost optimization and banking on most customers not complaining hard enough to actually force them to provide the paid-for service. "Software bug" being a convenient excuse the subcontractor can give to the rightfully enraged customer.
- maxime_cb 3y ago
  Author here. The reason I tend to think it's a software bug is that it seems that the system to dispatch deliveries is automated. The subcontractors get their orders from some kind of computerized system, it seems. That system seems to systematically fail to specify when they are to carry items indoors/upstairs. Whether that's due to negligence or intentional malfeasance, don't know. What I do know from experience is that there are numerous bugs on their website, besides the "unknown error" problem I've listed. It just seems like really shittily built software... So I would tend to think there's an issue with really poor software engineering practices at that company.
  - abnercoimbre 3y ago
    I loved your article. You might consider joining the Handmade Network [0] and related conferences that I run [1] because this is precisely the sort of behavior we want to curb in our industry. Just know some of us are fighting this. [0] https://handmade.network https://handmade.network [1] https://handmadecities.com https://handmadecities.com
    - maxime_cb 3y ago
      Looks pretty cool :)
kevin_nisbet 3y ago
I think my message is, don't lose sight on the mission of the software that you're shipping. My story is, I found a bug in some new equipment that would've broken 911 calls. While other engineers were just trying to reboot random equipment to make the problem go away, I insisted we pause to figure it out. Turns out, due to a couple of bugs, the new network equipment could only handle a couple of 911 calls before failing. This was for a national cellular network... so 911 was kind of important.
- Aperocky 3y ago
  Did you get recognition for recognizing the problem and dealing with it?
  - klysm 3y ago
    Going to go out on a limb and say no because it didn’t generate revenue
- eastbound 3y ago
  Were you working on Android? It is famous for failing to call 911 in emergency situations.
- c4ptnjack 3y ago
  Additionally, I assume this was on the cellular network completely and not on the dispatcher/rf side of things?
anonymousiam 3y ago
Well written article, but most of the "harms" described (aside from the Therac-25) are a direct result of attempting to replace human-to-human contact with technology. 1) Instead of relying on a phone alarm to wake you up, use a device designed for that purpose, or request a wake-up call (on a landline, if you have one). 2) Overcome the conflict between having a phone and not wanting to answer it. 3) When unusual and specific instructions are required, make sure you speak with a person to confirm them.
- groestl 3y ago
  And Therac-25 was trying to remove hardware interlocks from Therac-6 and Therac-20, to fully rely on software, AFAIK.
- tczMUFlmoNk 3y ago
  I agree with you that the trajectory of removing human-to-human contact, even as a fallback, is causing harm. And I agree with your interpretation of your point (3): having an option to speak to a human would have gone a long way here. I'm not sure that I agree with your point (1): a modern smartphone is a device designed for that purpose, among other things, and it is virtuous to use a multi-purpose device for this instead of building, transporting, and disposing of a dozen bespoke devices for each such purpose. But reducing the complex interpersonal relationship between the author and her mom to just "overcome the conflict" is a really low-bar take. It feels callous to suggest that it would be so easy, or to assume that she's not trying to do this, or even that this is the right path forward for her. Technology should serve us and adapt to the diversity of our needs, as we change as people and those needs change with us.
- taneq 3y ago
  I don’t think “software is unreliable and you should never trust it” is the outcome we should be hoping for here. If I burn to death in a Ford Pinto, would you tell me I should have ridden a horse?
  - intelVISA 3y ago
    Depends, was the horse sold to you with a potentially fatal, albeit financially acceptable, defect?
sitkack 3y ago
Everything is mission critical to someone.
meghan_rain 3y ago
> Thankfully, Android now has “bedtime mode” feature, which allows me to make it so that phone calls won’t cause my phone to ring between 10AM and 8:30AM. If my mom happens to die in a hospital in the middle of the night, I’ll just have to find out and be sad the next day. wtf
- usui 3y ago
  Is this a comedic "wtf"? If nt, I think you should recognize the flippant tone, not to be taken too seriously.
deleted 3y ago
[deleted]
acenes 3y ago
A recent example: https://www.nytimes.com/2023/05/26/science/moon-crash-japan-ispace.html https://www.nytimes.com/2023/05/26/science/moon-crash-japan-...
deanCommie 3y ago
> I wish that Android had an option to set a specific person to never cause the phone to ring, and it seems like that should be an easy feature to implement that would have a real positive impact on the quality of lives of many people, but I digress. FWIW, Android does have that feature - at least for sure the Pixel Launcher (that the author uses) does: You can set some contacts as "Favourites" and have it configured that Favourites get to bypass Do Not Distrurb. Good blog nonetheless!
- smabie 3y ago
  That's not the same feature tho
  - deanCommie 3y ago
    My mistake. I thought the author wanted to make sure her mother wouldn't wind up dead or in jail. Instead she is unclear about what she actually wants: > I still have the other problem that my mom could decide to randomly call me in the daytime as well, and unfortunately I rarely want to take her phone calls. However, I also don’t want her to end up homeless or in jail (which has happened before, but that’s a story for another time), and so I don’t want to block her and completely lose the ability to receive her calls. So...does she want her mom's phone calls to get through, or doesn't she?
anonymousDan 3y ago
If you want to see a really sad example, check out the Royal Mail/Fujitsu postmaster scandal in the UK, where postmasters (franchisees) ended up being thrown in jail, losing everything they owned, families destroyed, suicides, etc. All because of bugs in Fujitsu accounting software (https://www.bbc.co.uk/news/business-56718036 https://www.bbc.co.uk/news/business-56718036)
pards 3y ago
Knight Capital [0] should serve as a warning to devs and ops alike. They lost about $450 million in about 30 mins due to some poor coding, a botched deployment, and botched rollbacks. [0]: https://en.wikipedia.org/wiki/Knight_Capital_Group https://en.wikipedia.org/wiki/Knight_Capital_Group
joezydeco 3y ago
Almost any and every story you'll see in this thread has already been collected and discussed in the RISKS mailing list and archive. There's nearly 38 years of articles waiting for you to peruse, all lovingly curated by Peter Neumann. https://catless.ncl.ac.uk/risks/ https://catless.ncl.ac.uk/risks/
Ubergeek99 3y ago
But the biggest "real world harm" is that the mother is able to call her (the author). What kind of world we live in that schizophrenic parents can get ahold of us? That really is a big bug. Apple and Android should do something about that... before it's too late and irreversible harm is done.
- maxime_cb 3y ago
  Author here. If you've grown up in a "normal", functional family, with two loving parents, and you enjoy talking to them on the phone, you should consider yourself very lucky. I only take calls from my mother when I feel up to it. The reason for that is that she has no concept of boundaries. Her mental illness prevents her from grasping that concept. Her default behavior tends towards what a normal person would call harassment. If you can't relate or understand a situation like that, it might just be because you've had a relatively safe, coddled, privileged life.
brokenmachine 3y ago
https://wikipedia.org/wiki/British_Post_Office_scandal https://wikipedia.org/wiki/British_Post_Office_scandal
etimberg 3y ago
Surprised the Patriot missile failure during the Gulf War wasn't mentioned https://en.wikipedia.org/wiki/MIM-104_Patriot#Failure_at_Dhahran https://en.wikipedia.org/wiki/MIM-104_Patriot#Failure_at_Dha...
WalterBright 3y ago
> if you’re a software engineer and you’re working on safety-critical systems, you absolutely must do due diligence and implement proper validation and testing, otherwise you could be putting human lives at risk. I'll be blunt. This is the wrong lesson. The correct lesson is: Assume the software will do the worst thing possible. How will you prevent that from causing a catastrophe? I.e. make the Therac-25 physically incapable of generating an excessive dose. This is how airplanes are made safe. Not by imagining you can create perfect hardware or software, but by assuming any part can fail, and how does the airplane continue to fly safely?
- WalterBright 3y ago
  > A malfunctioning alarm clock probably isn’t going to directly cause a person’s death, but it definitely has the potential to cause real-world harm. When I absolutely have to get up on time, I set two alarm clocks. When traveling, I'll set my phone, and also arrange for a wakeup call. I learned about having an independent backup from working at Boeing engineering.
  - interfixus 3y ago
    That was my thought, too. Day to day, my phone alarm is fine (and I damn well tested it thoroughly before actually relying on it), but if anything life-and-death is scheduled, I'll use at least one independent extra device. And all this even though I always wake up on time and never am late. Likewise, the phone call problem has an easy solution: Assign different sounds to different callers; low and unobtrusive to the unwanted ones. And the delivery: Never trust such a thing without sufficient fallbacks. If you want safety and full control, organise the thing yourself. All in all, I get a vibe that the poster may be a wonderful coder, but really hasn't got the paranoid mindset required for mission critical work.
  - deleted 3y ago
    [deleted]
- physicles 3y ago
  > Assume the software will do the worst thing possible. That's certainly a valuable lesson when there's hardware and software involved, and the consequences of the software screwing up are potentially lethal. But what about the case of the home depot shopping cart, or the iPhone alarm? In those cases there's only software. "Assume the software will do worst thing possible" doesn't make sense.
jmholla 3y ago
> [On Android] why in the fuck can’t there simply be two separate fucking sliders for “Ring volume” and “Notification volume”? Are there not for everyone? On every Android phone I've had in recent memory I've had his feature. I'm certain I have it on my current phone because I just looked. In fact, they're called exactly what the author refers to them as.
- dvlsg 3y ago
  I have one slider for "ring & notification volume" on mine. I also have media, call, and alarm volume as 3 other sliders, for a total of 4.
  - jmholla 3y ago
    Interesting. I'm on a stock Motorola and they are separate. I have five sliders.
    - Suppafly 3y ago
      maybe newer motorolas have customized it, but on mine i have media,call, ring, and alarm.
xupybd 3y ago
I disagree with the author, all of those bugs they list cause people to get upset. Not harm. You are talking about the cost of quality. Software quality comes at a cost. It's up to the business to determine how much it will spend on quality. If we had a zero bug policy in non safety critical applications the cost of goods would be too high.
physicles 3y ago
I really appreciate the author shining a lens on domains other than aviation, health care, etc. There's plenty of potential for software to do harm in everyday domains. The rage I feel when software that I use breaks, I take that rage and channel it into customer empathy. If my stuff causes someone to get annoyed, or lose a lot of time, or both, that really sucks and it needs to be fixed. I wish we could do more of that in this industry. I got bit by that iPhone alarm bug a couple times. Couldn't believe something like that would slip into production and stay there for as long as it did.
Fragoel2 3y ago
We (me and co-authors) recently published a study in which we found some examples of real-world harm in user reviews of IoT devices (https://www.sciencedirect.com/science/article/pii/S1574119223000445 https://www.sciencedirect.com/science/article/pii/S157411922...). These devices are supposed to be extremely pervasive in the future, hence we need to do better to ensure their safety.
saint-loup 3y ago
There's quite an interesting book on the topic of real-wold harm, mainly from an user interface point of view. https://www.tragicdesign.com/ https://www.tragicdesign.com/
deleted 3y ago
[deleted]

217 points by thepbone 3y ago

It's not even software bugs, just bad design. As technology has gotten more complex we haven't really kept up with quality, imo. For example Google home devices that I have, such as the nest displays and nest minis are absolute crap. At first they worked fine, but bad antenna design means that their wifi connection is absolutely terrible. The nest displays after operating for some time just turn into laggy messes. Both my Macbook and my Windows machine (gaming) have random issues that pop up now and then. Firefox has a longstanding bug where using location causes wifi SSID scans to be spammed endlessly resulting in periodic 10s lag spikes for all networking on the entire machine (& I'm sure Chrome has similarly bad bugs), fixed by doing a Firefox "refresh" where it keeps data but reinstalls itself. Pretty much all software lacks the options we want, in the places we want them to be and requirements can differ completely from person to person. It's a really tough problem to solve to be honest. Sure bugs can be fixed, but some bugs only impact 0.5% of users and some "bugs" are just bad UI/UX and are bugs to some people but not others.

hlieberman 3y ago

I talked about this several years ago[1], but I strongly believe that we, as a profession, don't invest nearly enough into thinking through all the possible consequences of the things that we design. It's easy to write off that what you're doing "doesn't matter" or "can't hurt", but the world is far too interconnected for us to be so nonchalant about our work. [1]: https://blog.setec.io/2015/11/01/ethics.html https://blog.setec.io/2015/11/01/ethics.html

renewiltord 3y ago
I think we do actually. In terms of economic value / accidental death this is probably the most rigorous engineering discipline. The big difference with software engineering that other engineering disciplines fail at is asking the question "Is it worth the risk?" and then answering it well. For instance, historically and presently, bridge builders accept a much higher risk of killing someone than software engineers. Given the risk that most bridge engineers take, most software engineers would instead just do something else.
giantrobot 3y ago
There's also the issue that code designed for one purpose often gets used for different purposes. You literally can't think through all of the consequences for a design because the total number of combinations is enormous. Even when you do think through all of the possible combinations you can't know the runtime state of all of those combinations. Formal verification of software exists but can only really be trusted if running on hardware with some multiple redundancy and formal verification of its own.
keyringlight 3y ago
Mike Monteiro did a talk [1] along these lines - "How Designers Destroyed the World". [1] https://www.youtube.com/watch?v=qIcM21l61TE https://www.youtube.com/watch?v=qIcM21l61TE

Zealotux 3y ago

Sometimes, even perfectly functional code can do harm as well: https://www.freecodecamp.org/news/the-code-im-still-ashamed-of-e4c021dff55e https://www.freecodecamp.org/news/the-code-im-still-ashamed-... https://news.ycombinator.com/item?id=12965589 https://news.ycombinator.com/item?id=12965589

lostlogin 3y ago
Philips sell MRI scanners. I used to do angiograms with them. My workflow was to start a cheap and cheerful scan that showed the location of the injected contrast. As it got close to the vessel of interest I’d click and hold the button that started the high resolution angio run. When it was at the vessel of interest I’d lift my finger off the button and the scan would start. This all happened over about 10-20 seconds after the injection. Blood moves quickly. Start the scan too early and you miss everything, too late and you get veins rather than arteries. In a point release update with no mention in the release notes they changed the behaviour of the scanner - it then started the scan on clicking down in the button, not on lifting off. A time critical scan on a patient due for theatre was messed up by me. Their renal function was poor and the scan couldn’t be repeated for a few days.
- renewiltord 3y ago
  Great story. It's interesting that regulatory barriers create these large companies that have no user interest at heart. Bundled software and hardware from a hardware company: guaranteed disaster. Perhaps only Apple has succeeded at this.
  - klysm 3y ago
    It’s unfortunate because the potential value in vertical ingestion is quite large, as apple have been demonstrating for some time.
- sitkack 3y ago
  That sounds like a Boeing playbook. Changing action on button down vs up is a HUGE deal.

hoosieree 3y ago

I teach engineering ethics and while the example NSPE cases[1] skew more toward civil engineering, there are some cases relevant to software and computer engineers. But I find it helpful to draw examples from current events. For example, use of facial recognition tech in policing. TV and movies give the impression that a satellite can identify the perp with 99.4% accuracy if you yell "enhance" enough times. Meanwhile, in reality we're happy when our classifier can tell a "3" from a "B" in a real image. [1] https://www.nspe.org/resources/ethics/ethics-resources/board-ethical-review-cases https://www.nspe.org/resources/ethics/ethics-resources/board...

JdeBP 3y ago
There's also the decades-long tale of the software bug that put some U.K. sub-postmasters in prison, bankrupted others, and caused a few to commit suicide.
- dm_me_dogs 3y ago
  For anyone wanting more context: https://en.wikipedia.org/wiki/British_Post_Office_scandal https://en.wikipedia.org/wiki/British_Post_Office_scandal
bigbacaloa 3y ago
There are engineering ethics courses?
- WalterBright 3y ago
  There was an article 3 days ago about rampant cheating in an ethics class. https://news.ycombinator.com/item?id=36082650 https://news.ycombinator.com/item?id=36082650
- 0xffff2 3y ago
  It's a single one-term course in most universities I believe, but engineering ethics is a required subject for ABET-accredited (the standard in the US) engineering degrees.

superice 3y ago

Story time! When I was a much more junior programmer I once received a call from a client, who asked me if I could double check the code sending the container weights from a container terminal to the ship planning system. She explained a captain of one of the ships had called the planners and remarked he was both deeper in the water and tilted quite a bit more than he anticipated. So I do my research, and it turns out we were sending weights including the metal of the container itself from the one system, but interpreting them as net weights, so excluding container weight itself. So we were off by about 3000kg per container, which is bad enough if the container is 25000kg total, but even worse when transporting empties, where we were off by 100%. Thank goodness it is drilled into captains that there are strict limits when it comes to stability, and I have never met a captain who will depart if they are not absolutely positive about the stability of their ship, but man I have spent a good few nights lying awake thinking about what could have happened. I thought I was just working on some boring logistics software at the time, where the worst that could happen was losing a container for a day or so. It was a rude awakening.

SoftTalker 3y ago
> I have never met a captain who will depart if they are not absolutely positive about the stability of their ship Guess you have not dealt with air freight pilots in the third world or bush country?
- 6D794163636F756 3y ago
  Or that's why they wanted the check. They felt like it was a noticable problem so they got someone to look
- BalinKing 3y ago
  I had interpreted the GP as specifically talking about maritime captains, which I imagine could be a very different boat (pardon the pun).
  - Aerbil313 3y ago
    Can someone please explain why native English speakers always feel the need to indicate and express the pun? Literally every single time I see “pun intended” or “no pun intended” I stop for a while and try to understand why the author felt the need to wrie that.

SoftTalker 3y ago

Good post, but the second story about the Android phone was more a complaint about desired features that the phone didn't provide, more than a bug. I'd put that in a different category. Software can't be everything to everyone.

TeMPOraL 3y ago
On the other hand, this technology has been around for 30 years. The feature in question reaches back to the earliest mobile phones. Not smartphones - think earlier than Nokia 3310. On the gripping hand, "silence.mp3" has been the solution for the author's "mom problem" for a good 20 years too. I hope they don't mean to imply you can't assign individual ringtones to contacts on stock Android? Even the K800i I had as a teenager supported that. On the yet another hand (we're moving fast into octopod territory here), for a moment here I was wondering what the author was smoking with the "ringtone & notifications" complaint. I don't remember seeing an Android phone that did not have separate volume slider for each. Then I realized, I never actually owned a stock Android phone. Still, to support the author's point against Google specifically, Samsung managed to not have this problem for at least 10 years now.
- SoftTalker 3y ago
  My silence.mp3 story on Android was this: I was using an android device to play background music through a PA during breaks in activity. Every time I'd start a song there was a "pop" I assume by the audio output amplifier switching on or waking up. My solution to avoid this was to play "silence.mp3" in a repeat loop in another app, and then my music app would play over that and there were no pops when I started a song. IDK if the pop problem has been fixed, or whether it was a software or hardware issue.
- maxime_cb 3y ago
  > I was wondering what the author was smoking with the "ringtone & notifications" complaint. I don't remember seeing an Android phone that did not have separate volume slider for each. This is why I included a screenshot of the volume sliders my Google Pixel displays. Samsung doesn't use stock android OS, and some Samsung users tend to assume every android phone works the same.

RobotToaster 3y ago

The infamous bug(s) in the Fujitsu system provided to the British post office caused several suicides. https://en.wikipedia.org/wiki/British_Post_Office_scandal#Horizon_computer_system https://en.wikipedia.org/wiki/British_Post_Office_scandal#Ho...

goda90 3y ago
>The Post Office resisted the SPMs' reports of faults in the system, insisted that the SPMs make up any shortfall of money and, when asked by an SPM, denied that other SPMs had reported problems. Seems more like a very toxic culture of shifting blame led to those suicides, with the bugs just being a catalyst.
- yifanl 3y ago
  The bugs were causing very real financial distress it sounds like, if the software was partially responsible for determining their continued employment
  - nitwit005 3y ago
    Normal people would see dozens of staff members reporting the same issues as a reason to get the bugs fixed. Some of the stuff that came to light in that scandal was so extreme, that it's clear the leadership was a bit deranged. Crazy people produce crazy results.
stephen_g 3y ago
Crazy how they just refused to believe (or properly assess whether it was true) that the software could have any issues, despite having reports of errors starting very soon after it went in. I find it extraordinary that they secured convictions despite the fact there wouldn’t have been any unexplained wealth or any money popping up elsewhere - although it seems in a lot of cases solicitors basically convinced the postmasters to make false confessions (kind of like plea bargains I guess).
GoblinSlayer 3y ago

jmclnx 3y ago

>The screen on my phone showed that the alarm I had set was in the process of ringing, but for some reason, the phone wasn’t vibrating or making any sound. And this is kind of an example of why it will be impossible to mitigate Climate Change. When I was young, clocks did not requite any power from the grid. It wound them, set the alarm and you heard it. Now we have faucets, toilets that use power to turn on/off and flush. Almost everything that was manual (mechanical) is now powered. That power needs to come from somewhere. In these examples, there is no real reason to have faucets, toilets and alarm clocks connected to the grid. Yes, I know in the case of the Cell Phone Alarm, it is probably a wash, but a mechanical clock in your house is probably a good thing to have.

jrockway 3y ago
It's most likely to me that many more people have overslept because they forgot to wind their clock than have overslept because their iPhone hit a weird bug where the alarm sound didn't play. Simply because mechanical clocks have been around much longer.
- SoftTalker 3y ago
  Also at a certain time in my teenage years I could sleep through a ringing alarm clock with absolutely no awareness of it. The spring for the bell-ringing mechanism would unwind after maybe 30 seconds or so, and the ringing would stop, and I'd sleep completely through it. Not sure the powered alarms would have done any better, but maybe after 10 minutes of ringing it might have woken me up.
alpaca128 3y ago
> a mechanical clock in your house is probably a good thing to have. There are much better and more reliable options. Solar powered watches that can run months without recharging, some digital watches run a decade with a single battery. A mechanical watch, on the other hand, has to be wound regularly and is usually far less accurate. I like automatic mechanical watches but I don't think they can compete in any technical aspect, that's not the reason one would use them nowadays.
3y ago

kernal 3y ago

>Unfortunately, Android [9.0] suffers from poor user interface design in a few areas, and one of the most annoying flaws in its user interface is simply that the stock Android OS doesn’t have flexible enough options when it comes to controlling when the phone rings, which is one of the most important aspects of a phone. Do Not Disturb, at the time of Android 9, did support controlling when the phone rings. >Bedtime mode is quite useful, but I still have the other problem that my mom could decide to randomly call me in the daytime as well, and unfortunately I rarely want to take her phone calls. However, I also don’t want her to end up homeless or in jail (which has happened before, but that’s a story for another time), and so I don’t want to block her and completely lose the ability to receive her calls. This results in me having to almost always have my phone set to “do not disturb”, so that I don’t have to be disturbed at random times by unwanted phone calls Could he not have used a custom ringtone for his mother?

pw6hv 3y ago
*she

grepLeigh 3y ago

Great name for this kind of blog (pointersgonewild) > Long story short: a software bug caused the machine to occasionally give radiation doses that were sometimes hundreds of times greater than normal Oh my god. I have 15+ years working with "mission critical (software) infrastructure" and thought my job was important/hard because I played shepherd for production database fleets. Certain kinds of mistakes could bankrupt the business, so I had to make systems resilient to human errors. Today, I run a 3D printer software startup. Thinking deeply about the safety mechanisms I need to control machines running at 250-300°C. Precision errors when shooting radiation at a person is a whole different level. Wow.

masklinn 3y ago
The Therac-25 story is pretty wild, especially as it was a reuse (with modification) of an existing codebase, but for cost savings the -25 was designed without hardware interlocks, under the assumption that the software would run fine, as it had run fine on the -20. Turns out the software had run fine on the -6 -20 because of the mechanical interlocks which prevented it from critically fucking up, but there was no reporting built into the software interlocks, so no way to know when they'd triggered. And that was before additional modifications were added for the -25.
- anonymousiam 3y ago
  Not all that different from the Ariane-5 maiden launch, except for human lives being at stake vs. billions of Euros (adjusted for inflation). https://en.wikipedia.org/wiki/Ariane_flight_V88 https://en.wikipedia.org/wiki/Ariane_flight_V88
  - nirui 3y ago
    Well, for me, the level of anxiety is definitely different. I somehow have a few degrees of fear towards machines that could emit harmful level of ionizing radiation, and I can be nervous if the machine is operating on me (say doing X-Ray at the dentist). Plus, I also have some anxiety when handling concurrent programming, which according to the Wikipedia is what caused the error in Therac-25. So I guess I got double anxieties on that story.

OnlyMortal 3y ago

A story related to me from a friend… A torpedo system was designed not to hit the submarine that fired it. It would detonate if it was aiming itself back. So, testing came along, the torpedoes armed and… the sub turned itself around. Bang.

meghan_rain 3y ago
Sorry, I don't understand what happened?
- isidor3 3y ago
  Sounds like the torpedoes were armed, but not yet fired. So the sub itself turning around in a circle triggered the self-destruct mechanism while the weapons were still in the tubes.
- orbz 3y ago
  I think the implication is that the torpedos had not yet left their tubes, so when the sub turned around with them still in it they registered that as a situation where they should destruct.

awesome_dude 3y ago

I'm surprised that the Metric <-> Imperial bug that caused NASA to lose 125 million wasn't mentioned https://en.wikipedia.org/wiki/Mars_Climate_Orbiter https://en.wikipedia.org/wiki/Mars_Climate_Orbiter

contingencies 3y ago
It's certain there is >$10B loss per annum on metric-imperial. Think about it: human error, conversion process losses, training materials, advertising materials, two physical versions of everything being produced, two physical versions in the supply chain... I for one plan to operate a metric shop in the US. Using imperial outside of supplier interface will be a formal warning. Will see how that goes...

soult 3y ago

It's interesting that the author picked the home depot delivery as an example, because I don't think it was caused by software bugs at all. The delivery notifications right at the end of the delivery slot happened because the driver has an unrealistic schedule, and to hit their metrics, they will mark deliveries as "delivered" rather than missing an assigned delivery slot. And that the expensive two-man-handling and delivery to the apartment service got "lost" between seller and subcontractor is more likely cost optimization and banking on most customers not complaining hard enough to actually force them to provide the paid-for service. "Software bug" being a convenient excuse the subcontractor can give to the rightfully enraged customer.

maxime_cb 3y ago
Author here. The reason I tend to think it's a software bug is that it seems that the system to dispatch deliveries is automated. The subcontractors get their orders from some kind of computerized system, it seems. That system seems to systematically fail to specify when they are to carry items indoors/upstairs. Whether that's due to negligence or intentional malfeasance, don't know. What I do know from experience is that there are numerous bugs on their website, besides the "unknown error" problem I've listed. It just seems like really shittily built software... So I would tend to think there's an issue with really poor software engineering practices at that company.
- abnercoimbre 3y ago
  I loved your article. You might consider joining the Handmade Network [0] and related conferences that I run [1] because this is precisely the sort of behavior we want to curb in our industry. Just know some of us are fighting this. [0] https://handmade.network https://handmade.network [1] https://handmadecities.com https://handmadecities.com
  - maxime_cb 3y ago
    Looks pretty cool :)

kevin_nisbet 3y ago

I think my message is, don't lose sight on the mission of the software that you're shipping. My story is, I found a bug in some new equipment that would've broken 911 calls. While other engineers were just trying to reboot random equipment to make the problem go away, I insisted we pause to figure it out. Turns out, due to a couple of bugs, the new network equipment could only handle a couple of 911 calls before failing. This was for a national cellular network... so 911 was kind of important.

Aperocky 3y ago
Did you get recognition for recognizing the problem and dealing with it?
- klysm 3y ago
  Going to go out on a limb and say no because it didn’t generate revenue
eastbound 3y ago
Were you working on Android? It is famous for failing to call 911 in emergency situations.
c4ptnjack 3y ago
Additionally, I assume this was on the cellular network completely and not on the dispatcher/rf side of things?

anonymousiam 3y ago

Well written article, but most of the "harms" described (aside from the Therac-25) are a direct result of attempting to replace human-to-human contact with technology. 1) Instead of relying on a phone alarm to wake you up, use a device designed for that purpose, or request a wake-up call (on a landline, if you have one). 2) Overcome the conflict between having a phone and not wanting to answer it. 3) When unusual and specific instructions are required, make sure you speak with a person to confirm them.

groestl 3y ago
And Therac-25 was trying to remove hardware interlocks from Therac-6 and Therac-20, to fully rely on software, AFAIK.
tczMUFlmoNk 3y ago
I agree with you that the trajectory of removing human-to-human contact, even as a fallback, is causing harm. And I agree with your interpretation of your point (3): having an option to speak to a human would have gone a long way here. I'm not sure that I agree with your point (1): a modern smartphone is a device designed for that purpose, among other things, and it is virtuous to use a multi-purpose device for this instead of building, transporting, and disposing of a dozen bespoke devices for each such purpose. But reducing the complex interpersonal relationship between the author and her mom to just "overcome the conflict" is a really low-bar take. It feels callous to suggest that it would be so easy, or to assume that she's not trying to do this, or even that this is the right path forward for her. Technology should serve us and adapt to the diversity of our needs, as we change as people and those needs change with us.
taneq 3y ago
I don’t think “software is unreliable and you should never trust it” is the outcome we should be hoping for here. If I burn to death in a Ford Pinto, would you tell me I should have ridden a horse?
- intelVISA 3y ago
  Depends, was the horse sold to you with a potentially fatal, albeit financially acceptable, defect?

sitkack 3y ago

Everything is mission critical to someone.

meghan_rain 3y ago

> Thankfully, Android now has “bedtime mode” feature, which allows me to make it so that phone calls won’t cause my phone to ring between 10AM and 8:30AM. If my mom happens to die in a hospital in the middle of the night, I’ll just have to find out and be sad the next day. wtf

usui 3y ago
Is this a comedic "wtf"? If nt, I think you should recognize the flippant tone, not to be taken too seriously.

deleted 3y ago

[deleted]

acenes 3y ago

A recent example: https://www.nytimes.com/2023/05/26/science/moon-crash-japan-ispace.html https://www.nytimes.com/2023/05/26/science/moon-crash-japan-...

deanCommie 3y ago

> I wish that Android had an option to set a specific person to never cause the phone to ring, and it seems like that should be an easy feature to implement that would have a real positive impact on the quality of lives of many people, but I digress. FWIW, Android does have that feature - at least for sure the Pixel Launcher (that the author uses) does: You can set some contacts as "Favourites" and have it configured that Favourites get to bypass Do Not Distrurb. Good blog nonetheless!

smabie 3y ago
That's not the same feature tho
- deanCommie 3y ago
  My mistake. I thought the author wanted to make sure her mother wouldn't wind up dead or in jail. Instead she is unclear about what she actually wants: > I still have the other problem that my mom could decide to randomly call me in the daytime as well, and unfortunately I rarely want to take her phone calls. However, I also don’t want her to end up homeless or in jail (which has happened before, but that’s a story for another time), and so I don’t want to block her and completely lose the ability to receive her calls. So...does she want her mom's phone calls to get through, or doesn't she?

anonymousDan 3y ago

If you want to see a really sad example, check out the Royal Mail/Fujitsu postmaster scandal in the UK, where postmasters (franchisees) ended up being thrown in jail, losing everything they owned, families destroyed, suicides, etc. All because of bugs in Fujitsu accounting software (https://www.bbc.co.uk/news/business-56718036 https://www.bbc.co.uk/news/business-56718036)

pards 3y ago

Knight Capital [0] should serve as a warning to devs and ops alike. They lost about $450 million in about 30 mins due to some poor coding, a botched deployment, and botched rollbacks. [0]: https://en.wikipedia.org/wiki/Knight_Capital_Group https://en.wikipedia.org/wiki/Knight_Capital_Group

joezydeco 3y ago

Almost any and every story you'll see in this thread has already been collected and discussed in the RISKS mailing list and archive. There's nearly 38 years of articles waiting for you to peruse, all lovingly curated by Peter Neumann. https://catless.ncl.ac.uk/risks/ https://catless.ncl.ac.uk/risks/

Ubergeek99 3y ago

But the biggest "real world harm" is that the mother is able to call her (the author). What kind of world we live in that schizophrenic parents can get ahold of us? That really is a big bug. Apple and Android should do something about that... before it's too late and irreversible harm is done.

maxime_cb 3y ago
Author here. If you've grown up in a "normal", functional family, with two loving parents, and you enjoy talking to them on the phone, you should consider yourself very lucky. I only take calls from my mother when I feel up to it. The reason for that is that she has no concept of boundaries. Her mental illness prevents her from grasping that concept. Her default behavior tends towards what a normal person would call harassment. If you can't relate or understand a situation like that, it might just be because you've had a relatively safe, coddled, privileged life.

brokenmachine 3y ago

https://wikipedia.org/wiki/British_Post_Office_scandal https://wikipedia.org/wiki/British_Post_Office_scandal

etimberg 3y ago

Surprised the Patriot missile failure during the Gulf War wasn't mentioned https://en.wikipedia.org/wiki/MIM-104_Patriot#Failure_at_Dhahran https://en.wikipedia.org/wiki/MIM-104_Patriot#Failure_at_Dha...

WalterBright 3y ago

> if you’re a software engineer and you’re working on safety-critical systems, you absolutely must do due diligence and implement proper validation and testing, otherwise you could be putting human lives at risk. I'll be blunt. This is the wrong lesson. The correct lesson is: Assume the software will do the worst thing possible. How will you prevent that from causing a catastrophe? I.e. make the Therac-25 physically incapable of generating an excessive dose. This is how airplanes are made safe. Not by imagining you can create perfect hardware or software, but by assuming any part can fail, and how does the airplane continue to fly safely?

WalterBright 3y ago
> A malfunctioning alarm clock probably isn’t going to directly cause a person’s death, but it definitely has the potential to cause real-world harm. When I absolutely have to get up on time, I set two alarm clocks. When traveling, I'll set my phone, and also arrange for a wakeup call. I learned about having an independent backup from working at Boeing engineering.
- interfixus 3y ago
  That was my thought, too. Day to day, my phone alarm is fine (and I damn well tested it thoroughly before actually relying on it), but if anything life-and-death is scheduled, I'll use at least one independent extra device. And all this even though I always wake up on time and never am late. Likewise, the phone call problem has an easy solution: Assign different sounds to different callers; low and unobtrusive to the unwanted ones. And the delivery: Never trust such a thing without sufficient fallbacks. If you want safety and full control, organise the thing yourself. All in all, I get a vibe that the poster may be a wonderful coder, but really hasn't got the paranoid mindset required for mission critical work.
- deleted 3y ago
  [deleted]
physicles 3y ago
> Assume the software will do the worst thing possible. That's certainly a valuable lesson when there's hardware and software involved, and the consequences of the software screwing up are potentially lethal. But what about the case of the home depot shopping cart, or the iPhone alarm? In those cases there's only software. "Assume the software will do worst thing possible" doesn't make sense.

jmholla 3y ago

> [On Android] why in the fuck can’t there simply be two separate fucking sliders for “Ring volume” and “Notification volume”? Are there not for everyone? On every Android phone I've had in recent memory I've had his feature. I'm certain I have it on my current phone because I just looked. In fact, they're called exactly what the author refers to them as.

dvlsg 3y ago
I have one slider for "ring & notification volume" on mine. I also have media, call, and alarm volume as 3 other sliders, for a total of 4.
- jmholla 3y ago
  Interesting. I'm on a stock Motorola and they are separate. I have five sliders.
  - Suppafly 3y ago
    maybe newer motorolas have customized it, but on mine i have media,call, ring, and alarm.

xupybd 3y ago

I disagree with the author, all of those bugs they list cause people to get upset. Not harm. You are talking about the cost of quality. Software quality comes at a cost. It's up to the business to determine how much it will spend on quality. If we had a zero bug policy in non safety critical applications the cost of goods would be too high.

physicles 3y ago

I really appreciate the author shining a lens on domains other than aviation, health care, etc. There's plenty of potential for software to do harm in everyday domains. The rage I feel when software that I use breaks, I take that rage and channel it into customer empathy. If my stuff causes someone to get annoyed, or lose a lot of time, or both, that really sucks and it needs to be fixed. I wish we could do more of that in this industry. I got bit by that iPhone alarm bug a couple times. Couldn't believe something like that would slip into production and stay there for as long as it did.

Fragoel2 3y ago

We (me and co-authors) recently published a study in which we found some examples of real-world harm in user reviews of IoT devices (https://www.sciencedirect.com/science/article/pii/S1574119223000445 https://www.sciencedirect.com/science/article/pii/S157411922...). These devices are supposed to be extremely pervasive in the future, hence we need to do better to ensure their safety.

saint-loup 3y ago

There's quite an interesting book on the topic of real-wold harm, mainly from an user interface point of view. https://www.tragicdesign.com/ https://www.tragicdesign.com/

deleted 3y ago

[deleted]