4 ms·

I saw a video about a fatal oil drilling accident where the resulting investigation found that an operator had ignored the alarm which indicated the underlying

I saw a video about a fatal oil drilling accident where the resulting investigation found that an operator had ignored the alarm which indicated the underlying issue because there were other alarms sounding for things he'd been told to ignore because they were benign. This is apparently common enough cause of problems that there is a term for it (other than "crying wolf"): https://en.wikipedia.org/wiki/Alarm_fatigue https://en.wikipedia.org/wiki/Alarm_fatigue I've seen it during my career as a software developer, too. Heaps of alert spam that remained unread because 99% of them were useless and not indicative of a problem on our end. It's important to be careful about alerts so you know when real problems occur.

Muromec 2mo ago
This also how a part of how a certain flagship got promoted to a submarine recently.
soco 2mo ago
And because of this, no browser of mine is allowed to send notifications, and also no app by default - they have to prove themselves useful first. Sorry fellow developers working on notifications, your work goes to the drain (mostly).
- rithdmc 2mo ago
  I recently unsubscribed from a grocery delivery service becuase they kept pushing adverts through their app. I gave them notification permission so I could see when my delivery is nearby, not so they could spam me. Voting with my wallet on this one. (For the curious, a subscription means free delivery. The subscription costs 10 euro a month, and if you don't have a subscription, the delivery fee is around 4 euro)
  - bluefirebrand 2mo ago
    > For the curious, a subscription means free delivery It sounds like a subscription means ad-funded delivery to me though
    - aprentic 2mo ago
      That's clearly how it turned out but going in, I would have expected the a subscription to be a discount in exchange for a commitment of repeat business.
      - bluefirebrand 2mo ago
        Yeah I would expect a subscription to remove ads Personally I would absolutely refuse to pay for services that still continue to serve me ads even though I'm paying I loathe advertising. I don't understand why most people seem so apathetic about it
    - rithdmc 2mo ago
      You don't need a subscription to use the app, and I presume these ads are pushed to all users.
- deleted 2mo ago
  [deleted]
- voidUpdate 2mo ago
  I've recently had to add a notifications system for an app I manage for a client. The client requested it, and I work as part of a larger company, so I couldn't exactly say no, but I felt pretty icky doing it
- sixothree 2mo ago
  > no browser of mine is allowed to send notifications I do this for different reasons. I just don't want every site knowing when I'm logged in and using my computer, which computer that is, and whatever else information they're grabbing all day.
cucumber3732842 2mo ago
In the US spewing all the alarms at the line level employees and fatiguing the shit out of them is a direct (and predictable, but of course everyone who complained in advance got sneered at by the usual demographics) result of OSHA policy. Same bunch of moronic incentives that result in there being a policy against everything and employees technically violating the rules to work normally. Whenever you see an alarm fatigue workplace there's usually a rule in the rulebook that says they have to stop what they're doing for every alarm until it's triaged and documented. This is how the rules financially incentivize the company to act even if it's stupid and results in more screw ups.
- ubertaco 2mo ago
  Yeah, if only the government didn't regulate that pesky checks notes workplace safety, then for-profit companies could be free to act for the good of all, like they always do!
  - deleted 2mo ago
    [deleted]
  - bigstrat2003 2mo ago
    Nobody has said that, so you're arguing against a strawman. The parent poster made the argument that there's too much government regulation such that it causes fatigue and ignoring the rules, not that there should be zero.
- pjc50 2mo ago
  > a policy against everything and employees technically violating the rules to work normally. This is called "normalization of deviance" and generally should be avoided, because it often precedes a serious accident.
  - cucumber3732842 2mo ago
    >This is called "normalization of deviance" You don't even need to get to the point of breaking the rules for it to be a problem. This is stupid human factors type stuff. The filthy capitalists don't want their oil rigs going kaboom and their expensively trained workers having injury downtime. There is a natural incentive alignment to some extent here. And then the useful idiots show up, and rather than fostering that alignment they put their thumb on the scale and they change the calculations. The filthy capitalists crunch the numbers and they realize that over the life of the oil rig and a fleet of some number of them they'll make more money spewing alarms at the guys running the place until their eyes glaze over. Sure he'll miss something important blowing up the works with loss of life and limb 2/<big number> amount of the time instead of 1/<big number> if you'd have tuned it right buuuuut, you'll lose less money that way because if you do it the "right" way you'll get dinged by the OSHA people every time some petty accident type happens because you turned the alarm for that one off so as not to fatigue people. A typical example is some industrial facility where policy defines "hot" as anything lukewarm and up and so they implicitly train people to ignore it. People are opening "hot" equipment all the time, getting approvals to do it, ignoring warning lights, etc, etc, so basically no net improvement in actual safety over having nothing at all. And this is fine until one day someone gets killed because they let loose something that was 300deg not 110. You see this goddamn fact pattern all the time. Machinery beepers, hazard signage and barriers, machinery interlocks, etc, etc, it never fucking ends. Great job guys, Mission Accomplished (TM). When everything is a hazard nothing is.
philipwhiuk 2mo ago
On a finance level this is why correctly setting 'accidental trade limits' correctly is important. Otherwise traders see them so often they click through them.
sylware 2mo ago
In France, we have a bed story for children related to exactly that: "Pierre et le loup" (aka your "crying wolf"). It may be a EU wide tale.
- olsondv 2mo ago
  It’s a common tale in English speaking world as well, but we don’t use it as a bedtime story because of the ending. Edit. Didn’t realize it might just be an English version where the boy is killed and eaten. Aesop’s original didn’t have that. Just the sheep killed.
- boomlinde 2mo ago
  It's one of Aesop's fables, so it's probably widely known in all of Europe.
- eloisant 2mo ago
  You're thinking about The Boy who Cried Wolf, Pierre et le Loup is a different story.
  - quietbritishjim 2mo ago
    And Pierre et le Loup seems to be Peter and the Wolf which is a Russian story / piece of music.
OroPla 2mo ago
Reminds me of my time in a large dormitory. The first time the fire alarm went off at night without there even being a fire, I went outside. All the other times - and I lost track of counting - I just ignored it and went back to sleep.
- seki285 2mo ago
  This. The alarm would go off but I'd continue playing dota
- marcosdumay 2mo ago
  I once worked at a building that decided it was a good safety practice to test the fire alarms every 3 months.
  - Hugsbox 2mo ago
    My job does it weekly. It's a massive factory too, so the alarms are absolutely ear-splitting. They make a big deal out of safety, and wearing ear protection out on the production floor when I go out there, and all this other stuff, yet on a weekly basis they're absolutely shattering my ear drums. Love it.
    - Scoundreller 2mo ago
      We need louder sirens to counteract the hearing damage caused by our old sirens.
    - badpun 2mo ago
      Maybe they must be so loud because people wear hearing protection?
sporkland 2mo ago
In college they made us study the case of therac-25 so we understood the dangers of creating too many errors in our software. https://en.wikipedia.org/wiki/Therac-25 https://en.wikipedia.org/wiki/Therac-25
- quietbritishjim 2mo ago
  I didn't think that was about too many alarms going off and so that a real one was being ignored, and re-reading the article doesn't give that impression either. It was more to do with a bug in the user interface (it appeared that you were re-editing a field you'd just filled in but actually weren't) and a memory corruption bug, along with assuming that software always works so doesn't need any additional checks. Are you thinking of another software failure? (If by "dangers of creating too many errors" you just meant "don't create too many bugs" then that's not really relevant to this conversation, and also totally the wrong lesson to learn from that incident. The lesson is, software will inevitably have bugs, so make sure you have redundant checks to minimise their impact.)
  - bluGill 2mo ago
    The alarms were safety hardware interlocks. Nobody asked why and found the UI bugs that was causing this. The hardware interlocks were removed because the system had been working safely for so long. This was really feedback - nobody informed the developers that safety hardware interlocks were locking all the time. Edit: see replies, originally I wrote fuses, but hardware interlocks is more correct.
    - quietbritishjim 2mo ago
      The Wikipedia article the previous commenter linked doesn't mention this, and I've looked at a few sources linked from it and they don't mention it either. They mention that previous Therac machines had "hardware interlocks" to prevent this sort of issue - maybe those are the safety fuses you're both talking about? - but it says they were never included in Therac-25 in the first place, because the software was assumed to be so reliable, not that they were removed because they blew. Are you sure this is the same incident? Do you have a link to a source about it? Edit: This article cited in Wikipedia is much more comprehensive and mentions that Therac-20 did have these fuses, in addition to hardware interlocks. But they weren't disabled because the real alarms were drowned out by false alarms - instead they all represented real instances of the issue. https://web.archive.org/web/20041128024227/http://www.cs.umd.edu/class/spring2003/cmsc838p/Misc/therac.pdf https://web.archive.org/web/20041128024227/http://www.cs.umd... (Middle column of the 12th page of PDF, numbered page 29 of the publication.) > The Therac-20 at the University of Chicago is used to teach students in a radiation therapy school conducted by the center. The center’s physicist, Frank Borger, noticed that whenever a new class of students started using the Therac-20, fuses and breakers on the machine tripped, shutting down the unit. These failures, which had been occurring ever since the center had acquired the machine, might appear three times a week while new students operated the machine and then disappear for months. Borger determined that new students make lots of different types of mistakes and use “creative methods of editing” parameters on the console. Through experimentation, he found that certain editing sequences correlated with blown fuses and determined that the same computer bug (as in the Therac-25 soft- ware) was responsible. The physicist notified the FDA, which notified Therac-20 users.* > The software error is just a nuisance on the Therac-20 because this machine has independent hardware protective circuits for monitoring the electron-beam scanning. The protective circuits do not allow the beam to turn on, so there is no danger of radiation exposure to a patient. While the Therac-20 relies on mechanical interlocks for monitoring the machine, the Therac-25 relies largely on software.
cactacea 2mo ago
To go deeper than just alarming this is more accurately referred to as normalization of deviance https://en.wikipedia.org/wiki/Normalization_of_deviance https://en.wikipedia.org/wiki/Normalization_of_deviance

Muromec 2mo ago

This also how a part of how a certain flagship got promoted to a submarine recently.

soco 2mo ago

And because of this, no browser of mine is allowed to send notifications, and also no app by default - they have to prove themselves useful first. Sorry fellow developers working on notifications, your work goes to the drain (mostly).

rithdmc 2mo ago
I recently unsubscribed from a grocery delivery service becuase they kept pushing adverts through their app. I gave them notification permission so I could see when my delivery is nearby, not so they could spam me. Voting with my wallet on this one. (For the curious, a subscription means free delivery. The subscription costs 10 euro a month, and if you don't have a subscription, the delivery fee is around 4 euro)
- bluefirebrand 2mo ago
  > For the curious, a subscription means free delivery It sounds like a subscription means ad-funded delivery to me though
  - aprentic 2mo ago
    That's clearly how it turned out but going in, I would have expected the a subscription to be a discount in exchange for a commitment of repeat business.
    - bluefirebrand 2mo ago
      Yeah I would expect a subscription to remove ads Personally I would absolutely refuse to pay for services that still continue to serve me ads even though I'm paying I loathe advertising. I don't understand why most people seem so apathetic about it
  - rithdmc 2mo ago
    You don't need a subscription to use the app, and I presume these ads are pushed to all users.
deleted 2mo ago
[deleted]
voidUpdate 2mo ago
I've recently had to add a notifications system for an app I manage for a client. The client requested it, and I work as part of a larger company, so I couldn't exactly say no, but I felt pretty icky doing it
sixothree 2mo ago
> no browser of mine is allowed to send notifications I do this for different reasons. I just don't want every site knowing when I'm logged in and using my computer, which computer that is, and whatever else information they're grabbing all day.

cucumber3732842 2mo ago

In the US spewing all the alarms at the line level employees and fatiguing the shit out of them is a direct (and predictable, but of course everyone who complained in advance got sneered at by the usual demographics) result of OSHA policy. Same bunch of moronic incentives that result in there being a policy against everything and employees technically violating the rules to work normally. Whenever you see an alarm fatigue workplace there's usually a rule in the rulebook that says they have to stop what they're doing for every alarm until it's triaged and documented. This is how the rules financially incentivize the company to act even if it's stupid and results in more screw ups.

ubertaco 2mo ago
Yeah, if only the government didn't regulate that pesky checks notes workplace safety, then for-profit companies could be free to act for the good of all, like they always do!
- deleted 2mo ago
  [deleted]
- bigstrat2003 2mo ago
  Nobody has said that, so you're arguing against a strawman. The parent poster made the argument that there's too much government regulation such that it causes fatigue and ignoring the rules, not that there should be zero.
pjc50 2mo ago
> a policy against everything and employees technically violating the rules to work normally. This is called "normalization of deviance" and generally should be avoided, because it often precedes a serious accident.
- cucumber3732842 2mo ago
  >This is called "normalization of deviance" You don't even need to get to the point of breaking the rules for it to be a problem. This is stupid human factors type stuff. The filthy capitalists don't want their oil rigs going kaboom and their expensively trained workers having injury downtime. There is a natural incentive alignment to some extent here. And then the useful idiots show up, and rather than fostering that alignment they put their thumb on the scale and they change the calculations. The filthy capitalists crunch the numbers and they realize that over the life of the oil rig and a fleet of some number of them they'll make more money spewing alarms at the guys running the place until their eyes glaze over. Sure he'll miss something important blowing up the works with loss of life and limb 2/<big number> amount of the time instead of 1/<big number> if you'd have tuned it right buuuuut, you'll lose less money that way because if you do it the "right" way you'll get dinged by the OSHA people every time some petty accident type happens because you turned the alarm for that one off so as not to fatigue people. A typical example is some industrial facility where policy defines "hot" as anything lukewarm and up and so they implicitly train people to ignore it. People are opening "hot" equipment all the time, getting approvals to do it, ignoring warning lights, etc, etc, so basically no net improvement in actual safety over having nothing at all. And this is fine until one day someone gets killed because they let loose something that was 300deg not 110. You see this goddamn fact pattern all the time. Machinery beepers, hazard signage and barriers, machinery interlocks, etc, etc, it never fucking ends. Great job guys, Mission Accomplished (TM). When everything is a hazard nothing is.

philipwhiuk 2mo ago

On a finance level this is why correctly setting 'accidental trade limits' correctly is important. Otherwise traders see them so often they click through them.

sylware 2mo ago

In France, we have a bed story for children related to exactly that: "Pierre et le loup" (aka your "crying wolf"). It may be a EU wide tale.

olsondv 2mo ago
It’s a common tale in English speaking world as well, but we don’t use it as a bedtime story because of the ending. Edit. Didn’t realize it might just be an English version where the boy is killed and eaten. Aesop’s original didn’t have that. Just the sheep killed.
boomlinde 2mo ago
It's one of Aesop's fables, so it's probably widely known in all of Europe.
eloisant 2mo ago
You're thinking about The Boy who Cried Wolf, Pierre et le Loup is a different story.
- quietbritishjim 2mo ago
  And Pierre et le Loup seems to be Peter and the Wolf which is a Russian story / piece of music.

OroPla 2mo ago

Reminds me of my time in a large dormitory. The first time the fire alarm went off at night without there even being a fire, I went outside. All the other times - and I lost track of counting - I just ignored it and went back to sleep.

seki285 2mo ago
This. The alarm would go off but I'd continue playing dota
marcosdumay 2mo ago
I once worked at a building that decided it was a good safety practice to test the fire alarms every 3 months.
- Hugsbox 2mo ago
  My job does it weekly. It's a massive factory too, so the alarms are absolutely ear-splitting. They make a big deal out of safety, and wearing ear protection out on the production floor when I go out there, and all this other stuff, yet on a weekly basis they're absolutely shattering my ear drums. Love it.
  - Scoundreller 2mo ago
    We need louder sirens to counteract the hearing damage caused by our old sirens.
  - badpun 2mo ago
    Maybe they must be so loud because people wear hearing protection?

sporkland 2mo ago

In college they made us study the case of therac-25 so we understood the dangers of creating too many errors in our software. https://en.wikipedia.org/wiki/Therac-25 https://en.wikipedia.org/wiki/Therac-25

quietbritishjim 2mo ago
I didn't think that was about too many alarms going off and so that a real one was being ignored, and re-reading the article doesn't give that impression either. It was more to do with a bug in the user interface (it appeared that you were re-editing a field you'd just filled in but actually weren't) and a memory corruption bug, along with assuming that software always works so doesn't need any additional checks. Are you thinking of another software failure? (If by "dangers of creating too many errors" you just meant "don't create too many bugs" then that's not really relevant to this conversation, and also totally the wrong lesson to learn from that incident. The lesson is, software will inevitably have bugs, so make sure you have redundant checks to minimise their impact.)
- bluGill 2mo ago
  The alarms were safety hardware interlocks. Nobody asked why and found the UI bugs that was causing this. The hardware interlocks were removed because the system had been working safely for so long. This was really feedback - nobody informed the developers that safety hardware interlocks were locking all the time. Edit: see replies, originally I wrote fuses, but hardware interlocks is more correct.
  - quietbritishjim 2mo ago
    The Wikipedia article the previous commenter linked doesn't mention this, and I've looked at a few sources linked from it and they don't mention it either. They mention that previous Therac machines had "hardware interlocks" to prevent this sort of issue - maybe those are the safety fuses you're both talking about? - but it says they were never included in Therac-25 in the first place, because the software was assumed to be so reliable, not that they were removed because they blew. Are you sure this is the same incident? Do you have a link to a source about it? Edit: This article cited in Wikipedia is much more comprehensive and mentions that Therac-20 did have these fuses, in addition to hardware interlocks. But they weren't disabled because the real alarms were drowned out by false alarms - instead they all represented real instances of the issue. https://web.archive.org/web/20041128024227/http://www.cs.umd.edu/class/spring2003/cmsc838p/Misc/therac.pdf https://web.archive.org/web/20041128024227/http://www.cs.umd... (Middle column of the 12th page of PDF, numbered page 29 of the publication.) > The Therac-20 at the University of Chicago is used to teach students in a radiation therapy school conducted by the center. The center’s physicist, Frank Borger, noticed that whenever a new class of students started using the Therac-20, fuses and breakers on the machine tripped, shutting down the unit. These failures, which had been occurring ever since the center had acquired the machine, might appear three times a week while new students operated the machine and then disappear for months. Borger determined that new students make lots of different types of mistakes and use “creative methods of editing” parameters on the console. Through experimentation, he found that certain editing sequences correlated with blown fuses and determined that the same computer bug (as in the Therac-25 soft- ware) was responsible. The physicist notified the FDA, which notified Therac-20 users.* > The software error is just a nuisance on the Therac-20 because this machine has independent hardware protective circuits for monitoring the electron-beam scanning. The protective circuits do not allow the beam to turn on, so there is no danger of radiation exposure to a patient. While the Therac-20 relies on mechanical interlocks for monitoring the machine, the Therac-25 relies largely on software.

cactacea 2mo ago

To go deeper than just alarming this is more accurately referred to as normalization of deviance https://en.wikipedia.org/wiki/Normalization_of_deviance https://en.wikipedia.org/wiki/Normalization_of_deviance