6 ms·

> Of course the personnel in department Z might review it but ignore it That's the problem right there. One of my clients had a large IT organization of over 1

> Of course the personnel in department Z might review it but ignore it That's the problem right there. One of my clients had a large IT organization of over 1000+ employees, with strict change control rules, and procedures that were tracked in SN. Every time there was a change management meeting everyone who could possibly be effected would get an email from Service Now notifying them of the upcoming changes. Pretty much engineer ignored those emails, because there was so much going on in the org you'd get dozens of emails in a week and most of them you'd only be tangentially effected by, meanwhile you had your work to do. So the problem isn't getting the notifications out it's getting people to pay attention to them.

MichaelZuo 4y ago
I don't see how that's a problem for the organization. Individual preferences do vary, one ignores 90%, another 95%, another 100%. And the one who's ignoring 100% of them will likely eventually make a mistake that otherwise wouldn't have happened. But it will be fairly straightforward to resolve, after all there's an extensive paper trail as the chain of custody seems clear. Assuming the "change management meeting" emails were the approved means of communication.
- nordsieck 4y ago
  > I don't see how that's a problem for the organization. IMO, one of the lessons that came out of Chernobyl is that it absolutely is a problem for the organization. Exposing people to too many "alarms" that are constantly going off will cause people to start ignoring them. Part of good design is figuring out which things are truly important, and how to communicate that to the people who are supposed to be paying attention.
  - MichaelZuo 4y ago
    The analogy seems not to apply? The emails mentioned by the parent don't sound like alarms. Because an alarm is usually for 'drop everything and focus on this' situations. The equivalent in email terms would be a receiving an email with a subject in ALL CAPS bolded and underlined. Or in general intra-company communication terms, a phone call from your boss without any pleasantries and a serious voice.
    - Volundr 4y ago
      Alarm or not doesn't really matter. If a person is receiving a signal that does not affect them most of the time they WILL start to ignore that signal. Many will attempt to combat this with policies and consequences, "Make sure your reading these e-mails, or else!" but it's a fruitless endeavor. Humans will human. Better to recognize that and build your systems around it.
      - MichaelZuo 4y ago
        So what? If someone makes the wrong decisions because they start ignoring signals then don't promote them or give them important coordinating responsibilities. Those who are capable of filtering out a larger fraction of noise do exist. Of course there will always be folks whose preference is to read near 0% of their emails, but that doesn't imply organizations must be designed around them.
        post-it 4y ago
        Finding someone to blame doesn't matter if the company goes bust.
        MichaelZuo 4y ago
        It's not about playing the blame-game, it's simply to make sure the personnel in responsible positions are those who can handle it, in a verifiable on-the-record way.
        Volundr 4y ago
        > If someone makes the wrong decisions because they start ignoring signals then don't promote them or give them important coordinating responsibilities. Those who are capable of filtering out a larger fraction of noise do exist. This is simply wishful thinking. Outliers certainly exist, but the idea that there are sufficient number of them that you can just ignore human nature is a path to disaster. You'd have to somehow accurately measure not just who is opening these noisey e-mails, but what they are retaining from them, and measure it over a large period of time, knowing that the vast majority or going to fail. It's far cheaper and more reliable to fix your noisey system than to try to outwit human nature.
- Karellen 4y ago
  It sounds like what you're suggesting is that, so long as you know who to blame for the problem, it doesn't really matter how bad the problem is when it hits you? Even if the company goes insolvent because of the problem, if you've got someone to point to and say "their fault", it's not a problem for the organisation? That... doesn't sound like a great approach to me.
  - MichaelZuo 4y ago
    It does presume there is someone, or some group, keeping track of the tally on at least a weekly basis. If no one is keeping track of who is doing well and who to replace, then fixing that first is more important.
    - Karellen 4y ago
      Well, there's supposed to be a group keeping track of the alerts about anyone who is doing badly, but one of them is ignoring 90% of their alerts, another 95%, and another 100%. It's OK though, because the system is built to handle that kind of problem - by keeping a paper trail of who is to blame. Marvellous. :-)
      - MichaelZuo 4y ago
        If the organization is so large that it needs even the tracking group to have its own internal hierarchy then I'm afraid my wisdom is lacking.
fishpen0 4y ago
It sounds like the actual problem is getting too many meaningless notifications, inadvertently training people to ignore everything
- randomswede 4y ago
  In an ideal situation, you'd get all "planned maintenance" emails for things you care about and no emails for the rest of them. That (probably) means that the system for dealing with planning maintenances (well, usually, "approving them") needs to have a sufficiently good understanding of what humans care about what changes. At a previous job, the planned change tracking system was REALLY good at tracking what specific compute facility was going to be impacted by any specific change taking place in that facility. And had a really good way of allowing you to filter for "only places I have stuff running" (and I think, even some breakdown of general change types as well). It was, however, not easy to get notification of "there will be maintenance on submarine cable C, taking it off-line for 4 hours" or "there will be maintenance at cable station CS, taking cables C1, C2, and C3 down for 3h". And as one of the things "we" (the team i worked in then) was doing was world-wide low latency replication of data, we did actually care that cable C was going to be down. But, the only way we could find out was "read all upcoming changes" and stick them in the team calendar. Was it good? Eh, it worked. Was it the best process I've seen? Probably ,yes.

MichaelZuo 4y ago

I don't see how that's a problem for the organization. Individual preferences do vary, one ignores 90%, another 95%, another 100%. And the one who's ignoring 100% of them will likely eventually make a mistake that otherwise wouldn't have happened. But it will be fairly straightforward to resolve, after all there's an extensive paper trail as the chain of custody seems clear. Assuming the "change management meeting" emails were the approved means of communication.

nordsieck 4y ago
> I don't see how that's a problem for the organization. IMO, one of the lessons that came out of Chernobyl is that it absolutely is a problem for the organization. Exposing people to too many "alarms" that are constantly going off will cause people to start ignoring them. Part of good design is figuring out which things are truly important, and how to communicate that to the people who are supposed to be paying attention.
- MichaelZuo 4y ago
  The analogy seems not to apply? The emails mentioned by the parent don't sound like alarms. Because an alarm is usually for 'drop everything and focus on this' situations. The equivalent in email terms would be a receiving an email with a subject in ALL CAPS bolded and underlined. Or in general intra-company communication terms, a phone call from your boss without any pleasantries and a serious voice.
  - Volundr 4y ago
    Alarm or not doesn't really matter. If a person is receiving a signal that does not affect them most of the time they WILL start to ignore that signal. Many will attempt to combat this with policies and consequences, "Make sure your reading these e-mails, or else!" but it's a fruitless endeavor. Humans will human. Better to recognize that and build your systems around it.
    - MichaelZuo 4y ago
      So what? If someone makes the wrong decisions because they start ignoring signals then don't promote them or give them important coordinating responsibilities. Those who are capable of filtering out a larger fraction of noise do exist. Of course there will always be folks whose preference is to read near 0% of their emails, but that doesn't imply organizations must be designed around them.
      - post-it 4y ago
        Finding someone to blame doesn't matter if the company goes bust.
        MichaelZuo 4y ago
        It's not about playing the blame-game, it's simply to make sure the personnel in responsible positions are those who can handle it, in a verifiable on-the-record way.
      - Volundr 4y ago
        > If someone makes the wrong decisions because they start ignoring signals then don't promote them or give them important coordinating responsibilities. Those who are capable of filtering out a larger fraction of noise do exist. This is simply wishful thinking. Outliers certainly exist, but the idea that there are sufficient number of them that you can just ignore human nature is a path to disaster. You'd have to somehow accurately measure not just who is opening these noisey e-mails, but what they are retaining from them, and measure it over a large period of time, knowing that the vast majority or going to fail. It's far cheaper and more reliable to fix your noisey system than to try to outwit human nature.
Karellen 4y ago
It sounds like what you're suggesting is that, so long as you know who to blame for the problem, it doesn't really matter how bad the problem is when it hits you? Even if the company goes insolvent because of the problem, if you've got someone to point to and say "their fault", it's not a problem for the organisation? That... doesn't sound like a great approach to me.
- MichaelZuo 4y ago
  It does presume there is someone, or some group, keeping track of the tally on at least a weekly basis. If no one is keeping track of who is doing well and who to replace, then fixing that first is more important.
  - Karellen 4y ago
    Well, there's supposed to be a group keeping track of the alerts about anyone who is doing badly, but one of them is ignoring 90% of their alerts, another 95%, and another 100%. It's OK though, because the system is built to handle that kind of problem - by keeping a paper trail of who is to blame. Marvellous. :-)
    - MichaelZuo 4y ago
      If the organization is so large that it needs even the tracking group to have its own internal hierarchy then I'm afraid my wisdom is lacking.

fishpen0 4y ago

It sounds like the actual problem is getting too many meaningless notifications, inadvertently training people to ignore everything

randomswede 4y ago
In an ideal situation, you'd get all "planned maintenance" emails for things you care about and no emails for the rest of them. That (probably) means that the system for dealing with planning maintenances (well, usually, "approving them") needs to have a sufficiently good understanding of what humans care about what changes. At a previous job, the planned change tracking system was REALLY good at tracking what specific compute facility was going to be impacted by any specific change taking place in that facility. And had a really good way of allowing you to filter for "only places I have stuff running" (and I think, even some breakdown of general change types as well). It was, however, not easy to get notification of "there will be maintenance on submarine cable C, taking it off-line for 4 hours" or "there will be maintenance at cable station CS, taking cables C1, C2, and C3 down for 3h". And as one of the things "we" (the team i worked in then) was doing was world-wide low latency replication of data, we did actually care that cable C was going to be down. But, the only way we could find out was "read all upcoming changes" and stick them in the team calendar. Was it good? Eh, it worked. Was it the best process I've seen? Probably ,yes.