5 ms·
In my negative experiences, "blameless" turned in to "nobody did anything wrong" which, of course, undermines the whole point of finding out what actually happe
by nanis 5y ago
In my negative experiences, "blameless" turned in to "nobody did anything wrong" which, of course, undermines the whole point of finding out what actually happened so we can see if there is a thing we can do to reduce the likelihood of it happening again.
Sometimes, the root cause is indeed someone with the privilege but not the good sense ignoring warning signs. If we can't identify that problem, then we can't improve our odds for the next time.
- joshuamorton 5y agoA valid blameless answer then is "remove the privilege" and yes, despite whatever objections you'll raise, this is possible. Difficult, but possible. Like, even in the case of the extreme example of someone deciding to intentionally harm the company. You fire them, but then what, how do you prevent the next person to go rogue from causing equivalent harm?
- tonyedgecombe 5y agoThis is what slows big companies down. They accrete so many of these rules and restrictions that people can't do their work.
- rocqua 5y agoI believe more of these incidents should conclude: "this is better to accept as the cost of doing business than to try to spend money to fix".
- concordDance 5y agoOr people start ignoring the rules in order to do their work. Which has it's own problems (particularly as rules normally aren't divided into "important" and "unimportant".
- jbuhbjlnjbn 5y agoThis is the thorn in my foot. I refuse to ignore company rules, and also comment on gross negligence, which more often then not means I am the quarreler, not the good engineer in the eyes of coworkers and bosses.
- Kalium 5y ago> They accrete so many of these rules and restrictions that people can't do their work. I hear this from software engineers, from time to time. What do you mean they aren't allowed to SSH into prod anymore? How will they debug, update, or maintain anything? Sometimes this is your standard-issue hostile reaction to change. The old approach is what they are used to and they don't understand the need to change it (and "ask to understand" mostly so they can try and negotiate). This new world just seems to get in the way for no clear reason. Management neither appreciates nor understands the reluctance and just pushes on. Usually what needs to happen is a series of changes across the org. You roll out the change with references to policy to support it. Workflows get updated and reworked so that SSHing into prod is not, in fact, the way to update systems or view logs or whatever. Most of all, educational materials are provided. Often I find that people object to changes because they don't know another way to work. If all you've ever known is SSHing into prod to read logs, you've probably never heard of Kibana or used OpenTelemetry.
- marcosdumay 5y agoPublic policy is just waking for this, and private administration is way too unprofessional as a group to formalize the problem (thus some people understand it, most don't), but rule creation requires a cost-benefit analysis. Any new rule you create is a chance to analyze the entire set, and maybe redesign it.
- vajrabum 5y agoThat kind of result may mean you need a Molly guard. The original Molly was a toddler who reportedly pushed the Big Red Switch on an IBM 4341 twice in one day and so they put a cover over the switch to put an end to that sort of outage. Occasionally people need firing but even in that sort of circumstance there's an organization problem that needs addressing.