9 ms·

Part of my job is to help the executives that I report to understand why things went wrong from the security perspective in our business unit. These are purely

Part of my job is to help the executives that I report to understand why things went wrong from the security perspective in our business unit. These are purely internal discussions, not even investigations. There are no penalties, but really, for things as egregious as hard coded passwords. As will become clear in a moment, the fact that my executives care is quite unusual. Culturally the result is coverups and lies. Engineers lie, managers lie, test people lie, directors lie, senior directors lie, vice president lie, external interesting teams are negotiated into minimizing certain critical failures, and so on. Managers don’t want to hear it so that they can’t be accused of lying, vice presidents don’t wanna know, SVP’s just want green squares on the cross-BU PowerPoint. This is internal discussion of revenue impacting incidents. Do you know what executives do care about? Revenue. Lost deals. If the people who care about money, including the account teams, don’t care about security and severe quality issues enough to be honest enough to get to improvement, how could an external board accomplish anything for those very few incidents that actually become publicly visible? This isn’t like the NTSB; I spent my life reading NTSB accident reports. They have actual real authority, there are potentially issues that might impact someone more than being caught distorting things.

izacus 5y ago
I also wonder if "blameless postmortem" culture perhaps actively works against preventing these kind of incidents. It doesn't seem that anyone in IT is ever responsible for damage they cause. But yes, lying, "not seeing" and covering documentation is pretty much standard corporate behaviour I've seen around plenty of companies as well.
- foobiekr 5y ago
  I no longer believe in blameless post mortem as a general rule. I have, through experience, come to believe that the contexts where blameless post mortems work are the contexts where literally anything works because they are organizations that have high hiring bars and high expectations. My current employer is not one of them; we are a mountain of mediocrity and all blameless post mortems do is act as an excuse to avoid raising the bar.
  - jolux 5y ago
    The principle of blameless postmortems is not supposed to absolve anyone of the responsibility to change anything, it’s supposed to foreground that serious failures are organizational failures first and foremost, because it’s the organization that has an obligation not to fail, not individuals, who fail all the time as a rule.
    - tialaramex 5y ago
      Exactly this. I spend a bunch of time reading accident reports from agencies like RAIB and MAIB, but my real jobs have been closer to the Web PKI and thus m.d.s.policy Back in 2015, Symantec's CA issued some certificates that shouldn't have existed, including for names owned by Google. What's wrong there? Well, a blameless postmortem would probably tell you that your processes and procedures are bad, you are creating bogus certificates to "test" a real CA whose certificates are actually trusted in the real world. Need better processes, training, oversight to ensure things improve. What did Symantec do when they were caught? They fired the low-level employees who conducted the tests and wrote a blog post "A Tough Day as Leaders" which blamed the fired employees for getting it wrong. Some leadership (the blog post of course no longer exists although I assume it's archived somewhere). Less than two years later, Symantec is back in trouble again because an RA they've worked with has been issuing certificates, using their CA infrastructure (and thus, from our point of view they were issuing these certificates even if they unaccountably believed this isn't their fault) that should not exist. This time Symantec's bosses blame not only low-level employees, but also auditors, bosses at the Korean RA, and anybody else they can think of... except themselves. This is a gross failure of leadership. Once upon a time a US President said "The buck stops here", but Donald Trump was very clear, "The buck stops with everybody" and "I take no responsibility" and it seems Symantec's leadership were made in that image. They quit the CA business rather than do what it would take to fix the problem. If you conduct a "postmortem" after an incident, then "Nobody was to blame and nothing needs to change" is almost certainly just as much the wrong outcome as "It's Jane's fault, fire her". I mentioned I read MAIB reports. One MAIB report sticks out, after many years, in the following way: Unlike every other MAIB report I've read, this one has No recommendations. Someone died, and yet there is nothing to recommend. Why not? Well, the cause is very simple, two men on a fishing boat took a lot of heroin, and their boat crashed, it sank and one died. No need to recommend that you shouldn't take heroin while operating a fishing boat since heroin is an illegal drug already and operating a vessel under the influence of drugs or alcohol is already a crime too. If your next "blameless portmortem" doesn't have any recommendations, ask yourself, was what happened already a crime? Are the people involved dead or in prison and so either way beyond the value of recommending a different course of action next time? No? Then we need to recommend how to actually avoid it happening again.
- nanis 5y ago
  In my negative experiences, "blameless" turned in to "nobody did anything wrong" which, of course, undermines the whole point of finding out what actually happened so we can see if there is a thing we can do to reduce the likelihood of it happening again. Sometimes, the root cause is indeed someone with the privilege but not the good sense ignoring warning signs. If we can't identify that problem, then we can't improve our odds for the next time.
  - joshuamorton 5y ago
    A valid blameless answer then is "remove the privilege" and yes, despite whatever objections you'll raise, this is possible. Difficult, but possible. Like, even in the case of the extreme example of someone deciding to intentionally harm the company. You fire them, but then what, how do you prevent the next person to go rogue from causing equivalent harm?
    - tonyedgecombe 5y ago
      This is what slows big companies down. They accrete so many of these rules and restrictions that people can't do their work.
      - rocqua 5y ago
        I believe more of these incidents should conclude: "this is better to accept as the cost of doing business than to try to spend money to fix".
      - concordDance 5y ago
        Or people start ignoring the rules in order to do their work. Which has it's own problems (particularly as rules normally aren't divided into "important" and "unimportant".
        jbuhbjlnjbn 5y ago
        This is the thorn in my foot. I refuse to ignore company rules, and also comment on gross negligence, which more often then not means I am the quarreler, not the good engineer in the eyes of coworkers and bosses.
slyall 5y ago
I think you are overestimating the importance of "revenue impacting incidents" to company employees. If the company makes a couple of million extra or less this year it doesn't effect the majority of workers. Their bonus isn't going up or down etc. And remember this incident has already happened. By contrast if a report comes out blaming the loss on a worker, department or division then that could have major consequences. No matter how "blameless" it is, come next round of bonuses, promotions or layoffs everybody knows it'll be factored into the decisions. So people don't have an incentive to make themselves look bad and unlike with the NTSB there is no legal powers or fear of causing deaths behind the investigation.
- laurent92 5y ago
  I understand, but it sounds like we are digging ourselves into the same hole as USSR workers who were not incentivized to deliver working products, when we do that. It’s a civilizational peril. How do we solve cooperation at large scale? Is the only way to watch large companies accumulate bored employees and constantly recreate “the small guy”, the startup, which will finally make things right, until they become too big to be incentivized?
  - WalterBright 5y ago
    That's exactly how it works, and why big companies do not grow until they consume the entire world. They become complacent and unable to change, and a startup eventually takes their business away.
    - saalweachter 5y ago
      It's funny that people expect management to function better because it's employee-owners of a capitalist corporation, instead of feudal lords or members of a communist central committee.
      - laurent92 5y ago
        The feudal owners in a startup situation are the marketplace owners who organize all startups to compete. The day we lose is the day all of those marketplaces themselves are cannot be replaced.
Aeolun 5y ago
I care about the issues (as an angineer), but my experience with raising any security issues is that it results in a lot of pain for me personally. You report something to the ‘security’ team, and suddenly you’re responsible for doing all the work, as well as the prime suspect in an investigation ‘why didn’t you fix this before, since when did we know about this?!’ I’m absolutely incentiviced to just let any issue lie until it is discovered, because then it actually is the security teams problem.
- arch-ninja 5y ago
  I was expelled from a university for reporting security flaws, the solution we all seek is the simplest one and for administrators it's easiest to hurt the people making noise which commonly results in the noise going away. fight-or-flight response at it's finest. Edit: reminder that for the "common folk" these "security issues" are not a 5-minute fix, they are fundamentally different realities which require every machine on the network to be re-checked before they can be used again. There is a clear communication failure between the ones who want security and the ones who want "security".
  - wkavey 5y ago
    Would like to hear the story behind this one.
    - Game_Ender 5y ago
      It’s common, a friend almost got expelled for reporting a flaw in the universities ID card system. That friend did not brake anything, they did not sneak into any protected spaces. Just discovered and validated a flaw and then reported it.
    - jacobolus 5y ago
      A high school friend (2 decades ago) told a teacher that the system keeping track of student grades was vulnerable to attack. The teacher asked the student to demonstrate by attacking the system and adjusting one of his test scores down by 1 point. My friend obliged, and the teacher reported the vulnerability to the administration. An administrator threatened my friend with expulsion, but when he proposed to go public with his story in response, they decided they wouldn't expel him. The resolution was "please don't tell anyone", and the vulnerability was never fixed.
jjtheblunt 5y ago
Do bureacracies essentially provide complexity in which dysfunction readily hides?
oceanghost 5y ago
As an engineer I found if I didn't lie and pretend to support the false reality of the managers above me, I wouldn't have that job long. Once we had a very large effort, creating a new hardware and software platform for the companies products. The CEO demanded it be done in 9 months, our estimate was accurate almost to the day-- 18 months. Politically it was easier for the PMs to agree with 9 months and then have 9 one-month delays it took, instead of getting fired for insisting on a good faith estimate. What do you think the security in that product was like? Security directly conflicts with the business model of cutting every corner possible and loading engineers as much as possible. A real security program would have to start with reasonable hours and goals.
hollerith 5y ago
I got curious about what legal authority the NTSB has. Here is what I found: >like a cop, the NTSB can secure an accident scene and keep others away. It can examine the aircraft wreckage. It can have aircraft parts tested to help determine why the accident happened. It can even subpoena evidence. . . . If there is a lawsuit concerning the crash, the NTSB will not get involved. Not only is the NTSB’s report of the accident’s probable cause inadmissible, but the NTSB investigators are prohibited by law from testifying in court, even if they are served with a subpoena. Source: https://www.aviationlawmonitor.com/2009/04/ntsb/the-ntsbs-limited-role-after-an-aircraft-accident/ https://www.aviationlawmonitor.com/2009/04/ntsb/the-ntsbs-li... >The NTSB may issue a subpoena, enforceable in Federal District Court . . . For purposes of the Health Insurance Portability and Accountability Act of 1996 (HIPAA), Public Law 104-191, and the regulations promulgated by the Department of Health and Human Services, 45 CFR 164.501 et seq., the NTSB is a “public health authority” to which protected health information may be disclosed by a HIPAA “covered entity” without the prior written authorization of the subject of the records. Source: https://www.law.cornell.edu/cfr/text/49/831.59 https://www.law.cornell.edu/cfr/text/49/831.59
cratermoon 5y ago
> There are no penalties, but really, for things as egregious as hard coded passwords Is this sentence missing a word or phrase?
ganafagol 5y ago
Let's not conflate security and safety here. The boards hailed by the article are all about investigating safety failures, and so is the advocated invetigation board. Security is a different beast. It's sub-par in many airplanes/power plants/... too.
pomian 5y ago
That's very disappointing. I've always brought up everyone up to think that of all the professions, you can always trust an engineer. They deal in facts. They don't lie. Perhaps it's as simple that in the real world, engineers can't lie. Because the world wouldn't work is they did.
indymike 5y ago
> Do you know what executives do care about? Revenue. Lost deals. This is often why engineering needs aren't covered. Things are presented as risks and expense, instead of in terms of revenue. A $10,000 expense actually wipes out $10,000 in net profit, so you need to generate revenue sufficient to create $10,000 in net profit. Most companies have really rosy gross margins, but really tight net margins, so a $100K expense will take $2.8 Million in revenue to offset it. Finally, there is how risk frames what you present. If you come at me with "this might happen" the other side of the coin is "this might not happen", and most managers will avoid the certainty of expense for the possibility of an expense. If you are working with a CEO, valuation is where it's at. Try to understand the swing in company valuation based on profit. Present to the CEO like this: "X is highly likely to happen within Y months, and it will impact $Z in net profits, leading to a change in company valuation of up to $N." Make sure $N is enough to matter. You just took 98% of the arguments against taking action off the table.
- an_opabinia 5y ago
  > Most companies have really rosy gross margins, but really tight net margins, so a $100K expense will take $2.8 Million in revenue to offset it. I don’t think you are interpreting things correctly here. This is both counterintuitive and wrong.
  - indymike 5y ago
    I'm pretty sure I'm not wrong. Source: I own three companies. The $100k and $2.8M were just examples (very thin net profit). It's equally likely you could have $100k in expense and need $200-$400K in sales to offset it if you are in a very high margin business. Regardless, what matters is that the unexpected expense will reduce profit, every time, and by framing risk in terms of profit you will help your CEO make better decisions.
    - an_opabinia 5y ago
      You’re not “framing the risk in terms of profit.” You’re talking about a set of equations in your head, that only you know, that come up with weird results like “it takes way more revenue to ‘offset’ an expense” because you’re holding “net” or “gross” or whatever margin constant. Which is really confusing and surprising and not how anyone I’ve ever met talks about this stuff, and they’d be equally puzzled and I don’t think would experience an illuminating or aha moment or whatever. Personally I found it a little interesting, I get what you are going for, but it’s so strange and not really useful or true.

izacus 5y ago

I also wonder if "blameless postmortem" culture perhaps actively works against preventing these kind of incidents. It doesn't seem that anyone in IT is ever responsible for damage they cause. But yes, lying, "not seeing" and covering documentation is pretty much standard corporate behaviour I've seen around plenty of companies as well.

foobiekr 5y ago
I no longer believe in blameless post mortem as a general rule. I have, through experience, come to believe that the contexts where blameless post mortems work are the contexts where literally anything works because they are organizations that have high hiring bars and high expectations. My current employer is not one of them; we are a mountain of mediocrity and all blameless post mortems do is act as an excuse to avoid raising the bar.
- jolux 5y ago
  The principle of blameless postmortems is not supposed to absolve anyone of the responsibility to change anything, it’s supposed to foreground that serious failures are organizational failures first and foremost, because it’s the organization that has an obligation not to fail, not individuals, who fail all the time as a rule.
  - tialaramex 5y ago
    Exactly this. I spend a bunch of time reading accident reports from agencies like RAIB and MAIB, but my real jobs have been closer to the Web PKI and thus m.d.s.policy Back in 2015, Symantec's CA issued some certificates that shouldn't have existed, including for names owned by Google. What's wrong there? Well, a blameless postmortem would probably tell you that your processes and procedures are bad, you are creating bogus certificates to "test" a real CA whose certificates are actually trusted in the real world. Need better processes, training, oversight to ensure things improve. What did Symantec do when they were caught? They fired the low-level employees who conducted the tests and wrote a blog post "A Tough Day as Leaders" which blamed the fired employees for getting it wrong. Some leadership (the blog post of course no longer exists although I assume it's archived somewhere). Less than two years later, Symantec is back in trouble again because an RA they've worked with has been issuing certificates, using their CA infrastructure (and thus, from our point of view they were issuing these certificates even if they unaccountably believed this isn't their fault) that should not exist. This time Symantec's bosses blame not only low-level employees, but also auditors, bosses at the Korean RA, and anybody else they can think of... except themselves. This is a gross failure of leadership. Once upon a time a US President said "The buck stops here", but Donald Trump was very clear, "The buck stops with everybody" and "I take no responsibility" and it seems Symantec's leadership were made in that image. They quit the CA business rather than do what it would take to fix the problem. If you conduct a "postmortem" after an incident, then "Nobody was to blame and nothing needs to change" is almost certainly just as much the wrong outcome as "It's Jane's fault, fire her". I mentioned I read MAIB reports. One MAIB report sticks out, after many years, in the following way: Unlike every other MAIB report I've read, this one has No recommendations. Someone died, and yet there is nothing to recommend. Why not? Well, the cause is very simple, two men on a fishing boat took a lot of heroin, and their boat crashed, it sank and one died. No need to recommend that you shouldn't take heroin while operating a fishing boat since heroin is an illegal drug already and operating a vessel under the influence of drugs or alcohol is already a crime too. If your next "blameless portmortem" doesn't have any recommendations, ask yourself, was what happened already a crime? Are the people involved dead or in prison and so either way beyond the value of recommending a different course of action next time? No? Then we need to recommend how to actually avoid it happening again.
nanis 5y ago
In my negative experiences, "blameless" turned in to "nobody did anything wrong" which, of course, undermines the whole point of finding out what actually happened so we can see if there is a thing we can do to reduce the likelihood of it happening again. Sometimes, the root cause is indeed someone with the privilege but not the good sense ignoring warning signs. If we can't identify that problem, then we can't improve our odds for the next time.
- joshuamorton 5y ago
  A valid blameless answer then is "remove the privilege" and yes, despite whatever objections you'll raise, this is possible. Difficult, but possible. Like, even in the case of the extreme example of someone deciding to intentionally harm the company. You fire them, but then what, how do you prevent the next person to go rogue from causing equivalent harm?
  - tonyedgecombe 5y ago
    This is what slows big companies down. They accrete so many of these rules and restrictions that people can't do their work.
    - rocqua 5y ago
      I believe more of these incidents should conclude: "this is better to accept as the cost of doing business than to try to spend money to fix".
    - concordDance 5y ago
      Or people start ignoring the rules in order to do their work. Which has it's own problems (particularly as rules normally aren't divided into "important" and "unimportant".
      - jbuhbjlnjbn 5y ago
        This is the thorn in my foot. I refuse to ignore company rules, and also comment on gross negligence, which more often then not means I am the quarreler, not the good engineer in the eyes of coworkers and bosses.

slyall 5y ago

I think you are overestimating the importance of "revenue impacting incidents" to company employees. If the company makes a couple of million extra or less this year it doesn't effect the majority of workers. Their bonus isn't going up or down etc. And remember this incident has already happened. By contrast if a report comes out blaming the loss on a worker, department or division then that could have major consequences. No matter how "blameless" it is, come next round of bonuses, promotions or layoffs everybody knows it'll be factored into the decisions. So people don't have an incentive to make themselves look bad and unlike with the NTSB there is no legal powers or fear of causing deaths behind the investigation.

laurent92 5y ago
I understand, but it sounds like we are digging ourselves into the same hole as USSR workers who were not incentivized to deliver working products, when we do that. It’s a civilizational peril. How do we solve cooperation at large scale? Is the only way to watch large companies accumulate bored employees and constantly recreate “the small guy”, the startup, which will finally make things right, until they become too big to be incentivized?
- WalterBright 5y ago
  That's exactly how it works, and why big companies do not grow until they consume the entire world. They become complacent and unable to change, and a startup eventually takes their business away.
  - saalweachter 5y ago
    It's funny that people expect management to function better because it's employee-owners of a capitalist corporation, instead of feudal lords or members of a communist central committee.
    - laurent92 5y ago
      The feudal owners in a startup situation are the marketplace owners who organize all startups to compete. The day we lose is the day all of those marketplaces themselves are cannot be replaced.

Aeolun 5y ago

I care about the issues (as an angineer), but my experience with raising any security issues is that it results in a lot of pain for me personally. You report something to the ‘security’ team, and suddenly you’re responsible for doing all the work, as well as the prime suspect in an investigation ‘why didn’t you fix this before, since when did we know about this?!’ I’m absolutely incentiviced to just let any issue lie until it is discovered, because then it actually is the security teams problem.

arch-ninja 5y ago
I was expelled from a university for reporting security flaws, the solution we all seek is the simplest one and for administrators it's easiest to hurt the people making noise which commonly results in the noise going away. fight-or-flight response at it's finest. Edit: reminder that for the "common folk" these "security issues" are not a 5-minute fix, they are fundamentally different realities which require every machine on the network to be re-checked before they can be used again. There is a clear communication failure between the ones who want security and the ones who want "security".
- wkavey 5y ago
  Would like to hear the story behind this one.
  - Game_Ender 5y ago
    It’s common, a friend almost got expelled for reporting a flaw in the universities ID card system. That friend did not brake anything, they did not sneak into any protected spaces. Just discovered and validated a flaw and then reported it.
  - jacobolus 5y ago
    A high school friend (2 decades ago) told a teacher that the system keeping track of student grades was vulnerable to attack. The teacher asked the student to demonstrate by attacking the system and adjusting one of his test scores down by 1 point. My friend obliged, and the teacher reported the vulnerability to the administration. An administrator threatened my friend with expulsion, but when he proposed to go public with his story in response, they decided they wouldn't expel him. The resolution was "please don't tell anyone", and the vulnerability was never fixed.

jjtheblunt 5y ago

Do bureacracies essentially provide complexity in which dysfunction readily hides?

oceanghost 5y ago

As an engineer I found if I didn't lie and pretend to support the false reality of the managers above me, I wouldn't have that job long. Once we had a very large effort, creating a new hardware and software platform for the companies products. The CEO demanded it be done in 9 months, our estimate was accurate almost to the day-- 18 months. Politically it was easier for the PMs to agree with 9 months and then have 9 one-month delays it took, instead of getting fired for insisting on a good faith estimate. What do you think the security in that product was like? Security directly conflicts with the business model of cutting every corner possible and loading engineers as much as possible. A real security program would have to start with reasonable hours and goals.

hollerith 5y ago

I got curious about what legal authority the NTSB has. Here is what I found: >like a cop, the NTSB can secure an accident scene and keep others away. It can examine the aircraft wreckage. It can have aircraft parts tested to help determine why the accident happened. It can even subpoena evidence. . . . If there is a lawsuit concerning the crash, the NTSB will not get involved. Not only is the NTSB’s report of the accident’s probable cause inadmissible, but the NTSB investigators are prohibited by law from testifying in court, even if they are served with a subpoena. Source: https://www.aviationlawmonitor.com/2009/04/ntsb/the-ntsbs-limited-role-after-an-aircraft-accident/ https://www.aviationlawmonitor.com/2009/04/ntsb/the-ntsbs-li... >The NTSB may issue a subpoena, enforceable in Federal District Court . . . For purposes of the Health Insurance Portability and Accountability Act of 1996 (HIPAA), Public Law 104-191, and the regulations promulgated by the Department of Health and Human Services, 45 CFR 164.501 et seq., the NTSB is a “public health authority” to which protected health information may be disclosed by a HIPAA “covered entity” without the prior written authorization of the subject of the records. Source: https://www.law.cornell.edu/cfr/text/49/831.59 https://www.law.cornell.edu/cfr/text/49/831.59

cratermoon 5y ago

> There are no penalties, but really, for things as egregious as hard coded passwords Is this sentence missing a word or phrase?

ganafagol 5y ago

Let's not conflate security and safety here. The boards hailed by the article are all about investigating safety failures, and so is the advocated invetigation board. Security is a different beast. It's sub-par in many airplanes/power plants/... too.

pomian 5y ago

That's very disappointing. I've always brought up everyone up to think that of all the professions, you can always trust an engineer. They deal in facts. They don't lie. Perhaps it's as simple that in the real world, engineers can't lie. Because the world wouldn't work is they did.

indymike 5y ago

> Do you know what executives do care about? Revenue. Lost deals. This is often why engineering needs aren't covered. Things are presented as risks and expense, instead of in terms of revenue. A $10,000 expense actually wipes out $10,000 in net profit, so you need to generate revenue sufficient to create $10,000 in net profit. Most companies have really rosy gross margins, but really tight net margins, so a $100K expense will take $2.8 Million in revenue to offset it. Finally, there is how risk frames what you present. If you come at me with "this might happen" the other side of the coin is "this might not happen", and most managers will avoid the certainty of expense for the possibility of an expense. If you are working with a CEO, valuation is where it's at. Try to understand the swing in company valuation based on profit. Present to the CEO like this: "X is highly likely to happen within Y months, and it will impact $Z in net profits, leading to a change in company valuation of up to $N." Make sure $N is enough to matter. You just took 98% of the arguments against taking action off the table.

an_opabinia 5y ago
> Most companies have really rosy gross margins, but really tight net margins, so a $100K expense will take $2.8 Million in revenue to offset it. I don’t think you are interpreting things correctly here. This is both counterintuitive and wrong.
- indymike 5y ago
  I'm pretty sure I'm not wrong. Source: I own three companies. The $100k and $2.8M were just examples (very thin net profit). It's equally likely you could have $100k in expense and need $200-$400K in sales to offset it if you are in a very high margin business. Regardless, what matters is that the unexpected expense will reduce profit, every time, and by framing risk in terms of profit you will help your CEO make better decisions.
  - an_opabinia 5y ago
    You’re not “framing the risk in terms of profit.” You’re talking about a set of equations in your head, that only you know, that come up with weird results like “it takes way more revenue to ‘offset’ an expense” because you’re holding “net” or “gross” or whatever margin constant. Which is really confusing and surprising and not how anyone I’ve ever met talks about this stuff, and they’d be equally puzzled and I don’t think would experience an illuminating or aha moment or whatever. Personally I found it a little interesting, I get what you are going for, but it’s so strange and not really useful or true.