4 ms·

I'm with amir in comment 23 and with Aaron in previous comments. Stuff happens. And when there are multiple moving pieces, the process and policies are the issu

I'm with amir in comment 23 and with Aaron in previous comments. Stuff happens. And when there are multiple moving pieces, the process and policies are the issue, not the individuals. Since individuals rarely have a complete overview of the entire system. Also, as noted in the comments, it sets a bad precedent for people coming forward reporting issues.

foolfoolz 2y ago
jeremy rowley has been C level at digi cert for over 7 years and is/was CISO. if anyone should have a completely overview of the entire system. it’s him accountability at the top is a good thing
- Spivak 2y ago
  And this guy is the gold standard for accountability based on these comments. Whoever pressured him to resign I think is making a mistake. Bugs happen, and he's doing exactly what's expected of a leader in this situation. Anyone who thinks "this incident is personally my fault because I didn't read every line of code the devs who work for me wrote, and for this dishonor I am now unfit to lead" is a sane reaction to these events is not living the blameless postmortem life.
  - fizx 2y ago
    Seems like he fell on his own sword.
    - wizzwizz4 2y ago
      It's always the way. The people with honour tend to hold themselves to a high standard, and step back when they find they do not meet it. Their replacements are either the same in this regard, or they're not. Captains used to go down with their ships. In most organisations, this is no longer the case, because we lost all the captains willing to do so, without replacement. Resigning when you fail to prevent an incident rarely helps, directly. But it's not something the power-hungry do, unless forced (and if they expect to be forced, they will try to cover things up). It rarely fails to win my respect. As a move in a social game, I suspect that the general strategy "resign when an unacceptable failure has occurred" makes things better, overall, even if it doesn't directly benefit the organisation you're leaving. (I'm not sure whether this applies when you don't expect that your replacement feels the same way about duty.)
  - hackernewds 2y ago
    maybe he resigned on his own volition or they found systematic issues pertinent to him? either speculation is likely, unless you have counter evidence
    - braiamp 2y ago
      He explicitly expressed that he's taking the knife for the DigiCert failures. There's no need to speculate further than that.
  - czbond 2y ago
    Last I looked, a CISO's tenure was only 3-5 years (I'm being generous) because the stress is incredible and you end up being responsible to every other CxO in the company. You have much power, but little as well because every department claims security is the reason their projects are delayed - or if they move quickly and there is an issue, they point back at CISO.
  - hackernewds 2y ago
    see here, https://news.ycombinator.com/item?id=41177521 https://news.ycombinator.com/item?id=41177521 that it were self initiated
- ahnick 2y ago
  Questions I have in these types of situations: * Does replacing the CISO actually make the system more secure? Presumably he had a lot of tribal knowledge built-up and who is going to know the system better than him? * As systems get more and more complex, it's likely impossible for a single individual to truly understand and prevent these types of situations 100% of the time. It seems that any application that needs to be 100% secure (if that is even possible) has to be provably secure in a strict mathematical sense, which goes beyond individual culpability. * Does shooting the person accountable actually encourage responsible disclosure or discourage it?
  - fnimick 2y ago
    This one C level individual failed to do the most important part of the job, which is to build the team of people who have shared knowledge to understand and prevent these issues 100% of the time.
    - jaas 2y ago
      No team is going to prevent issues like this 100% of the time. That's a wildly unrealistic expectation. Wherever the bar is, it won't be 100%. That's why good leadership invests in the ability to respond well to mistakes that will inevitably be made.
      - chmod775 2y ago
        Still. These incidents should be so rare, that when they occur, it is more likely to be leadership failure than a series of unfortunate events. This replacing the leadership is always the right response.
  - HarryHirsch 2y ago
    Does replacing the CISO actually make the system more secure? Counterintuitively, probably yes. Tone flows from the top down, and if you want to change the tone you need to start at the top. It's very difficult to try and build a coalition to change the system from underneath. Presumably he had a lot of tribal knowledge built-up and who is going to know the system better than him? Likely he has a lot of political influence and knowledge of the system and for lasting change all of that has to go. If it has gotten that bad it's no good and needs to be swept away.
- exac 2y ago
  According to his LinkedIn he joined DigiCert as a lawyer. This is an organizational failure of DigiCert's leadership to put a non-technical person in the CISO role.
  - gonzo41 2y ago
    Anyone technical probably wanted 10X the CEO's pay for taking on such a huge responsibility as CISO for DigiCert.
    - behringer 2y ago
      This event may make up for that pay discrepancy.
  - czbond 2y ago
    Enough companies are looking for their CISO to be an attorney, or to also be an attorney, because you spend a lot of your time threading through laws, contracts, policies, company risks, and parter risks, etc. Much of it at that level is NOT architecture and software discussions. You wouldn't think the job would be similar to lead counsel, but unfortunately a majority of a certain company's risk now a days is in that area.
whalesalad 2y ago
> it sets a bad precedent for people coming forward reporting issues. Disagree entirely. Shit happens, and it’s important to accurately report things going on. Sure someone could interpret this as “we should fire the people who make mistakes” but I think it’s unrealistic to begin with and if it were to happen people would see through it.
Waterluvian 2y ago
Who better to hold to account for a failure to have proper processes in place?
nine_zeros 2y ago
Putting blame on individuals is the fastest path to corporate collapse. It usually happens when people up the ladder want to shrug off their responsibility completely. Loser mentality.
behringer 2y ago
Yeah what a terrible mistake to resign and for the CEO to accept such a resignation. Shows a complete lack of understanding of what accountability actually means and shows that Amit (CEO) has no idea how to handle a critical security event.
- timetopay 2y ago
  It also shows a (potential) lack of understanding of what's happening internally at the upper levels DigiCert. This might have been a known issue or some other very poorly handled thing, is part of a larger pattern or issue, and he's been given the chance to "resign". Don't condemn people until you have all the facts, which multiple reporters are working on figuring out right now.
  - behringer 2y ago
    They're the ones who gave us the facts. If they want to cover up or lie about it that's on them.
- ericjmorey 2y ago
  Seems like the kind of mistake that a CEO might resign over.
travisjungroth 2y ago
> the process and policies are the issue Those processes and policies are set by someone. From his resignation: > This incident made me realize that I am no longer the person for this role. Seems reasonable. I also find it ironic to discourage this because of the precedent it sets. That’s making him responsible for things out of his control.
xyst 2y ago
We don't know what happened in those board rooms between CISO and CEO. It's not clear if this was _strongly_ impressed upon Jeremy to resign because the market needs a blood sacrifice; or if Jeremy really did on his own will commit the corporate equivalent of seppuku. In the wake of bringing great shame, he just felt it no longer necessary for him to lead them to the future. Honestly, I can understand that. I can respect it too. I can imagine the stress over that period of time being very high. Personal attacks, business attacks, mounting legal issues from business _partners_. We work in computers sitting on our ass. We are not used to these stressful events. This was probably Jermey's _stress test_ and he unfortunately broke. I just hope he finds another gig or maybe starts up something on his own. I would probably follow him.
gklitz 2y ago
> Since individuals rarely have a complete overview of the entire system. I think the key phrase you are missing which is repeated many places in the timeline is “Engineering was not consulted”. His failure wasn’t that he doesn’t know the entire system, it was that he kept engineering in the dark, “investigated” on his own, and made conclusions to disregard the persistent researcher who kept banging at the door saying “there is an issue” despite him disregarding it again and again, notably without consulting engineering. You don’t need to know the entire system as CISO, but you certainly do need to know that you don’t know the entire system as CISO. This wasn’t an institutional failure, it was a personal failure at a very high level. I’m not arguing that the guy had to resign, merely pointing out that the narrative used to argue against his resignation is flawed.
- throwaway7ahgb 2y ago
  Agreed here, and how can you thoroughly investigate without engineering involved? Unless you suspect your own team somehow it doesn't make sense.
- braiamp 2y ago
  When I said "overview" I mostly meant both vision (as in observability) and understanding of everything. I would have said overseer, but then I would ask for the reader to crack the thesaurus. To not digress anymore, CISO not only need to have proper knowledge of what happens, but understand why happens, how happens and in the several contexts that happens. And that's hard.

foolfoolz 2y ago

jeremy rowley has been C level at digi cert for over 7 years and is/was CISO. if anyone should have a completely overview of the entire system. it’s him accountability at the top is a good thing

Spivak 2y ago
And this guy is the gold standard for accountability based on these comments. Whoever pressured him to resign I think is making a mistake. Bugs happen, and he's doing exactly what's expected of a leader in this situation. Anyone who thinks "this incident is personally my fault because I didn't read every line of code the devs who work for me wrote, and for this dishonor I am now unfit to lead" is a sane reaction to these events is not living the blameless postmortem life.
- fizx 2y ago
  Seems like he fell on his own sword.
  - wizzwizz4 2y ago
    It's always the way. The people with honour tend to hold themselves to a high standard, and step back when they find they do not meet it. Their replacements are either the same in this regard, or they're not. Captains used to go down with their ships. In most organisations, this is no longer the case, because we lost all the captains willing to do so, without replacement. Resigning when you fail to prevent an incident rarely helps, directly. But it's not something the power-hungry do, unless forced (and if they expect to be forced, they will try to cover things up). It rarely fails to win my respect. As a move in a social game, I suspect that the general strategy "resign when an unacceptable failure has occurred" makes things better, overall, even if it doesn't directly benefit the organisation you're leaving. (I'm not sure whether this applies when you don't expect that your replacement feels the same way about duty.)
- hackernewds 2y ago
  maybe he resigned on his own volition or they found systematic issues pertinent to him? either speculation is likely, unless you have counter evidence
  - braiamp 2y ago
    He explicitly expressed that he's taking the knife for the DigiCert failures. There's no need to speculate further than that.
- czbond 2y ago
  Last I looked, a CISO's tenure was only 3-5 years (I'm being generous) because the stress is incredible and you end up being responsible to every other CxO in the company. You have much power, but little as well because every department claims security is the reason their projects are delayed - or if they move quickly and there is an issue, they point back at CISO.
- hackernewds 2y ago
  see here, https://news.ycombinator.com/item?id=41177521 https://news.ycombinator.com/item?id=41177521 that it were self initiated
ahnick 2y ago
Questions I have in these types of situations: * Does replacing the CISO actually make the system more secure? Presumably he had a lot of tribal knowledge built-up and who is going to know the system better than him? * As systems get more and more complex, it's likely impossible for a single individual to truly understand and prevent these types of situations 100% of the time. It seems that any application that needs to be 100% secure (if that is even possible) has to be provably secure in a strict mathematical sense, which goes beyond individual culpability. * Does shooting the person accountable actually encourage responsible disclosure or discourage it?
- fnimick 2y ago
  This one C level individual failed to do the most important part of the job, which is to build the team of people who have shared knowledge to understand and prevent these issues 100% of the time.
  - jaas 2y ago
    No team is going to prevent issues like this 100% of the time. That's a wildly unrealistic expectation. Wherever the bar is, it won't be 100%. That's why good leadership invests in the ability to respond well to mistakes that will inevitably be made.
    - chmod775 2y ago
      Still. These incidents should be so rare, that when they occur, it is more likely to be leadership failure than a series of unfortunate events. This replacing the leadership is always the right response.
- HarryHirsch 2y ago
  Does replacing the CISO actually make the system more secure? Counterintuitively, probably yes. Tone flows from the top down, and if you want to change the tone you need to start at the top. It's very difficult to try and build a coalition to change the system from underneath. Presumably he had a lot of tribal knowledge built-up and who is going to know the system better than him? Likely he has a lot of political influence and knowledge of the system and for lasting change all of that has to go. If it has gotten that bad it's no good and needs to be swept away.
exac 2y ago
According to his LinkedIn he joined DigiCert as a lawyer. This is an organizational failure of DigiCert's leadership to put a non-technical person in the CISO role.
- gonzo41 2y ago
  Anyone technical probably wanted 10X the CEO's pay for taking on such a huge responsibility as CISO for DigiCert.
  - behringer 2y ago
    This event may make up for that pay discrepancy.
- czbond 2y ago
  Enough companies are looking for their CISO to be an attorney, or to also be an attorney, because you spend a lot of your time threading through laws, contracts, policies, company risks, and parter risks, etc. Much of it at that level is NOT architecture and software discussions. You wouldn't think the job would be similar to lead counsel, but unfortunately a majority of a certain company's risk now a days is in that area.

whalesalad 2y ago

> it sets a bad precedent for people coming forward reporting issues. Disagree entirely. Shit happens, and it’s important to accurately report things going on. Sure someone could interpret this as “we should fire the people who make mistakes” but I think it’s unrealistic to begin with and if it were to happen people would see through it.

Waterluvian 2y ago

Who better to hold to account for a failure to have proper processes in place?

nine_zeros 2y ago

Putting blame on individuals is the fastest path to corporate collapse. It usually happens when people up the ladder want to shrug off their responsibility completely. Loser mentality.

behringer 2y ago

Yeah what a terrible mistake to resign and for the CEO to accept such a resignation. Shows a complete lack of understanding of what accountability actually means and shows that Amit (CEO) has no idea how to handle a critical security event.

timetopay 2y ago
It also shows a (potential) lack of understanding of what's happening internally at the upper levels DigiCert. This might have been a known issue or some other very poorly handled thing, is part of a larger pattern or issue, and he's been given the chance to "resign". Don't condemn people until you have all the facts, which multiple reporters are working on figuring out right now.
- behringer 2y ago
  They're the ones who gave us the facts. If they want to cover up or lie about it that's on them.
ericjmorey 2y ago
Seems like the kind of mistake that a CEO might resign over.

travisjungroth 2y ago

> the process and policies are the issue Those processes and policies are set by someone. From his resignation: > This incident made me realize that I am no longer the person for this role. Seems reasonable. I also find it ironic to discourage this because of the precedent it sets. That’s making him responsible for things out of his control.

xyst 2y ago

We don't know what happened in those board rooms between CISO and CEO. It's not clear if this was _strongly_ impressed upon Jeremy to resign because the market needs a blood sacrifice; or if Jeremy really did on his own will commit the corporate equivalent of seppuku. In the wake of bringing great shame, he just felt it no longer necessary for him to lead them to the future. Honestly, I can understand that. I can respect it too. I can imagine the stress over that period of time being very high. Personal attacks, business attacks, mounting legal issues from business _partners_. We work in computers sitting on our ass. We are not used to these stressful events. This was probably Jermey's _stress test_ and he unfortunately broke. I just hope he finds another gig or maybe starts up something on his own. I would probably follow him.

gklitz 2y ago

> Since individuals rarely have a complete overview of the entire system. I think the key phrase you are missing which is repeated many places in the timeline is “Engineering was not consulted”. His failure wasn’t that he doesn’t know the entire system, it was that he kept engineering in the dark, “investigated” on his own, and made conclusions to disregard the persistent researcher who kept banging at the door saying “there is an issue” despite him disregarding it again and again, notably without consulting engineering. You don’t need to know the entire system as CISO, but you certainly do need to know that you don’t know the entire system as CISO. This wasn’t an institutional failure, it was a personal failure at a very high level. I’m not arguing that the guy had to resign, merely pointing out that the narrative used to argue against his resignation is flawed.

throwaway7ahgb 2y ago
Agreed here, and how can you thoroughly investigate without engineering involved? Unless you suspect your own team somehow it doesn't make sense.
braiamp 2y ago
When I said "overview" I mostly meant both vision (as in observability) and understanding of everything. I would have said overseer, but then I would ask for the reader to crack the thesaurus. To not digress anymore, CISO not only need to have proper knowledge of what happens, but understand why happens, how happens and in the several contexts that happens. And that's hard.