3 ms·

> cannot happen again. Hubris. Does not inspire confidence. > We resolved the vulnerability within 26 hours of its initial report, and additional security aud

> cannot happen again. Hubris. Does not inspire confidence. > We resolved the vulnerability within 26 hours of its initial report, and additional security audits were completed by February 2025. After reading the vulnerability report, I am impressed at how quickly you guys jumped on the fix, so kudos. Did the security audit lead to any significant remediation work? If you weren't following PoLP, I wonder what else may have been overlooked?

davej 2y ago
Fair point. Perhaps better phrased as "to ensure this scenario can't recur.". I'll edit my post. Yes, we re-architected our build container as part of remediation efforts, it was quite significant.
- deleted 2y ago
  [deleted]
- ddingus 2y ago
  That was solid. Nice way to handle a direct personal judgement! Not your first rodeo. Another way is to avoid absolutes and ultimatums as aggressively as one should avoid personal judgements. Better phrased as: "we did our best to prevent this scenario from happening again. Fact is it just could happen! Nobody likes that reality, and overall when we think about all this stuff, networked computing is a sad state of affairs.. Best to just be 100 percent real about it all, if you ask me. At the very least people won't nail you on little things, which leaves you something you may trade on when a big thing happens. And yeah, this is unsolicited and worth exactly what you paid. Was just sharing where I ended up on these things in case it helps
- AzzyHN 2y ago
  You're still doing better than many larger teams handling larger projects :D
abhiaagarwal 2y ago
Based on the claims on the blog, it feels reasonable to say that this "cannot" occur again.
- beardedwizard 2y ago
  Based on which claim? That 12 months from now they might accidentally discover a new bug just as serious?
  - GavinMcG 2y ago
    If you think someone is obviously wrong, it might be worth pausing for a second and considering where you might just be referring to different things. Here, you seem to understand “this” to mean “a serious bug.” Since it’s obvious that a serious bug could happen, it seems likely that the author meant “this” to mean “the kind of bug that led to the breach we’re presently discussing.”
    - beardedwizard 2y ago
      I do not assume anyone is obviously wrong and prefer to ask questions. Most bugs exist in classes, and variants are something you typically consider when a bug results in a production incident. I'm not sure I read anything that makes me confident this class of bugs could never recur. I could be reasonably confident this _exact_ bug in this _exact_ scenario may not happen again, but that only makes me more concerned about variants that may have equal or more serious implications. So I'm wondering which claim did it for you? I only really saw pen test as a concrete action.
TZubiri 2y ago
[flagged]
- braiamp 2y ago
  This is the wrong response, because that means that the learning would be lost. The security community didn't want that to happen when one of the CA's got a vulnerability, we do not want it to happen to other companies. We want companies to succeed and get better, being shameful doesn't help towards that. Learning the right lessons does, and resigning means that you are learning the wrong ones.
  - TZubiri 2y ago
    I don't think the lesson is lost. The opposite. If you get a slap on the wrist, do you learn? No, you play it down. However if a dev who gets caught doing a bad is forced to resign. Then all the rest of the devs doing the same thing will shape up.
    - otterley 2y ago
      Under what theory of psychology are you operating? This is along the same lines as the theory that punishment is an effective deterrent of crime, which we know isn’t true from experience.
      - slibhb 2y ago
        While I think that resigning is stupid here, asserting that "punishment doesn't deter crime" is just absurd. It does!
        AdieuToLogic 2y ago
        > While I think that resigning is stupid here, asserting that "punishment doesn't deter crime" is just absurd. It does! Punishment does not deter crime. The threat of punishment does to a degree. IOW, most people will be unaware of a person being sent to prison for years until and unless they have committed a similar offense. But everyone is aware of repercussions possible should they violate known criminal laws.

davej 2y ago

Fair point. Perhaps better phrased as "to ensure this scenario can't recur.". I'll edit my post. Yes, we re-architected our build container as part of remediation efforts, it was quite significant.

deleted 2y ago
[deleted]
ddingus 2y ago
That was solid. Nice way to handle a direct personal judgement! Not your first rodeo. Another way is to avoid absolutes and ultimatums as aggressively as one should avoid personal judgements. Better phrased as: "we did our best to prevent this scenario from happening again. Fact is it just could happen! Nobody likes that reality, and overall when we think about all this stuff, networked computing is a sad state of affairs.. Best to just be 100 percent real about it all, if you ask me. At the very least people won't nail you on little things, which leaves you something you may trade on when a big thing happens. And yeah, this is unsolicited and worth exactly what you paid. Was just sharing where I ended up on these things in case it helps
AzzyHN 2y ago
You're still doing better than many larger teams handling larger projects :D

abhiaagarwal 2y ago

Based on the claims on the blog, it feels reasonable to say that this "cannot" occur again.

beardedwizard 2y ago
Based on which claim? That 12 months from now they might accidentally discover a new bug just as serious?
- GavinMcG 2y ago
  If you think someone is obviously wrong, it might be worth pausing for a second and considering where you might just be referring to different things. Here, you seem to understand “this” to mean “a serious bug.” Since it’s obvious that a serious bug could happen, it seems likely that the author meant “this” to mean “the kind of bug that led to the breach we’re presently discussing.”
  - beardedwizard 2y ago
    I do not assume anyone is obviously wrong and prefer to ask questions. Most bugs exist in classes, and variants are something you typically consider when a bug results in a production incident. I'm not sure I read anything that makes me confident this class of bugs could never recur. I could be reasonably confident this _exact_ bug in this _exact_ scenario may not happen again, but that only makes me more concerned about variants that may have equal or more serious implications. So I'm wondering which claim did it for you? I only really saw pen test as a concrete action.

TZubiri 2y ago

[flagged]

braiamp 2y ago
This is the wrong response, because that means that the learning would be lost. The security community didn't want that to happen when one of the CA's got a vulnerability, we do not want it to happen to other companies. We want companies to succeed and get better, being shameful doesn't help towards that. Learning the right lessons does, and resigning means that you are learning the wrong ones.
- TZubiri 2y ago
  I don't think the lesson is lost. The opposite. If you get a slap on the wrist, do you learn? No, you play it down. However if a dev who gets caught doing a bad is forced to resign. Then all the rest of the devs doing the same thing will shape up.
  - otterley 2y ago
    Under what theory of psychology are you operating? This is along the same lines as the theory that punishment is an effective deterrent of crime, which we know isn’t true from experience.
    - slibhb 2y ago
      While I think that resigning is stupid here, asserting that "punishment doesn't deter crime" is just absurd. It does!
      - AdieuToLogic 2y ago
        > While I think that resigning is stupid here, asserting that "punishment doesn't deter crime" is just absurd. It does! Punishment does not deter crime. The threat of punishment does to a degree. IOW, most people will be unaware of a person being sent to prison for years until and unless they have committed a similar offense. But everyone is aware of repercussions possible should they violate known criminal laws.