4 ms·

They often do. The value of those kinds of blanket security audits is questionable, however. (This is one of the reasons I'm generally pro-OSS for digital infr

They often do. The value of those kinds of blanket security audits is questionable, however. (This is one of the reasons I'm generally pro-OSS for digital infrastructure: security quickly becomes a compliance game at the scale of government, meaning that it's more about diligently completing checklists and demonstrating that diligence than about critically evaluating a component's security. OSS doesn't make software secure, but it does make it easier for the interested public to catch things before they become crises.)

deepsun 2y ago
Well, the value is ok, if considered seriously. Also, any certificate bears a certificator company name. We can always say "company A was hacked despite having its security certified by company B". So that company B at least share some blame.
- ethbr1 2y ago
  In practice, most commercial attestations/certifications contain enough weasel language that the certifier isn't responsible for anything missed (i.e. reasonable effort only). But yes, there are many standards for this (e.g. SOC Type 2 reports). In defense of their utility, the good ones tend to focus on (a) whether a control/policy for a sensitive operation exists at all in the product/company & (b) whether those controls implemented are effectively adhered to during an audited period.
  - r00fus 2y ago
    We're talking about getting a judgement in the court of public opinion not a court of law, and no one is exempt from the former.
    - ipaddr 2y ago
      Many live in a special labelled class that cannot be criticized
  - AmericanChopper 2y ago
    That’s not really how they work. The auditor attests that they were provided with evidence that the systems/business units audited were compliant at the time of auditing. That doesn’t mean that the business didn’t intentionally fake the evidence, or that the business is compliant at any time subsequent to the assessment. An auditor would certainly have some consequences if they were exposed for auditing negligently. This is how the PCI SSC manages to claim that no compliant merchant/service provider has ever been breached, because they assume being breached means that the breached party was non-compliant at the time of the breach. Which is probably a technically true statement, but is a bit misleading about what they’re actually claiming that means.
  - deepsun 2y ago
    Yes, certifiers are not responsible in legal sense, but nothing stops us from posting crap about them on internets.
doctorpangloss 2y ago
> The value of those kinds of blanket security audits is questionable, You're totally right. Why are people afraid to say that they're worthless? Why caveat or equivocate? Adversaries in computer security do not mince words.
- pinkmuffinere 2y ago
  “Worthless” is quite a strong claim. There isn’t much work I’ve encountered that’s truly “worthless”, even though bad work can make me quite upset. Anyways, that’s why I would often caveat.
  - stackskipton 2y ago
    I'll say they are worthless because most of time they are dragging time away from things that could improve security. For example, $LastJob we spent a ton of time on SOC2 compliance and despite having applications with known vulnerabilities, we got hacked and ended up all over the news. Maybe of instead of spending all the time getting SOC2 compliance finished, we could have worked at upgrading those apps. Actually, I doubt they would have upgraded the apps and pocketed the profits instead but SOC2 is providing cover instead of real change.
    - james_marks 2y ago
      SOC2 covers a set of vectors (mostly social/separation of controls from what I’ve seen), and you were attacked on another vector. Maybe the org prioritized poorly and sucks overall, but that doesn’t mean SOC2 or compliance generally is worthless.
      - stackskipton 2y ago
        >SOC2 covers a set of vectors (mostly social/separation of controls from what I’ve seen) THAT WAS THE PROBLEM. My bad, I thought most hacks were due poor software management but I'm glad SOC2 truly addressed the real problem.
kva-gad-fly 2y ago
Even if these govt. security audits are checkboxes, dont they require some nominal pentesting and black box testing, which test for things like SQL injection? That shoudl have caught these types of exposures?
- lvturner 2y ago
  It may not apply to this specific incident, but pen-testing only ensures you meet a minimum standard at a specific point in time. I almost feel I could write novels (if only I had time and could adequately structure my thoughts!) on this and adjacent topics but the simple fact is that the SDLC in a lot of enterprises/organizations is fundamentally broken, unfortunately a huge portion of what breaks it tends to occur long before a developer even starts bashing out some code.

deepsun 2y ago

Well, the value is ok, if considered seriously. Also, any certificate bears a certificator company name. We can always say "company A was hacked despite having its security certified by company B". So that company B at least share some blame.

ethbr1 2y ago
In practice, most commercial attestations/certifications contain enough weasel language that the certifier isn't responsible for anything missed (i.e. reasonable effort only). But yes, there are many standards for this (e.g. SOC Type 2 reports). In defense of their utility, the good ones tend to focus on (a) whether a control/policy for a sensitive operation exists at all in the product/company & (b) whether those controls implemented are effectively adhered to during an audited period.
- r00fus 2y ago
  We're talking about getting a judgement in the court of public opinion not a court of law, and no one is exempt from the former.
  - ipaddr 2y ago
    Many live in a special labelled class that cannot be criticized
- AmericanChopper 2y ago
  That’s not really how they work. The auditor attests that they were provided with evidence that the systems/business units audited were compliant at the time of auditing. That doesn’t mean that the business didn’t intentionally fake the evidence, or that the business is compliant at any time subsequent to the assessment. An auditor would certainly have some consequences if they were exposed for auditing negligently. This is how the PCI SSC manages to claim that no compliant merchant/service provider has ever been breached, because they assume being breached means that the breached party was non-compliant at the time of the breach. Which is probably a technically true statement, but is a bit misleading about what they’re actually claiming that means.
- deepsun 2y ago
  Yes, certifiers are not responsible in legal sense, but nothing stops us from posting crap about them on internets.

doctorpangloss 2y ago

> The value of those kinds of blanket security audits is questionable, You're totally right. Why are people afraid to say that they're worthless? Why caveat or equivocate? Adversaries in computer security do not mince words.

pinkmuffinere 2y ago
“Worthless” is quite a strong claim. There isn’t much work I’ve encountered that’s truly “worthless”, even though bad work can make me quite upset. Anyways, that’s why I would often caveat.
- stackskipton 2y ago
  I'll say they are worthless because most of time they are dragging time away from things that could improve security. For example, $LastJob we spent a ton of time on SOC2 compliance and despite having applications with known vulnerabilities, we got hacked and ended up all over the news. Maybe of instead of spending all the time getting SOC2 compliance finished, we could have worked at upgrading those apps. Actually, I doubt they would have upgraded the apps and pocketed the profits instead but SOC2 is providing cover instead of real change.
  - james_marks 2y ago
    SOC2 covers a set of vectors (mostly social/separation of controls from what I’ve seen), and you were attacked on another vector. Maybe the org prioritized poorly and sucks overall, but that doesn’t mean SOC2 or compliance generally is worthless.
    - stackskipton 2y ago
      >SOC2 covers a set of vectors (mostly social/separation of controls from what I’ve seen) THAT WAS THE PROBLEM. My bad, I thought most hacks were due poor software management but I'm glad SOC2 truly addressed the real problem.

kva-gad-fly 2y ago

Even if these govt. security audits are checkboxes, dont they require some nominal pentesting and black box testing, which test for things like SQL injection? That shoudl have caught these types of exposures?

lvturner 2y ago
It may not apply to this specific incident, but pen-testing only ensures you meet a minimum standard at a specific point in time. I almost feel I could write novels (if only I had time and could adequately structure my thoughts!) on this and adjacent topics but the simple fact is that the SDLC in a lot of enterprises/organizations is fundamentally broken, unfortunately a huge portion of what breaks it tends to occur long before a developer even starts bashing out some code.