4 ms·

This is true, but security teams often work on tooling dedicated to reduce the n. of CVEs so that a company can keep compliance. That is in fact part of complia

by mk89 4mo ago

This is true, but security teams often work on tooling dedicated to reduce the n. of CVEs so that a company can keep compliance. That is in fact part of compliance itself to have an automated/reliable processo to tackle CVEs...

tptacek 4mo ago
Which compliance regime are you referring to that cares about CVE counts as a metric?
- mk89 4mo ago
  Not as a metric, but it basically becomes one, like with Fedramp. You need to fix also moderate/low CVEs within a certain time frame. So CVE count becomes relevant, because the target is zero, although it doesn't mandate "zero CVEs" but that's finally what the desired outcome is. It's basically unrealistic to ignore that number, because it's unlikely that you have a steady 1000 CVEs (that are being continuously fixed and new ones discovered), but more like "a few exceptions".
  - tptacek 4mo ago
    I don't do FedRAMP and will have to take your word for that, but none of SOC2, 27001, or HIPAA/HITRUST care about CVE counts.
    - kasey_junk 4mo ago
      PCI doesn’t mention cve by name but does require vulnerability accounting and requires action if they are found, the action required driven by severity. I could see a (poor) control being written around keeping counts down.
      - tptacek 4mo ago
        Right, you can write a bad SOC2 control that cares about CVE counts too!
        tialaramex 4mo ago
        I've met good auditors. I mean, I've met terrible auditors too, but the good ones stick in my mind more because they ask insightful questions about my software or sometimes software in general. It's a problem that this is often seen as a box ticking exercise, done right it can be a really great opportunity to improve but so often instead the priority is to get the paperwork done and too bad if you achieved nothing by it.
        tptacek 4mo ago
        If we're talking about actual auditors, not tech consultants who call themselves auditors but people actually trained as auditors, I'd take it as a bad sign if they asked a bunch of specific unbidden questions about software details. That's not the job.
        tialaramex 4mo ago
        The specific example I'm thinking of most strongly is when we were purchased one of the auditors looking over our software noticed that unlike most of the big company's other software at that time (about 2012) ours was HTTPS-only. Their checklist told them they need only to check there's HTTPS for authentication pages, and they asked if that's actually enough, is it fine if everything else is just plain HTTP as they've seen elsewhere and as their checklist asks? I of course said it isn't, because as we both know, it isn't. "That's not the job" is I think the most useless possible observation here. The best outcome from audit isn't that you checked all the boxes, that's just resources expended for no benefit, the best outcome is that audit found a nasty problem early so that you could fix it now. The biggest problem we have in the Web PKI with auditors is that they'd so much rather tick boxes than tell their client - who they are billing $$$ - where the problems are. This presumably feels good to the suits, but if there's a problem and the auditors don't tell you the chances are somebody else finds it and then you're in worse trouble.
1718627440 4mo ago
But that's not a wrong approach. First you want to as many vulnerabilities as you can, than you want to fix as many as you can. If you rate the developing department for that, that's another story.

by mk89 4mo ago

tptacek 4mo ago

Which compliance regime are you referring to that cares about CVE counts as a metric?

mk89 4mo ago
Not as a metric, but it basically becomes one, like with Fedramp. You need to fix also moderate/low CVEs within a certain time frame. So CVE count becomes relevant, because the target is zero, although it doesn't mandate "zero CVEs" but that's finally what the desired outcome is. It's basically unrealistic to ignore that number, because it's unlikely that you have a steady 1000 CVEs (that are being continuously fixed and new ones discovered), but more like "a few exceptions".
- tptacek 4mo ago
  I don't do FedRAMP and will have to take your word for that, but none of SOC2, 27001, or HIPAA/HITRUST care about CVE counts.
  - kasey_junk 4mo ago
    PCI doesn’t mention cve by name but does require vulnerability accounting and requires action if they are found, the action required driven by severity. I could see a (poor) control being written around keeping counts down.
    - tptacek 4mo ago
      Right, you can write a bad SOC2 control that cares about CVE counts too!
      - tialaramex 4mo ago
        I've met good auditors. I mean, I've met terrible auditors too, but the good ones stick in my mind more because they ask insightful questions about my software or sometimes software in general. It's a problem that this is often seen as a box ticking exercise, done right it can be a really great opportunity to improve but so often instead the priority is to get the paperwork done and too bad if you achieved nothing by it.
        tptacek 4mo ago
        If we're talking about actual auditors, not tech consultants who call themselves auditors but people actually trained as auditors, I'd take it as a bad sign if they asked a bunch of specific unbidden questions about software details. That's not the job.
        tialaramex 4mo ago
        The specific example I'm thinking of most strongly is when we were purchased one of the auditors looking over our software noticed that unlike most of the big company's other software at that time (about 2012) ours was HTTPS-only. Their checklist told them they need only to check there's HTTPS for authentication pages, and they asked if that's actually enough, is it fine if everything else is just plain HTTP as they've seen elsewhere and as their checklist asks? I of course said it isn't, because as we both know, it isn't. "That's not the job" is I think the most useless possible observation here. The best outcome from audit isn't that you checked all the boxes, that's just resources expended for no benefit, the best outcome is that audit found a nasty problem early so that you could fix it now. The biggest problem we have in the Web PKI with auditors is that they'd so much rather tick boxes than tell their client - who they are billing $$$ - where the problems are. This presumably feels good to the suits, but if there's a problem and the auditors don't tell you the chances are somebody else finds it and then you're in worse trouble.

1718627440 4mo ago

But that's not a wrong approach. First you want to as many vulnerabilities as you can, than you want to fix as many as you can. If you rate the developing department for that, that's another story.