5 ms·

I assumed that all software in the supply chain for defense would go through an audit that exposes things like this. Did that never happen?

LinuxBender 6y ago
They might ask if password complexity/rotation is enforced. They probably don't ask if any accounts (service accounts, etc..) are excluded from rotation. People are also taught to not volunteer anything to an auditor. ... and this does not even get into the subject of ssh keys and ssh CA's, which most auditors don't even ask about at all. I personally put less value in audits and more value in Red Teams being given full immunity to penetrate every nook and cranny. I would like to see more companies incentivize and reward in-depth penetration testing in all environments, including production. For the corporate leaders reading this, there is risk, but the reward is uncovering many future landmines your operations, code deployment teams and internet user base would have stepped on.
- RegnisGnaw 6y ago
  This is true. At my $PREVIOUS work place, we answered all questions from auditors and RFPs using the most favorable interpretation towards us of the question. For example, one of our audits for our product asked if we implemented admin idle session timeout with timeout of less than 15 minutes. There was 3 admin panels: one with 10 minute idle session timeout, one with 1 hour idle session timeout, and one with no idle session timeout. The manager said after I explained the 3 admin panels to him something like "since one does have idle session timeout of less than 15 minutes, we answer yes to the audit question".
  - kryogen1c 6y ago
    > we answered all questions from auditors and RFPs using the most favorable interpretation towards us of the question. I am currently living this life. The problem is that the system is setup as a race to the bottom with opposed incentives. If I answer with the strictest interpretation with my paranoid blue-team attitude, we appear worse than our competitors and are immediately in a worse business position - regardless of our relative or absolute security posture. This is why the Department of Defense is moving away from self-attestation in 800-171 to outside assessment in CMMC. Why not standardize on ISO and SOC2? I dont know very much about it, but I suspect those big-boy standards arent suitable for small-business America/sub-subcontractors
    - yjftsjthsd-h 6y ago
      > Why not standardize on ISO and SOC2? I dont know very much about it, but I suspect those big-boy standards arent suitable for small-business America/sub-subcontractors Also, how is that any different? I've been through SOC2 and it was just the same "here's a bunch of questions that the auditors want us to answer and provide evidence for". Maybe a little better, but still something that you could bias by providing answers that were ... technically true...
      - LinuxBender 6y ago
        To add to this, SOC1/SOC2 is only as useful as the security policies and standards and that assumes they were written around everything the company actually does. Auditors are only validating to some degree that you are doing what you say you are doing, but what you say you are doing may not be all-inclusive and respective of what you actually do.
jeffbee 6y ago
Those audits are completely meaningless.
- _jal 6y ago
  This is self-evidently false. Even casting it as something like, "security audits do not ensure security sufficiently to be worth the cost and effort" is incorrect. You can certainly get an auditor who is green, incompetent or has their own agenda. Welcome to humanity. But on average, they do more or less what they're supposed to do.
_jal 6y ago
Audits don't involve fine-tooth-comb review of everything - PwC does not go through your git commits line-by-line. They tend to verify specific critical things and verify process for everything else. To the extent your processes don't match what is reported, your audit is a work of fiction. And of course things will always fall through any available cracks, people skip process for various reasons, etc.
macinjosh 6y ago
Audits are a joke. I've been through several myself.
awb 6y ago
Some security procedures are automated, others are a manual checklist. Mistakes happen, but this was a big one.
vonmoltke 6y ago
In the supply chain, not necessarily. Any software that goes onto a classified information system is supposed to go through a security review and audit, though. Based on my first-hand experience with various DOD agencies, though, I don't think many of the people working the information security side of the house are as up-to-date on technology as they should be.

LinuxBender 6y ago

They might ask if password complexity/rotation is enforced. They probably don't ask if any accounts (service accounts, etc..) are excluded from rotation. People are also taught to not volunteer anything to an auditor. ... and this does not even get into the subject of ssh keys and ssh CA's, which most auditors don't even ask about at all. I personally put less value in audits and more value in Red Teams being given full immunity to penetrate every nook and cranny. I would like to see more companies incentivize and reward in-depth penetration testing in all environments, including production. For the corporate leaders reading this, there is risk, but the reward is uncovering many future landmines your operations, code deployment teams and internet user base would have stepped on.

RegnisGnaw 6y ago
This is true. At my $PREVIOUS work place, we answered all questions from auditors and RFPs using the most favorable interpretation towards us of the question. For example, one of our audits for our product asked if we implemented admin idle session timeout with timeout of less than 15 minutes. There was 3 admin panels: one with 10 minute idle session timeout, one with 1 hour idle session timeout, and one with no idle session timeout. The manager said after I explained the 3 admin panels to him something like "since one does have idle session timeout of less than 15 minutes, we answer yes to the audit question".
- kryogen1c 6y ago
  > we answered all questions from auditors and RFPs using the most favorable interpretation towards us of the question. I am currently living this life. The problem is that the system is setup as a race to the bottom with opposed incentives. If I answer with the strictest interpretation with my paranoid blue-team attitude, we appear worse than our competitors and are immediately in a worse business position - regardless of our relative or absolute security posture. This is why the Department of Defense is moving away from self-attestation in 800-171 to outside assessment in CMMC. Why not standardize on ISO and SOC2? I dont know very much about it, but I suspect those big-boy standards arent suitable for small-business America/sub-subcontractors
  - yjftsjthsd-h 6y ago
    > Why not standardize on ISO and SOC2? I dont know very much about it, but I suspect those big-boy standards arent suitable for small-business America/sub-subcontractors Also, how is that any different? I've been through SOC2 and it was just the same "here's a bunch of questions that the auditors want us to answer and provide evidence for". Maybe a little better, but still something that you could bias by providing answers that were ... technically true...
    - LinuxBender 6y ago
      To add to this, SOC1/SOC2 is only as useful as the security policies and standards and that assumes they were written around everything the company actually does. Auditors are only validating to some degree that you are doing what you say you are doing, but what you say you are doing may not be all-inclusive and respective of what you actually do.

jeffbee 6y ago

Those audits are completely meaningless.

_jal 6y ago
This is self-evidently false. Even casting it as something like, "security audits do not ensure security sufficiently to be worth the cost and effort" is incorrect. You can certainly get an auditor who is green, incompetent or has their own agenda. Welcome to humanity. But on average, they do more or less what they're supposed to do.

_jal 6y ago

Audits don't involve fine-tooth-comb review of everything - PwC does not go through your git commits line-by-line. They tend to verify specific critical things and verify process for everything else. To the extent your processes don't match what is reported, your audit is a work of fiction. And of course things will always fall through any available cracks, people skip process for various reasons, etc.

macinjosh 6y ago

Audits are a joke. I've been through several myself.

awb 6y ago

Some security procedures are automated, others are a manual checklist. Mistakes happen, but this was a big one.

vonmoltke 6y ago

In the supply chain, not necessarily. Any software that goes onto a classified information system is supposed to go through a security review and audit, though. Based on my first-hand experience with various DOD agencies, though, I don't think many of the people working the information security side of the house are as up-to-date on technology as they should be.