4 ms·

Honestly, this misses a huge point and does more harm than good. Having some of these things in place is fine of course, but it don't really help you much, nor

by teilo 7y ago

Honestly, this misses a huge point and does more harm than good. Having some of these things in place is fine of course, but it don't really help you much, nor is most of it required. SOC2 does not require: SSO, MDM, or almost anything specifically (And what's with the JAMF pro recommendation? MDM is more than iPhones). What it does require are controls. It does not specify what those controls are. You need to create them yourself, and they need to be sufficient to meet the requirements of the Trust Services Criteria. And to have controls, you need to know what you are controlling and why. This brings me to my point: To succeed in SOC2 you actually have to have a plan. You have to have written, reviewed, implemented, and audited policies. THAT is the single biggest hurdle for most companies. You also need to know how to do a risk assessment on your organization. This is not terribly hard, but it is essential to SOC2. And finally, you need buy-in from the governing bodies of your organization. None of these things I mentioned are products. You can't buy them. They take work. A hell of a lot of work. I will guarantee that most companies doing this for the first time do not have half of the processes in place that SOC2 requires. I doubt that 20% have internal audit procedures, for example, or sufficient end-user training. And one more thing that will take nearly anyone who had done security by surprise if they don't know it's there. The COSO Principles. These were added to SOC2 in the wake of the Lehman Bros/Enron scandal. They extend SOC2 to the entire governance structure of your organization, and include controls on financial reporting, hiring practices, employee review processes, etc. This is a major PITA, because it lies far outside the bailiwick of most CISOs or equivalent, and, again, it requires extensive cooperation from the rest of senior management. As you can probably tell, I've done this before.

tptacek 7y ago
We've watched our clients get SOC2 certified and been directly involved in the IRL and audit interview processes, and circulated this among peers that have also been SOC2 certified, and we're pretty comfortable with the advice we're offering. So: As we've said repeatedly, SOC2 doesn't "require" much of anything. It certainly doesn't call SSO or MDM out by name. What it instead involves are abstract control points that are addressed with evidence. As the top of the post says, the point is to engage in security engineering practices that generate evidence useful in SOC2 engagements. So, you can certainly freestyle your application and infra access controls, and make up policy responses to the IRL questions on the fly. You can also sit around and worry about which of the "COSO Principles" you're covered on or not covered on. Or, you can do some basic sensible security engineering things that are almost certain to cover the majority of the evidence requests your auditors generate, in most cases without even needing to be aware of the AICPA guidance line items. You make your own process recommendation here: do a "hell of a lot of work" to have "written and audited policies" and a "risk assessment" and become aware of the "COSO Principles". I have no trouble believing that approach works, because after seeing and being involved in a bunch of SOC2 audits and talking to people who did SOC2 audits, I believe everything works. More importantly, we see startups led around by the nose by insane questionnaires (and a SOC2 IRL is certainly that). We've talked to firms that have been told by their SOC2 auditors to install IDS systems, WAFs, and Mac AV software. We are alarmed by the prospect of a startup without in-house security or compliance expertise reading what's out there about SOC2 --- especially the AICPA documentation, or example IRLs --- and then just going and doing that stuff. They'd be wasting a tremendous amount of time without realizing that smart startups skip a lot of that work and deploy none of the junk and get SOC2'd just fine. So, if you're going to do something, do something sensible. What I think people should be especially careful about is taking advice from people who have just done SOC2 at a single company or with a single auditor. They are all different. There are processes where you'll end up explaining what a URL is. There are processes where you'll end up explaining why you have VPC flow logs disabled. The value we're offering here is that we've seen it done repeatedly, with multiple auditors.
- teilo 7y ago
  Look, I agree with pretty much all of what you say. I would never suggest any company, whether a startup or an established enterprise (which is my case) go through the SOC2 process unguided, attempting to interpret the ACIPA docs on their own. That is a recipe for disaster. The TSC is almost inscrutable on its own. The AICPA cannot write in plain english to save their life. That said, I don't understand this statement: "You can also sit around and worry about which of the "COSO Principles" you're covered on or not covered on." In the end, to pass a SOC2 audit, you have to have evidence for each of the applicable TSC points, including the COSO Principles, which have little or nothing to do with information security but are all about general business governance. It's not a question of "sitting around wondering." It's understanding what the COSO Principles actually are after, so that you can assemble the appropriate evidence. And that means working with HR and finance. You make it sound like the COSO Principles are irrelevant. Believe me, if I could skip them I would. If I could skip a risk assessment, I would. If I could skip all the "written and audited policies" I would. But you can't skip them. And in the case of a risk assessment, you shouldn't skip them. I mean, that's the whole point in the end: knowing your risks, and having the controls in place to mitigate them. So yes, of course you need to be sensible. But you also have to do the work, and the work is hard. It is by far the last favorite part of my job. But my point is: It's not about the product category boxes you have ticked. Security products are important. They make compliance much easer. No question. But for SOC2 to have real value to an organization (and I think there is real value that can be gained from it), it is far more about process, and it is process that is the most difficult thing to implement and religiously maintain and document.
  - tptacek 7y ago
    This has simply not been my experience of how SOC2 processes have actually worked at real startups. The experience I've seen, repeatedly, in variations but always on this one theme, is of a collection of mostly boilerplate policies being exchanged for a turbocharged vendorsec questionnaire, followed by a series of interviews. The line items on the questionnaire are answered in any ways that is convenient for the auditee and credible for the auditor. At no point is anyone on the engineering side made familiar with TSC points or COSO; the runic accountancy is driven be the auditor. I would, in particular, not recommend that startups thinking about pursuing SOC2 spend a lot of time familiarizing themselves with the TSC or (god forbid) the COSO Framework, because it simply isn't going to matter. My general belief is that SOC2 is of no intrinsic value to organizations, and going into it with a mind towards letting your engineering practice be guided by SOC2 rather than dictating SOC2 is a recipe for heartbreak. Among security engineers (and managers) at tech startups I've talked to, I don't seem to be far out of the mainstream. SOC2 is like most university degrees: it's a demonstration of seriousness, and nothing more. If you think this blog post is about "buying the right products", you've entirely missed its point.
lvh 7y ago
> And what's with the JAMF pro recommendation? MDM is more than iPhones JAMF Pro is not just for iPhones.
- teilo 7y ago
  I know what Jamf Pro is. We manage all our Mac an iOS devices with it. But it's not for Android, or Chrome, or Windows. This article makes the assumption that all startups use Mac. That's a rather dubious assumption.
  - tptacek 7y ago
    No it isn't. Have you met a startup?
  - lvh 7y ago
    Saying it was just for phones is your quote, so I think your reaction is a little misplaced. If you don’t want people to tell you what Jamf Pro does then maybe don’t make being wrong about your what Jamf Pro does part of your argument. (Plus, if you think this is about buying any particular product, you have woefully missed the point.)
    - teilo 7y ago
      Maybe don't read one parenthetical statement, assume it has any bearing on my argument, ignore the rest of what I wrote, and treat me like I don't know anything. That's just rude. I bought the damn thing specifically to manage a fleet of ~350 Macs. I know what it is.

by teilo 7y ago

tptacek 7y ago

We've watched our clients get SOC2 certified and been directly involved in the IRL and audit interview processes, and circulated this among peers that have also been SOC2 certified, and we're pretty comfortable with the advice we're offering. So: As we've said repeatedly, SOC2 doesn't "require" much of anything. It certainly doesn't call SSO or MDM out by name. What it instead involves are abstract control points that are addressed with evidence. As the top of the post says, the point is to engage in security engineering practices that generate evidence useful in SOC2 engagements. So, you can certainly freestyle your application and infra access controls, and make up policy responses to the IRL questions on the fly. You can also sit around and worry about which of the "COSO Principles" you're covered on or not covered on. Or, you can do some basic sensible security engineering things that are almost certain to cover the majority of the evidence requests your auditors generate, in most cases without even needing to be aware of the AICPA guidance line items. You make your own process recommendation here: do a "hell of a lot of work" to have "written and audited policies" and a "risk assessment" and become aware of the "COSO Principles". I have no trouble believing that approach works, because after seeing and being involved in a bunch of SOC2 audits and talking to people who did SOC2 audits, I believe everything works. More importantly, we see startups led around by the nose by insane questionnaires (and a SOC2 IRL is certainly that). We've talked to firms that have been told by their SOC2 auditors to install IDS systems, WAFs, and Mac AV software. We are alarmed by the prospect of a startup without in-house security or compliance expertise reading what's out there about SOC2 --- especially the AICPA documentation, or example IRLs --- and then just going and doing that stuff. They'd be wasting a tremendous amount of time without realizing that smart startups skip a lot of that work and deploy none of the junk and get SOC2'd just fine. So, if you're going to do something, do something sensible. What I think people should be especially careful about is taking advice from people who have just done SOC2 at a single company or with a single auditor. They are all different. There are processes where you'll end up explaining what a URL is. There are processes where you'll end up explaining why you have VPC flow logs disabled. The value we're offering here is that we've seen it done repeatedly, with multiple auditors.

teilo 7y ago
Look, I agree with pretty much all of what you say. I would never suggest any company, whether a startup or an established enterprise (which is my case) go through the SOC2 process unguided, attempting to interpret the ACIPA docs on their own. That is a recipe for disaster. The TSC is almost inscrutable on its own. The AICPA cannot write in plain english to save their life. That said, I don't understand this statement: "You can also sit around and worry about which of the "COSO Principles" you're covered on or not covered on." In the end, to pass a SOC2 audit, you have to have evidence for each of the applicable TSC points, including the COSO Principles, which have little or nothing to do with information security but are all about general business governance. It's not a question of "sitting around wondering." It's understanding what the COSO Principles actually are after, so that you can assemble the appropriate evidence. And that means working with HR and finance. You make it sound like the COSO Principles are irrelevant. Believe me, if I could skip them I would. If I could skip a risk assessment, I would. If I could skip all the "written and audited policies" I would. But you can't skip them. And in the case of a risk assessment, you shouldn't skip them. I mean, that's the whole point in the end: knowing your risks, and having the controls in place to mitigate them. So yes, of course you need to be sensible. But you also have to do the work, and the work is hard. It is by far the last favorite part of my job. But my point is: It's not about the product category boxes you have ticked. Security products are important. They make compliance much easer. No question. But for SOC2 to have real value to an organization (and I think there is real value that can be gained from it), it is far more about process, and it is process that is the most difficult thing to implement and religiously maintain and document.
- tptacek 7y ago
  This has simply not been my experience of how SOC2 processes have actually worked at real startups. The experience I've seen, repeatedly, in variations but always on this one theme, is of a collection of mostly boilerplate policies being exchanged for a turbocharged vendorsec questionnaire, followed by a series of interviews. The line items on the questionnaire are answered in any ways that is convenient for the auditee and credible for the auditor. At no point is anyone on the engineering side made familiar with TSC points or COSO; the runic accountancy is driven be the auditor. I would, in particular, not recommend that startups thinking about pursuing SOC2 spend a lot of time familiarizing themselves with the TSC or (god forbid) the COSO Framework, because it simply isn't going to matter. My general belief is that SOC2 is of no intrinsic value to organizations, and going into it with a mind towards letting your engineering practice be guided by SOC2 rather than dictating SOC2 is a recipe for heartbreak. Among security engineers (and managers) at tech startups I've talked to, I don't seem to be far out of the mainstream. SOC2 is like most university degrees: it's a demonstration of seriousness, and nothing more. If you think this blog post is about "buying the right products", you've entirely missed its point.

lvh 7y ago

> And what's with the JAMF pro recommendation? MDM is more than iPhones JAMF Pro is not just for iPhones.

teilo 7y ago
I know what Jamf Pro is. We manage all our Mac an iOS devices with it. But it's not for Android, or Chrome, or Windows. This article makes the assumption that all startups use Mac. That's a rather dubious assumption.
- tptacek 7y ago
  No it isn't. Have you met a startup?
- lvh 7y ago
  Saying it was just for phones is your quote, so I think your reaction is a little misplaced. If you don’t want people to tell you what Jamf Pro does then maybe don’t make being wrong about your what Jamf Pro does part of your argument. (Plus, if you think this is about buying any particular product, you have woefully missed the point.)
  - teilo 7y ago
    Maybe don't read one parenthetical statement, assume it has any bearing on my argument, ignore the rest of what I wrote, and treat me like I don't know anything. That's just rude. I bought the damn thing specifically to manage a fleet of ~350 Macs. I know what it is.