4 ms·

I dislike any compliance document that requires paid & external vendors, so would love to see that factored out SOC I vs SOC II helps get at these kinds of dis

I dislike any compliance document that requires paid & external vendors, so would love to see that factored out SOC I vs SOC II helps get at these kinds of distinctions in practice. I've seen a lot of conversations enabled by that. "We did the SOC I software checklist. At some point, we'll pay vendors $50K-250K for SOC II, feel free to fast track that now as part of our contract." I get why it's there, but this kind of thing is also why, despite being designed to address a real need, initiatives like FedRAMP have been slow & expensive disasters in practice. We should be pushing to self-serve & automated accreditation, and all the way to 1 person projects. Anything that puts third parties, people, and $$$ in the critical path needs to be split out.

wly_cdgr 5y ago
I get why it's there, but then I also get why the guy who runs my corner store gives Jimmy a $200 interest-free loan every Thursday
Kalium 5y ago
If you have a way to automatically handle all the auditing that goes into evaluating all the not-strictly-technical controls that are part of SOC and PCI-DSS and similar, a lot of people will be very interested. Based on this list, how would you automatically validate that vulnerability reports are handled in a reasonable timeframe? How would you do self-serve validation for incident handling timelines? How do you quickly and easily automate assessments of subprocessor data handling? Quick, easy, strong, self-service, automated accreditation is a wonderful goal! It's critically important to make this stuff as easy as possible because there are features to ship and customer needs to meet. Security must be a baseline for everyone, and achievable by everyone, or else it's just a way for big companies to squeeze out small ones It just might be worth considering carefully that there may be systems at hand that blend humans and computers. It may perhaps be possible that information security could be more than just an engineering problem. If I may propose a different framing? Information security is primarily a human endeavor. It is mostly about how humans and systems made of humans behave. Information security is about process. Some parts of it can be partially handled by computers, but most of it is deeply not susceptible to automation.
- tptacek 5y ago
  The SOC audit isn't really doing any meaningful technical evaluation. You're not going to get any engineering benefit from it.
  - whitej125 5y ago
    In my experience, the auditors themselves aren't really going to provide much value in terms of an evaluation. It was a little like having a Physics teacher ask each student to write their own final exam... and then take it. The teacher opined on the number of questions being asked but that was about it. All they are doing is recording your questions and your answers and certifying that you were indeed the person that took that test. That being said.. I do think you can learn a lot going through the experience of a SOC II. You force yourself to drown out the noisy world a bit and think really critically and thoroughly about security. You need to learn how to articulate security to the entire company, to clients, etc. And you need to back this up with data... not hand waves. SOC II was a pain... but a good learning experience too.
  - freeone3000 5y ago
    It verifies you have answers to the questions asked. "Has your GDPR data deletion process met its 30-day requirement?" means you (1) have this process, (2) are evaluating this process continually, (3) on correctness and timeliness. What could be more important than verifiability of correct processes?
    - lmeyerov 5y ago
      I agree that living up to standards, and specifically the engineering / operations efforts improvements to do them, is valuable. However, it's not hard to imagine automated flows for verifying this. In this case, specifying endpoints and providing automation scripts for doing GPDR flows takes care of most of it. A lot of these are converging on the same check boxes, so get rid of the people and $ aspect. A team should be able to put together COTS OSS, run on a cheapo cloud, and test as part of CI/CD . We need to reach the point properly configured RoR/Django on docker + some sidecars (ELK, autotls, ..) can do that.
whitej125 5y ago
Think you might mean SOC II Type 1 and SOC II Type 2 (vs SOC I)? SOC II Type 1/2 cover the "scopes" that more generally pertain to "secure" development. A SOC II Type 1 is basically coming up with that checklist. The third-party auditor will then measure your performance against that checklist (you provide the evidence) for a single point in time. The Type 1 is considered the baby step into the Type 2. A SOC II Type 2 is generally taking that checklist from the Type 1... but the auditor is randomly sampling for evidence over a time range (usually a year). Generally once you've done your first Type 2... you're doing it continually each and every year.
- coenhyde 5y ago
  idk why you are being downvoted. I think the op had this misunderstanding.
  - lmeyerov 5y ago
    Yes, typed too quickly. I like the continuous monitoring aspect of II, just not the vendor burden for it. Want to test continuous monitoring? Allow submission of API endpoints for test/recall that a program can check your response on. The rest can be a questionnaire.
rawoke083600 5y ago
>I dislike any compliance document that requires paid & external vendors, so would love to see that factored out Yes I agree, although my my data might just be bad/skewed, my experience as a `freelance-security-auditor` (just a side hobby). Ever time I reported a serious website vulnerability to a company. Most of the time, their initial response was "But we paid for pen-testing/sec-audits !" or something to that regard.

wly_cdgr 5y ago

I get why it's there, but then I also get why the guy who runs my corner store gives Jimmy a $200 interest-free loan every Thursday

Kalium 5y ago

If you have a way to automatically handle all the auditing that goes into evaluating all the not-strictly-technical controls that are part of SOC and PCI-DSS and similar, a lot of people will be very interested. Based on this list, how would you automatically validate that vulnerability reports are handled in a reasonable timeframe? How would you do self-serve validation for incident handling timelines? How do you quickly and easily automate assessments of subprocessor data handling? Quick, easy, strong, self-service, automated accreditation is a wonderful goal! It's critically important to make this stuff as easy as possible because there are features to ship and customer needs to meet. Security must be a baseline for everyone, and achievable by everyone, or else it's just a way for big companies to squeeze out small ones It just might be worth considering carefully that there may be systems at hand that blend humans and computers. It may perhaps be possible that information security could be more than just an engineering problem. If I may propose a different framing? Information security is primarily a human endeavor. It is mostly about how humans and systems made of humans behave. Information security is about process. Some parts of it can be partially handled by computers, but most of it is deeply not susceptible to automation.

tptacek 5y ago
The SOC audit isn't really doing any meaningful technical evaluation. You're not going to get any engineering benefit from it.
- whitej125 5y ago
  In my experience, the auditors themselves aren't really going to provide much value in terms of an evaluation. It was a little like having a Physics teacher ask each student to write their own final exam... and then take it. The teacher opined on the number of questions being asked but that was about it. All they are doing is recording your questions and your answers and certifying that you were indeed the person that took that test. That being said.. I do think you can learn a lot going through the experience of a SOC II. You force yourself to drown out the noisy world a bit and think really critically and thoroughly about security. You need to learn how to articulate security to the entire company, to clients, etc. And you need to back this up with data... not hand waves. SOC II was a pain... but a good learning experience too.
- freeone3000 5y ago
  It verifies you have answers to the questions asked. "Has your GDPR data deletion process met its 30-day requirement?" means you (1) have this process, (2) are evaluating this process continually, (3) on correctness and timeliness. What could be more important than verifiability of correct processes?
  - lmeyerov 5y ago
    I agree that living up to standards, and specifically the engineering / operations efforts improvements to do them, is valuable. However, it's not hard to imagine automated flows for verifying this. In this case, specifying endpoints and providing automation scripts for doing GPDR flows takes care of most of it. A lot of these are converging on the same check boxes, so get rid of the people and $ aspect. A team should be able to put together COTS OSS, run on a cheapo cloud, and test as part of CI/CD . We need to reach the point properly configured RoR/Django on docker + some sidecars (ELK, autotls, ..) can do that.

whitej125 5y ago

Think you might mean SOC II Type 1 and SOC II Type 2 (vs SOC I)? SOC II Type 1/2 cover the "scopes" that more generally pertain to "secure" development. A SOC II Type 1 is basically coming up with that checklist. The third-party auditor will then measure your performance against that checklist (you provide the evidence) for a single point in time. The Type 1 is considered the baby step into the Type 2. A SOC II Type 2 is generally taking that checklist from the Type 1... but the auditor is randomly sampling for evidence over a time range (usually a year). Generally once you've done your first Type 2... you're doing it continually each and every year.

coenhyde 5y ago
idk why you are being downvoted. I think the op had this misunderstanding.
- lmeyerov 5y ago
  Yes, typed too quickly. I like the continuous monitoring aspect of II, just not the vendor burden for it. Want to test continuous monitoring? Allow submission of API endpoints for test/recall that a program can check your response on. The rest can be a questionnaire.

rawoke083600 5y ago

>I dislike any compliance document that requires paid & external vendors, so would love to see that factored out Yes I agree, although my my data might just be bad/skewed, my experience as a `freelance-security-auditor` (just a side hobby). Ever time I reported a serious website vulnerability to a company. Most of the time, their initial response was "But we paid for pen-testing/sec-audits !" or something to that regard.