4 ms·

I wish we could just all reject SOC2, it's such a grift. Bankers come in to read some docs (that they don't understand), look at screenshots (that they don't un

I wish we could just all reject SOC2, it's such a grift. Bankers come in to read some docs (that they don't understand), look at screenshots (that they don't understand), take up 100s of thousands in cash and time, and then write a document (that they don't understand) that no one will read or care about (except to fuel their own SOC2). The harm is so significant. Tons of these due diligence terms are driven not by any security engineer, but by a legal or compliance team - I've even had members of the security team outright apologize for having to send it over.

HFguy 6y ago
What do bankers have to do with SOC2? Do you mean consultants?
- privacy-matters 6y ago
  I think he means a CPA. Certified public accountant. They are the ones who actually write the report.
jiveturkey 6y ago
perhaps you don't understand the way a typical company operates. security is all but ignored. SOC2 is an absolute necessity. it's a forcing function for baseline processes and controls, that 100% would not exist by default if companies were left to their own devices. for sure, i'm not talking about the 5% of competent companies.
- staticassertion 6y ago
  Let's oversimplify things and say there are two types of companies: 1. Those who care about security 2. Those who do not care about security For the (1), SOC2 provides no value, because any structure it lends (it lends none, but you'll end up choosing some NIST thing or whatever) is something you could have implemented for much less money. Remember, you'll spend ~1 full security engineer worth of money/ time, so you could hire a FTE to just do these things. Except you won't be constrained in nearly the same ways. 2. Companies that don't care will just grift the grifters. It's simple - there are lots of easy checkboxes, and most of it is just documenting processes. Anyone who's gone through a SOC2 should see how easy it is to "game" it. It's tedious, but a large company will just hire their way out of it, and have a compliance team that's almost certainly isolated from security. Because it's gameable SOC2 is far easier for large companies to push off. They can hire a compliance team, call it 'security', and move on. Small companies, and/ or companies that care about security, are left having to dedicate their much more limited resources to compliance over implementing meaningful controls. A small company isn't going to know the many 'tricks' for doing minimal work to pass, which is a really important quality of SOC2 - you want the least policy to pass, otherwise you're setting yourself up for either stagnation or an even longer report next year, since changes between reports have to be documented and go through the process. Large companies can just get away with way more. As one simple example, let's say you have 1 FTE seceng. For compliance, they could spend N% of their time setting up logging, documenting that logging, writing docs on their IR policy, etc. Or, without SOC2, they could spend N% of their time setting up logging, writing good detections, understanding and exploring their infrastructure, documenting in a much more natural way at a lower cost, etc. And then that budget could go towards improving infra, tooling, training, new hires, etc, to do that work even more effectively. How many breaches has SOC2 stopped? Because clearly it hasn't been the deterrent in many cases - how many companies get owned, while being compliant, due to unpatched vulns (something any auditor is guaranteed to ask about)? What if they'd spent the few hundred thousand a year on a few more seceng? The way companies scale security puts ~10-500 employees to every seceng, meaning that even cutting a few would be a massive increase in risk. In short, the companies that are already ignoring security will have no problem doing so when they're large, and smaller companies, or companies that do care, will only be drained by SOC2. edit: I will also say that, * I won't state that SOC2 is universally useless. * This is a very hard problem. It's a regulation on a quality that is a very fast moving target with weak consensus.
bob33212 6y ago
I found the SOC 2 very useful. Not because I make any Ops or Engineering changes based on the process. It made me think about risks to the business as a whole and the concept of controls to mitigate and monitor those risks. For example, what is everyone doing to minimize 3rd party risks? How do you know that the whole team understands why PII data should be avoided when possible?
- staticassertion 6y ago
  SOC2 is exclusively a forcing function, you could have adopted a NIST framework for risk assessment and controls and gotten that same value, and then had an extra 6 figures of budget to implement high value security work.
  - Spivak 6y ago
    Same with the CIS controls which at least come with a tool to test servers against their host benchmarks -- CIS Level 2 is a fairly practical "Server Hardening for Beginners."
Kalium 6y ago
Being a security person myself, I've found plenty of value in reading SOC2 reports. They're one of the few relatively standardized ways to address the laundry list of vendor due diligence questions. Is it a good experience for small companies? No. Does it make it easy for your vendors to use cool new technologies? No. Do I, someone advising on whether or not we should buy something, care about either of those in the moment? Also no. And I spend a rather distressingly large amount of my time trying to talk engineers out of using cool new technology for novelty's sake anyway. You're absolutely right. SOC2 can, and I assume often does, go quite badly awry and waste literally everyone's time and money. I just know that I've found some value in it. And it helps provide a sound basis for making the vendor agree to assume liability for when they screw up due to grifting.
- tensor 6y ago
  I think the fact that you don't care about challenges for small companies and new technology is a huge problem. Stopping new entrants into a space is pretty much anti-competition. Preventing new technologies is anti-innovation. Competition and innovation are the most important things to a healthy economy and market. So essentially, SOC2 both stifles innovation and ruins economies.
  - Kalium 6y ago
    You're absolutely right. New entrants, new technologies, and new people in a space are critical for the health of a market. I just think it's possible that when procuring a tool for a given purpose, a company's chief concern might be about the safety of the tool and vendor rather than the health of the overall market. Your experience may well differ! Also, I feel the need to clarify my remarks. I, someone advising on whether or not my employer should buy something, do not prioritize the health of the market or cool new technology or a good experience for the vendor over the safety of the tool and the vendor when advising on the purchasing decision. In other circumstances, I can and sometimes do make decisions different. I hope this is removes any misunderstanding that I may have engendered by failing to write clearly.
    - 6y ago
acdha 6y ago
This is like saying Agile or DevOps is a mistake because the usual grifters sold a PowerPoint to the C-levels but didn’t change the culture. If you approach anything as a formality, you can definitely minimize the benefit you’ll get but that’s saying more about your organization.
- Spivak 6y ago
  That's the parent's point! The certification itself is completely meaningless. If a company has their SOC2 it basically tells you almost nothing about their security culture. I've never been at a company that has taken SOC audits seriously -- plenty of companies that take security seriously but SOC compliance is an exercise in paperwork not security.
  - acdha 6y ago
    If that their point, it wasn't well expressed. It doesn't seem right to call something “a grift” because you chose to treat it like a formality when you could have used it as a good faith engagement to actually try to make it better. That's why I made the Agile comparison — organizations can have positive or pointless adoption without that saying more than “you get out of it what you put into it”.
    - staticassertion 6y ago
      I call it a grift because it really feels like it. It's this weird "you have to do this or else" thing that provides 0 additional value (SOC2 is not prescriptive, it is just a forcing function, you could have... just done it). A bunch of unqualified people coming in to rank the unrankable in a vague, gameable, expensive way is my definition of a grift. Then, on top of that, you have a sort of ring of high status organizations that will audit you and charge a fortune, but they've convinced all of these other orgs that their audits are Definitely Better (companies audited by them get breached all the time too), and now you've got this whole racket on top of the grift. Agile isn't forced on anyone. People won't skip over your product because you're "not really doing agile" or whatever. You don't get forced into yearly audits where you pay some outsider with an "agile certification" and 0 technical knowledge to take up multiple engineers for months to write reports. This post explains more: https://news.ycombinator.com/item?id=24505894 https://news.ycombinator.com/item?id=24505894
Khaine 6y ago
I don't think you understand SOC2 at all. The business itself defines what controls they want to be tested and included in the SOC2 report. This is why SOC2 reports are simultaneously the worst and best (in most cases, only way to get assurance) assurance you can get over a third party. The problem is, as you say people who consume SOC2 reports don't understand what they are getting.
- staticassertion 6y ago
  Well I think I do understand SOC2 :) I am well aware of how controls are defined, it was only just an hour ago that I was reviewing our controls with the feedback from our auditor.
  - Khaine 6y ago
    So how would you look to get assurance over third parties without SOC reports? I am happy to concede that SOC reports are terrible, but at the moment they are one of the few effective ways to get assurance over third parties.
    - staticassertion 6y ago
      As I said in another post, > This is a very hard problem. It's a regulation on a quality that is a very fast moving target with weak consensus. Doing this in a regulatory way just isn't viable today. > the few effective ways I disagree that they're effective. I don't think they are. Lots of companies with a SOC2 are breached, no company that people consider 'very secure' has earned that reputation due to SOC2 (or any other compliance for that matter).

HFguy 6y ago

What do bankers have to do with SOC2? Do you mean consultants?

privacy-matters 6y ago
I think he means a CPA. Certified public accountant. They are the ones who actually write the report.

jiveturkey 6y ago

perhaps you don't understand the way a typical company operates. security is all but ignored. SOC2 is an absolute necessity. it's a forcing function for baseline processes and controls, that 100% would not exist by default if companies were left to their own devices. for sure, i'm not talking about the 5% of competent companies.

staticassertion 6y ago
Let's oversimplify things and say there are two types of companies: 1. Those who care about security 2. Those who do not care about security For the (1), SOC2 provides no value, because any structure it lends (it lends none, but you'll end up choosing some NIST thing or whatever) is something you could have implemented for much less money. Remember, you'll spend ~1 full security engineer worth of money/ time, so you could hire a FTE to just do these things. Except you won't be constrained in nearly the same ways. 2. Companies that don't care will just grift the grifters. It's simple - there are lots of easy checkboxes, and most of it is just documenting processes. Anyone who's gone through a SOC2 should see how easy it is to "game" it. It's tedious, but a large company will just hire their way out of it, and have a compliance team that's almost certainly isolated from security. Because it's gameable SOC2 is far easier for large companies to push off. They can hire a compliance team, call it 'security', and move on. Small companies, and/ or companies that care about security, are left having to dedicate their much more limited resources to compliance over implementing meaningful controls. A small company isn't going to know the many 'tricks' for doing minimal work to pass, which is a really important quality of SOC2 - you want the least policy to pass, otherwise you're setting yourself up for either stagnation or an even longer report next year, since changes between reports have to be documented and go through the process. Large companies can just get away with way more. As one simple example, let's say you have 1 FTE seceng. For compliance, they could spend N% of their time setting up logging, documenting that logging, writing docs on their IR policy, etc. Or, without SOC2, they could spend N% of their time setting up logging, writing good detections, understanding and exploring their infrastructure, documenting in a much more natural way at a lower cost, etc. And then that budget could go towards improving infra, tooling, training, new hires, etc, to do that work even more effectively. How many breaches has SOC2 stopped? Because clearly it hasn't been the deterrent in many cases - how many companies get owned, while being compliant, due to unpatched vulns (something any auditor is guaranteed to ask about)? What if they'd spent the few hundred thousand a year on a few more seceng? The way companies scale security puts ~10-500 employees to every seceng, meaning that even cutting a few would be a massive increase in risk. In short, the companies that are already ignoring security will have no problem doing so when they're large, and smaller companies, or companies that do care, will only be drained by SOC2. edit: I will also say that, * I won't state that SOC2 is universally useless. * This is a very hard problem. It's a regulation on a quality that is a very fast moving target with weak consensus.

bob33212 6y ago

I found the SOC 2 very useful. Not because I make any Ops or Engineering changes based on the process. It made me think about risks to the business as a whole and the concept of controls to mitigate and monitor those risks. For example, what is everyone doing to minimize 3rd party risks? How do you know that the whole team understands why PII data should be avoided when possible?

staticassertion 6y ago
SOC2 is exclusively a forcing function, you could have adopted a NIST framework for risk assessment and controls and gotten that same value, and then had an extra 6 figures of budget to implement high value security work.
- Spivak 6y ago
  Same with the CIS controls which at least come with a tool to test servers against their host benchmarks -- CIS Level 2 is a fairly practical "Server Hardening for Beginners."

Kalium 6y ago

Being a security person myself, I've found plenty of value in reading SOC2 reports. They're one of the few relatively standardized ways to address the laundry list of vendor due diligence questions. Is it a good experience for small companies? No. Does it make it easy for your vendors to use cool new technologies? No. Do I, someone advising on whether or not we should buy something, care about either of those in the moment? Also no. And I spend a rather distressingly large amount of my time trying to talk engineers out of using cool new technology for novelty's sake anyway. You're absolutely right. SOC2 can, and I assume often does, go quite badly awry and waste literally everyone's time and money. I just know that I've found some value in it. And it helps provide a sound basis for making the vendor agree to assume liability for when they screw up due to grifting.

tensor 6y ago
I think the fact that you don't care about challenges for small companies and new technology is a huge problem. Stopping new entrants into a space is pretty much anti-competition. Preventing new technologies is anti-innovation. Competition and innovation are the most important things to a healthy economy and market. So essentially, SOC2 both stifles innovation and ruins economies.
- Kalium 6y ago
  You're absolutely right. New entrants, new technologies, and new people in a space are critical for the health of a market. I just think it's possible that when procuring a tool for a given purpose, a company's chief concern might be about the safety of the tool and vendor rather than the health of the overall market. Your experience may well differ! Also, I feel the need to clarify my remarks. I, someone advising on whether or not my employer should buy something, do not prioritize the health of the market or cool new technology or a good experience for the vendor over the safety of the tool and the vendor when advising on the purchasing decision. In other circumstances, I can and sometimes do make decisions different. I hope this is removes any misunderstanding that I may have engendered by failing to write clearly.
  - 6y ago

acdha 6y ago

This is like saying Agile or DevOps is a mistake because the usual grifters sold a PowerPoint to the C-levels but didn’t change the culture. If you approach anything as a formality, you can definitely minimize the benefit you’ll get but that’s saying more about your organization.

Spivak 6y ago
That's the parent's point! The certification itself is completely meaningless. If a company has their SOC2 it basically tells you almost nothing about their security culture. I've never been at a company that has taken SOC audits seriously -- plenty of companies that take security seriously but SOC compliance is an exercise in paperwork not security.
- acdha 6y ago
  If that their point, it wasn't well expressed. It doesn't seem right to call something “a grift” because you chose to treat it like a formality when you could have used it as a good faith engagement to actually try to make it better. That's why I made the Agile comparison — organizations can have positive or pointless adoption without that saying more than “you get out of it what you put into it”.
  - staticassertion 6y ago
    I call it a grift because it really feels like it. It's this weird "you have to do this or else" thing that provides 0 additional value (SOC2 is not prescriptive, it is just a forcing function, you could have... just done it). A bunch of unqualified people coming in to rank the unrankable in a vague, gameable, expensive way is my definition of a grift. Then, on top of that, you have a sort of ring of high status organizations that will audit you and charge a fortune, but they've convinced all of these other orgs that their audits are Definitely Better (companies audited by them get breached all the time too), and now you've got this whole racket on top of the grift. Agile isn't forced on anyone. People won't skip over your product because you're "not really doing agile" or whatever. You don't get forced into yearly audits where you pay some outsider with an "agile certification" and 0 technical knowledge to take up multiple engineers for months to write reports. This post explains more: https://news.ycombinator.com/item?id=24505894 https://news.ycombinator.com/item?id=24505894

Khaine 6y ago

I don't think you understand SOC2 at all. The business itself defines what controls they want to be tested and included in the SOC2 report. This is why SOC2 reports are simultaneously the worst and best (in most cases, only way to get assurance) assurance you can get over a third party. The problem is, as you say people who consume SOC2 reports don't understand what they are getting.

staticassertion 6y ago
Well I think I do understand SOC2 :) I am well aware of how controls are defined, it was only just an hour ago that I was reviewing our controls with the feedback from our auditor.
- Khaine 6y ago
  So how would you look to get assurance over third parties without SOC reports? I am happy to concede that SOC reports are terrible, but at the moment they are one of the few effective ways to get assurance over third parties.
  - staticassertion 6y ago
    As I said in another post, > This is a very hard problem. It's a regulation on a quality that is a very fast moving target with weak consensus. Doing this in a regulatory way just isn't viable today. > the few effective ways I disagree that they're effective. I don't think they are. Lots of companies with a SOC2 are breached, no company that people consider 'very secure' has earned that reputation due to SOC2 (or any other compliance for that matter).