4 ms·

Counterpoint, in my career, I've seen a lot of Security Audits that seem on paper to be very thorough and detailed but when you dig deeper, you can see that it

Counterpoint, in my career, I've seen a lot of Security Audits that seem on paper to be very thorough and detailed but when you dig deeper, you can see that it only touched the surface and the auditor missed a lot of things.

delusional 4y ago
I have in fact only seen these kinds.
- vidanay 4y ago
  Same with certifications. (e.g. ISO 900x certification - you can still build a shit product, but it will be a very well documented shit product.)
tremon 4y ago
I'd guess there's a difference between an external audit performed for regulatory of financial reasons, and an audit by the internal legal team because of a pending legal case or regulatory scrutiny.
- bravetraveler 4y ago
  There's a very wide gap in what needs to be covered as a result of this consideration! Disaster recovery/business continuity? Absolutely, you need it documented so that someone 'off the street' could be helpful. Implementing yet another service gateway? Probably going to take a lot of conversations with people familiar with the streets involved. Auditors won't care until it's realized. They'll want things like interconnection diagrams -- painting a picture so they can follow your adherence. We have very high compliance requirements where I work -- about half of my time is spent on it. They're less worried about how the bread is made, more that you can keep making bread the way you say you do.
kixiQu 4y ago
I mean, sure, security is complicated, lots of judgment calls, but within this mindset, if a team/org/company said "we have no idea where we're using log4j" it was then their job to have someone working around the clock until they figured that out and fixed it. Figuring out how to meet new requirements that weren't around when systems were begun is, IMO, fully just part of the job. For FB engineers to be throwing up their hands in front of a legal system like oh nooooo how could we possibly track things seems like a stalling technique and makes all of tech look bad.
- ACow_Adonis 4y ago
  as per my other comment, in most of the sufficiently big or historical cases you have to distinguish between two things: - reality - the socially acceptable fiction in the report the exercise would be to create an output that allows people to pretend we know where all instances of "log4j" were used, or that sufficient depth and resources have been expended on such. but in a sufficiently large and complex organisation, we (and by we I mean technically able, knowledgeable or sufficiently aware individuals) know that the content of such is likely to be as factually true as reading a historian's report entitled: "enumeration of all points in past century where cutlery has been used".
  - kixiQu 4y ago
    On a spectrum of ridiculousness vs. necessity running from "enumerate where cutlery was used for the past one hundred years" to "keep tabs on the radioactive material we have in storage", it is a choice to vacuum up user data and treat it as the former rather than the latter. It is not unavoidable; it is that Facebook does not want to avoid it. If they did want to avoid it, they could do so in the same way that my employer woke me up to verify that there was no log4j running on some machines that did not have Java installed: unpleasant campaigns, building automated checks, checking the checks, pouring manual effort into the gaps, paying the costs. Slowing down development, maybe. Shifting resources. Rendering some designs infeasible (okay, well, not for log4j, but for a compliance thing I'm thinking of). It is very important that we are precise here: they are trying to come up with a story to make this about technicalities, and it is actually about their priorities. People who've done large-scale engineering need to be clear that even within a large organization, things do get hunted down if and only if the organization prioritizes them. Measurements and control mechanisms are built. Systems, social and technical, are designed. And if they're not, Facebook can't honestly say oh but it was so hard it's impossible, we're just too big. We're not talking about detecting I-know-it-when-I-see-it obscenity or whatever social thing people hope AI will fix for them, we're talking about knowing how data is being plumbed from one place to another, and where it lives when it's at rest. It's an embarrassment that this is being presented as a Hard Problem.
krinchan 4y ago
Someone has been the subject of a SOC audit.

delusional 4y ago

I have in fact only seen these kinds.

vidanay 4y ago
Same with certifications. (e.g. ISO 900x certification - you can still build a shit product, but it will be a very well documented shit product.)

tremon 4y ago

I'd guess there's a difference between an external audit performed for regulatory of financial reasons, and an audit by the internal legal team because of a pending legal case or regulatory scrutiny.

bravetraveler 4y ago
There's a very wide gap in what needs to be covered as a result of this consideration! Disaster recovery/business continuity? Absolutely, you need it documented so that someone 'off the street' could be helpful. Implementing yet another service gateway? Probably going to take a lot of conversations with people familiar with the streets involved. Auditors won't care until it's realized. They'll want things like interconnection diagrams -- painting a picture so they can follow your adherence. We have very high compliance requirements where I work -- about half of my time is spent on it. They're less worried about how the bread is made, more that you can keep making bread the way you say you do.

kixiQu 4y ago

I mean, sure, security is complicated, lots of judgment calls, but within this mindset, if a team/org/company said "we have no idea where we're using log4j" it was then their job to have someone working around the clock until they figured that out and fixed it. Figuring out how to meet new requirements that weren't around when systems were begun is, IMO, fully just part of the job. For FB engineers to be throwing up their hands in front of a legal system like oh nooooo how could we possibly track things seems like a stalling technique and makes all of tech look bad.

ACow_Adonis 4y ago
as per my other comment, in most of the sufficiently big or historical cases you have to distinguish between two things: - reality - the socially acceptable fiction in the report the exercise would be to create an output that allows people to pretend we know where all instances of "log4j" were used, or that sufficient depth and resources have been expended on such. but in a sufficiently large and complex organisation, we (and by we I mean technically able, knowledgeable or sufficiently aware individuals) know that the content of such is likely to be as factually true as reading a historian's report entitled: "enumeration of all points in past century where cutlery has been used".
- kixiQu 4y ago
  On a spectrum of ridiculousness vs. necessity running from "enumerate where cutlery was used for the past one hundred years" to "keep tabs on the radioactive material we have in storage", it is a choice to vacuum up user data and treat it as the former rather than the latter. It is not unavoidable; it is that Facebook does not want to avoid it. If they did want to avoid it, they could do so in the same way that my employer woke me up to verify that there was no log4j running on some machines that did not have Java installed: unpleasant campaigns, building automated checks, checking the checks, pouring manual effort into the gaps, paying the costs. Slowing down development, maybe. Shifting resources. Rendering some designs infeasible (okay, well, not for log4j, but for a compliance thing I'm thinking of). It is very important that we are precise here: they are trying to come up with a story to make this about technicalities, and it is actually about their priorities. People who've done large-scale engineering need to be clear that even within a large organization, things do get hunted down if and only if the organization prioritizes them. Measurements and control mechanisms are built. Systems, social and technical, are designed. And if they're not, Facebook can't honestly say oh but it was so hard it's impossible, we're just too big. We're not talking about detecting I-know-it-when-I-see-it obscenity or whatever social thing people hope AI will fix for them, we're talking about knowing how data is being plumbed from one place to another, and where it lives when it's at rest. It's an embarrassment that this is being presented as a Hard Problem.

krinchan 4y ago

Someone has been the subject of a SOC audit.