3 ms·

How often I had to deal with some auditor from "a big 5", sold as senior engineer and he was simply going through checklists. Unix security analyst, reading a c

How often I had to deal with some auditor from "a big 5", sold as senior engineer and he was simply going through checklists. Unix security analyst, reading a checklist, while trying to type "unix" commands on HP-UX, AIX.. was funny if we weren't paying for that. In another hand "no director get fired by going through auditing process with some big company, but some do get fired, exactly because they didn't want to"... I remember some "senior network analyst" having to go through a checklist while analyzing a network hub. Pathetic, but reality.

eggsmediumrare 4y ago
Pilots with 40 years of experience use checklists every time they fly.
- netjiro 4y ago
  As long as the checklist catches more costly errors than the time to use it and the (very likely) negative secondary effects to critical thinking and innovation. Love tools that help, eg checklists, when used right. But over time they have a very strong tendency to become set-in-stone dogma, at which time they will create a priesthood and a very large dead zone where thinking and flexibility is no longer allowed.
- pelasaco 4y ago
  > Pilots with 40 years of experience use checklists every time they fly. I love when people just add comment for pure sake of commenting. The auditing companies don't get a "cyber security expert" with 40 years of experience, but get undergrads, suit them up and sell them as experts. Because of the checklists, that without knowing anything about the context, are almost useless and generate mostly noise.
  - intelVISA 4y ago
    Can confirm boss, lscpu is GREEN on the no "iTLB multihit" vulns. We're clear. No I don't know what a VMM is but it's probably not relevant.
  - eggsmediumrare 4y ago
    The comment I was replying to seemed to be suggesting that both inexperience AND checklists are the problem. Checklists are never a bad thing.
  - NoPicklez 4y ago
    This is such a bullshit statement. Auditing companies absolutely get cyber security experts. Graduates coming out of University obviously don't have the experience of that of veterans but are in a team of experts with experience in that area. The checklists and workpapers are a way to ensure that there is a level of standardisation in the work that is being performed. If you have assessed a clients firewall or operating system as a veteran you are encouraged to replicate how you did that in a work paper or checklist to cover bases. Big 4 auditing firms are some of the best in class when it comes to knowledge sharing. Yes sometimes a junior or a senior might miss something without the context, but that is why humans developed lips to talk through it. I can't count the amount of times I have challenged what a junior has said, or the conclusion a junior has come to based on what a client has said based on my experience. I was once a graduate myself and found many glaring gaps through the use of simple checklists. If that's all it takes, then the industry needs to write some of their own.
matwood 4y ago
Checklists are standard for many professions and are known to work. IT should probably be using more of them. https://en.wikipedia.org/wiki/Checklist https://en.wikipedia.org/wiki/Checklist
- pelasaco 4y ago
  the checklist without intelligence is almost useless or generates too much noise. That's however those companies are able to profit. Generate a checklist, hire a undergrad to fill the checklists, organize it with a nice report, full of data, which mostly are noise, and ask a bunch of money, which is exactly the topic. Consultancies know less than they claim, because they are sending junior people with checklists to audit your company.
- marcosdumay 4y ago
  > IT should probably be using more of them. Probably not. IT has its own procedures for enforcing process compliance [1] that are much less susceptible to deviation and much more efficient. It's ok if you fail to capture something and need to use less effective methods like checklists. But going to them by default or, like the GP's said, only using them are bad. (But anyway, the GP was complaining because he was expecting highly valuable customized advice, and instead he got low-value process adherence checks. How you do your adherence checks is a completely different subject.) 1 - https://en.wikipedia.org/wiki/Software https://en.wikipedia.org/wiki/Software
  - pelasaco 4y ago
    exactly, for the price we paid, I was expecting Mark Abene, Marcus Ranum, or Bruce Schneier, not the dressed up, Little Bobby Tables.
Havoc 4y ago
Better question is which idiot came up with the idea of hiring unix skills from an accounting firm...
- pelasaco 4y ago
  I don't know if you know but Deloitte, one of the companies cited in the article, does everything including "security audit"
- NoPicklez 4y ago
  Or I think this shows your lack of understanding of auditing. In large global financial audits, there are many complex accounting systems that run on Unix systems. These systems as part of the financial audit are audited to ensure there is an appropriate level of IT general controls that support the underlying financial statements. I.e. what's stopping the admin from making direct data changes to the underlying database that supports the financial statements. Furthermore, those traditional accounting firms (Big 4) no longer and haven't for a decade + only provided accounting. These firms have enormous skills in internal audit, governance risk and controls, data science, cyber security, Information Technology etc.
jabroni_salad 4y ago
One of my duties is auditing other companies firewalls and you would not believe the kind of sieves I come across that could have been solved with a simple checklist. Bonus points: Critical stuff like meraki licensing or certificates that could be solved with a simple calendar. This stuff is 90% organizational skills and 10% technical skills. My job as a consultant is mostly on the same level as helping people wash their hands. The typical HN person is used to being on skilled technical teams at companies that build technology as their business, but let me tell you that all your business software consumers are constantly in a dire state from top to bottom.

eggsmediumrare 4y ago

Pilots with 40 years of experience use checklists every time they fly.

netjiro 4y ago
As long as the checklist catches more costly errors than the time to use it and the (very likely) negative secondary effects to critical thinking and innovation. Love tools that help, eg checklists, when used right. But over time they have a very strong tendency to become set-in-stone dogma, at which time they will create a priesthood and a very large dead zone where thinking and flexibility is no longer allowed.
pelasaco 4y ago
> Pilots with 40 years of experience use checklists every time they fly. I love when people just add comment for pure sake of commenting. The auditing companies don't get a "cyber security expert" with 40 years of experience, but get undergrads, suit them up and sell them as experts. Because of the checklists, that without knowing anything about the context, are almost useless and generate mostly noise.
- intelVISA 4y ago
  Can confirm boss, lscpu is GREEN on the no "iTLB multihit" vulns. We're clear. No I don't know what a VMM is but it's probably not relevant.
- eggsmediumrare 4y ago
  The comment I was replying to seemed to be suggesting that both inexperience AND checklists are the problem. Checklists are never a bad thing.
- NoPicklez 4y ago
  This is such a bullshit statement. Auditing companies absolutely get cyber security experts. Graduates coming out of University obviously don't have the experience of that of veterans but are in a team of experts with experience in that area. The checklists and workpapers are a way to ensure that there is a level of standardisation in the work that is being performed. If you have assessed a clients firewall or operating system as a veteran you are encouraged to replicate how you did that in a work paper or checklist to cover bases. Big 4 auditing firms are some of the best in class when it comes to knowledge sharing. Yes sometimes a junior or a senior might miss something without the context, but that is why humans developed lips to talk through it. I can't count the amount of times I have challenged what a junior has said, or the conclusion a junior has come to based on what a client has said based on my experience. I was once a graduate myself and found many glaring gaps through the use of simple checklists. If that's all it takes, then the industry needs to write some of their own.

matwood 4y ago

Checklists are standard for many professions and are known to work. IT should probably be using more of them. https://en.wikipedia.org/wiki/Checklist https://en.wikipedia.org/wiki/Checklist

pelasaco 4y ago
the checklist without intelligence is almost useless or generates too much noise. That's however those companies are able to profit. Generate a checklist, hire a undergrad to fill the checklists, organize it with a nice report, full of data, which mostly are noise, and ask a bunch of money, which is exactly the topic. Consultancies know less than they claim, because they are sending junior people with checklists to audit your company.
marcosdumay 4y ago
> IT should probably be using more of them. Probably not. IT has its own procedures for enforcing process compliance [1] that are much less susceptible to deviation and much more efficient. It's ok if you fail to capture something and need to use less effective methods like checklists. But going to them by default or, like the GP's said, only using them are bad. (But anyway, the GP was complaining because he was expecting highly valuable customized advice, and instead he got low-value process adherence checks. How you do your adherence checks is a completely different subject.) 1 - https://en.wikipedia.org/wiki/Software https://en.wikipedia.org/wiki/Software
- pelasaco 4y ago
  exactly, for the price we paid, I was expecting Mark Abene, Marcus Ranum, or Bruce Schneier, not the dressed up, Little Bobby Tables.

Havoc 4y ago

Better question is which idiot came up with the idea of hiring unix skills from an accounting firm...

pelasaco 4y ago
I don't know if you know but Deloitte, one of the companies cited in the article, does everything including "security audit"
NoPicklez 4y ago
Or I think this shows your lack of understanding of auditing. In large global financial audits, there are many complex accounting systems that run on Unix systems. These systems as part of the financial audit are audited to ensure there is an appropriate level of IT general controls that support the underlying financial statements. I.e. what's stopping the admin from making direct data changes to the underlying database that supports the financial statements. Furthermore, those traditional accounting firms (Big 4) no longer and haven't for a decade + only provided accounting. These firms have enormous skills in internal audit, governance risk and controls, data science, cyber security, Information Technology etc.

jabroni_salad 4y ago

One of my duties is auditing other companies firewalls and you would not believe the kind of sieves I come across that could have been solved with a simple checklist. Bonus points: Critical stuff like meraki licensing or certificates that could be solved with a simple calendar. This stuff is 90% organizational skills and 10% technical skills. My job as a consultant is mostly on the same level as helping people wash their hands. The typical HN person is used to being on skilled technical teams at companies that build technology as their business, but let me tell you that all your business software consumers are constantly in a dire state from top to bottom.