3 ms·

Security is near and dear to me too, so I'll use that example for my rebuttal. Would you like to know how many companies I have found serious vulnerabilities i

by dsacco 10y ago

Security is near and dear to me too, so I'll use that example for my rebuttal. Would you like to know how many companies I have found serious vulnerabilities in which had previous audits for things like PCI and ISO 27001? Their CISOs had cute certifications, too. They could talk all day about what XSS is, what a good business continuity plan looks like, all the different types of "risk treatment"... I've sat on the other side of an audit, as an internal security engineer for a bank. Our QSA literally said to my team one day, "That sounds great, now can we switch gears and talk about your cyber program?" I like to think we honestly did well on those PCI audits, but I also know that we didn't have to do well. We could bullshit our way through it. And even if you try hard, it has little to no signal. Requiring engineers to know OWASP, or to memorize organizational risk facts doesn't work. Requiring your company to get a third party audit sort of works, if you know what a reputable firm actually looks like. Requiring your company to get third party "network segmentation scans" is a waste of time that will leave you wondering how a company stays in business that reports false positives all over your infra. The security industry is a rabbit hole of oblivion. Vendors don't know what they're doing. Consulting firms barely know what they're doing. What you suggest will not meaningfully change that. As I repeat time and time again, the most talented and effective individuals I've known in the security industry have no certifications, or reluctantly get them to shut HR up. Some of them never even went to college. You would need to dramatically rethink what an enforcing agency looks like to have effective certifications without allowing them to be the rent-collecting they currently are. I would be genuinely interested in hearing a concrete proposal for what an effective certification or licensing body would look like and how it would fairly enforce its requirements. If it's a model that could work, then fine; hit reset on the current organizations or dramatically improve their processes. But comparisons to other industries aren't good enough, and thus far the evidence is stacked against it.

OliverJones 10y ago
You are right about the porousness of certifications. I guess part of the problem is the legalistic approach we take to security, in practice. A crude example: to be secure, passwords have to be changed every 90 days. Check. If the QSA firm finds we're following all the rules, we get certified. But rules lull us into complacency. Rules give cybercreeps an accurate roadmap of our cyberdefenses. If they know ways around our defenses, we're pwned. So, adhering to the letter of the security regulations, while not actually bad, isn't enough. Dealing with the OWASP top ten is good. But, if I were a cybercreep, I'd be working on the down-chart problems, OWASP 13, 14, 15 etc. A related question: How do we frame laws to outlaw corruption among politicians? Laws create loopholes. As you point out, lots of charlatans and idiots hold themselves out as cybersecurity experts. How do we deal with the problem? My immediate objective is to keep cybercreeps outside my firewall and away from my customers. And if they get in, my objective is to expel them quickly. I argue that transparency is key. We can only defend against what we can see. The open source movement helps. I argue that a registered professional engineer certification provides at least some transparency about qualifications and ethical motivation, a transparency our industry now lacks.

by dsacco 10y ago

OliverJones 10y ago

You are right about the porousness of certifications. I guess part of the problem is the legalistic approach we take to security, in practice. A crude example: to be secure, passwords have to be changed every 90 days. Check. If the QSA firm finds we're following all the rules, we get certified. But rules lull us into complacency. Rules give cybercreeps an accurate roadmap of our cyberdefenses. If they know ways around our defenses, we're pwned. So, adhering to the letter of the security regulations, while not actually bad, isn't enough. Dealing with the OWASP top ten is good. But, if I were a cybercreep, I'd be working on the down-chart problems, OWASP 13, 14, 15 etc. A related question: How do we frame laws to outlaw corruption among politicians? Laws create loopholes. As you point out, lots of charlatans and idiots hold themselves out as cybersecurity experts. How do we deal with the problem? My immediate objective is to keep cybercreeps outside my firewall and away from my customers. And if they get in, my objective is to expel them quickly. I argue that transparency is key. We can only defend against what we can see. The open source movement helps. I argue that a registered professional engineer certification provides at least some transparency about qualifications and ethical motivation, a transparency our industry now lacks.