3 ms·

> The problem isn't that audits are inherently worthless, it's that most of these tools are very low-quality implementations of the concept. I think the proble

> The problem isn't that audits are inherently worthless, it's that most of these tools are very low-quality implementations of the concept. I think the problem is that the current audits are inherently worthless. I've never seen another industry that would accept a 95% false positive rate from a tool. But that's on the low end from my experiences (I've done 4 major codebase audits, and worked in the security industry for 5 years) Tools that routinely spit out 1000 plus vulnerabilities, and 6 months later you've checked off all of them without a single valid vulnerability in the list (but 3 you found yourself during that time that were missed entirely by the shitty regex that is really the entire tool in question).

bostik 4y ago
> current audits are inherently worthless And that's not helped by the warped incentives. Security has to be, fundamentally, assessed as a holistic thing and even highly skilled technical people can't really do that with modern systems. Now you add in auditors, a profession populated by glorified accountants, working off of gargantuan checklists. You invite and reward mediocrity. At best. Anyone good enough to actually understand and able to work with the technology will find a job in the industry, for twice the pay. And the same applies to regulators. They can't retain technically skilled staff, so they are filled with accountants. As a result we get rules and regulations, written by technically incompetent accountants, aimed at technically incompetent accountants. To fill the gaps we have shitty, false-positive ridden, noisy, outright useless tools that generate reports intended to satisfy accountants. Security has next to nothing to do with that. And to top it off, we have an entire industry riddled with so-called security engineers who know how to run a tool (with some marginally helpful StackOverflow pointers) and export a report, but can't for their life actually interpret, let alone VERIFY, any of it. That has as much to do with security engineering as burning ants with a magnifying glass has to do with biology.

bostik 4y ago

> current audits are inherently worthless And that's not helped by the warped incentives. Security has to be, fundamentally, assessed as a holistic thing and even highly skilled technical people can't really do that with modern systems. Now you add in auditors, a profession populated by glorified accountants, working off of gargantuan checklists. You invite and reward mediocrity. At best. Anyone good enough to actually understand and able to work with the technology will find a job in the industry, for twice the pay. And the same applies to regulators. They can't retain technically skilled staff, so they are filled with accountants. As a result we get rules and regulations, written by technically incompetent accountants, aimed at technically incompetent accountants. To fill the gaps we have shitty, false-positive ridden, noisy, outright useless tools that generate reports intended to satisfy accountants. Security has next to nothing to do with that. And to top it off, we have an entire industry riddled with so-called security engineers who know how to run a tool (with some marginally helpful StackOverflow pointers) and export a report, but can't for their life actually interpret, let alone VERIFY, any of it. That has as much to do with security engineering as burning ants with a magnifying glass has to do with biology.