3 ms·

I think I'd be looking for at least a refund on that pen test. I've never come across one that was anymore than a box ticking exercise.

by mcfedr 2y ago

spydum 2y ago
I've absolutely been involved (conducting, coordinating, and receiving) some high value pen tests over the years. One problem is there is no hard definition of what is considered a "pen test". I've seen very highly reputable vendors claim essentially out of the box nessus scans as pen tests, automated burpsuite scans as pen tests. In my own personal definition of a pen test: security practitioners may use those tools amongst others, but they generally leverage them as recon and then try to uncover pathways in from those vulns, in addition to abusing application logic and misconfiguration. Second problem: paid pen tests have limited scope and time constraints. If the application surface is sufficiently large, that engagement may simply not be big enough to conduct a thorough test. Contrast this with Bug Bounty hunters (and attackers): they have unbounded time and resources. They can literally keep testing until they find something.. and best part, there are so many of them! So these public bug disclosures are hard to compare to a private/paid for test. You could argue, the app owners didn't pay enough for a comprehensive test.. but the downside is: just because you paid more, doesn't mean the pen tester did a better job :( While they are high noise, I tend to think bug bounty programs are the best fit for the problem space. You end up with much deeper coverage, and a very positive ROI (even factoring in your engineers to triage the bounty reports).
eZpZpi 2y ago
Security is just box checking. Most IT work is. The deployed stack has limited set of parameters to learn and test for. Leetcode is popular hiring criteria for a reason; that kind of code checks the “KISS/don’t be clever” and DRY rediscovering known algorithms boxes Except in a few fields, most startups are pretty vanilla config ops and secops tasks. Recent popularity among the working class has inflated the egos of run of the mill office workers. “Programmers are lazy” has long been waved around like a badge of honor. Rather than Silicon Valley I’d like to see a Mad Men take on IT. Start in 06-ish with a bunch of entitled first world craft beer drunkards wasting nights on syntax art, framework wars, rise of cloud. End with Covid, launch LLM AI and a bunch of code school burnouts being laid off.
- Shocka1 2y ago
  This actually sounds like a great idea for a show and one I'd watch with great interest.

I think I'd be looking for at least a refund on that pen test. I've never come across one that was anymore than a box ticking exercise.

by mcfedr 2y ago

I think I'd be looking for at least a refund on that pen test. I've never come across one that was anymore than a box ticking exercise.

spydum 2y ago

I've absolutely been involved (conducting, coordinating, and receiving) some high value pen tests over the years. One problem is there is no hard definition of what is considered a "pen test". I've seen very highly reputable vendors claim essentially out of the box nessus scans as pen tests, automated burpsuite scans as pen tests. In my own personal definition of a pen test: security practitioners may use those tools amongst others, but they generally leverage them as recon and then try to uncover pathways in from those vulns, in addition to abusing application logic and misconfiguration. Second problem: paid pen tests have limited scope and time constraints. If the application surface is sufficiently large, that engagement may simply not be big enough to conduct a thorough test. Contrast this with Bug Bounty hunters (and attackers): they have unbounded time and resources. They can literally keep testing until they find something.. and best part, there are so many of them! So these public bug disclosures are hard to compare to a private/paid for test. You could argue, the app owners didn't pay enough for a comprehensive test.. but the downside is: just because you paid more, doesn't mean the pen tester did a better job :( While they are high noise, I tend to think bug bounty programs are the best fit for the problem space. You end up with much deeper coverage, and a very positive ROI (even factoring in your engineers to triage the bounty reports).

eZpZpi 2y ago

Security is just box checking. Most IT work is. The deployed stack has limited set of parameters to learn and test for. Leetcode is popular hiring criteria for a reason; that kind of code checks the “KISS/don’t be clever” and DRY rediscovering known algorithms boxes Except in a few fields, most startups are pretty vanilla config ops and secops tasks. Recent popularity among the working class has inflated the egos of run of the mill office workers. “Programmers are lazy” has long been waved around like a badge of honor. Rather than Silicon Valley I’d like to see a Mad Men take on IT. Start in 06-ish with a bunch of entitled first world craft beer drunkards wasting nights on syntax art, framework wars, rise of cloud. End with Covid, launch LLM AI and a bunch of code school burnouts being laid off.

Shocka1 2y ago
This actually sounds like a great idea for a show and one I'd watch with great interest.