4 ms·

> The worst offenders however are the requirements that some businesses simply cannot comply with unless they have some serious cash laying around. Examples of

> The worst offenders however are the requirements that some businesses simply cannot comply with unless they have some serious cash laying around. Examples of this are >> Quarterly external vulnerability scans must be performed by an Approved Scanning Vendor (ASV), approved by the Payment Card Industry Security Standards Council (PCI SSC). > and >> Is external penetration testing performed per the defined methodology, at least annually, and after any significant infrastructure or application changes to the environment (such as an operating system upgrade, a sub-network added to the environment, or an added web server)? Have you ever had a penetration test done? They basically run a load of OSS automated tools, generate a PDF report, and then charge you $1000s. It gives you no real insight and reveals nothing unless you've been a total noob. Why is this so expensive? Broadening of PCI scope + needlessly expensive compliance = Smells like a large opportunity.

metaverse 11y ago
We went through a SAQ D Service Provider 3.0, and paying for an ASV didn't hurt nearly as much as filling out that 80 page questionnaire... In fact, it reminded us apply some recent CVE's to our system before taking it to production. We used Comodo HackerGuardian which is $250/y, so you don't have to pay $1000s.
- akshatpradhan 11y ago
  I think my tool can actually help you. You document all your procedures in an organized structure using Github flavored markdown. http://cc-stg2.herokuapp.com/compliancechimp/documents/software-development-process-sdlc http://cc-stg2.herokuapp.com/compliancechimp/documents/softw... We're becoming the true Turbo Tax for compliance.
  - tomjen3 11y ago
    You probably have a good tool, but I am not even going to take a look at it when it doesn't have its own url. herokuapp just screams weekend project.
    - akshatpradhan 11y ago
      I do have a real tool. Its on ComplianceChimp.com. But its down right now because my TLS Certs expired a couple days ago and I never recorded my Key rotation procedures. I did think about temporarily disabling SSL, specifically for this HN post, but thought against it. We want to do compliance right. We're fixing the certs now and updating our Key rotation procedures for you all to see in our publicly viewable compliance workbook.
      - ceequof 11y ago
        Not to kick someone while they're down... but a security company screwing up their key rotation is not exactly a good sign.
        akshatpradhan 11y ago
        There's only so much I can get done with my dwindling runway =(. But you're right, that getting Key Rotation Procedures documented is the 1st thing I should have done. This is good feedback actually because now I know that after scoping the assets in my turbo tax-like tool, the very next thing a person should do is write down their key rotation procedures. Its also easier to write out as a procedure because its such a common yet forgetful practice. We're putting cycles into this right absolutely now.
dsacco 11y ago
I'm sorry that's been your experience. What should have happened is a primarily manual penetration test, administered by security engineers who themselves used to be fully competent software developers. Any automated tooling should have had strict manual verification and should not have been the focus of the test. Furthermore, superfluous results should not have been submitted in the PDF report. Strictly "policy" audits such as PCI compliance differ a bit, but in general they should still involve a technical deep dive into your product's infrastructure, conducted by consultants with expertise in multiple tech stacks and overall experience in a variety of frontend and backend frameworks. The final deliverable ("PDF report") also should have been hand-written, and in language that conveys technical expertise, complete with recommended steps towards remediation of any issues. My employer, Accuvant does this, as well as Matasano (more well known here on HN). As for why it's so expensive...well, I bill out at about $2000 per day. It really comes down to a lot of what people like patio11 and tptacek like to talk about here regarding consulting: 1. This is highly specialized work, with a much smaller population of competent engineers than typical web developers (for example). As such, it naturally receives a higher fee for supply and demand. Now, some people abuse this and run scans like Nessus and call it a day. These are not real infosec firms, they are parasites. 2. More specifically, we ask for it and we receive it, and we do exceedingly well. If people keep paying us five figures a week to perform a penetration test, we're not going to stop asking for it or reduce our prices.
- hueving 11y ago
  >These are not real infosec firms, they are parasites. The entire consulting penetration testing market is setup to encourage this behavior. There is no way to prove you actually did anything correct. Someone can write a wonderful PDF analysis by hand and still leave the system full of glaring holes. Customers can't tell a system is broken until it gets hacked. >More specifically, we ask for it and we receive it, and we do exceedingly well. If people keep paying us five figures a week to perform a penetration test, we're not going to stop asking for it or reduce our prices. Right, but many times I've seen companies do it because they are desperate to do it for compliance purposes. :/ Essentially there is a non-trivial portion of the market held up by regulatory demand. >Strictly "policy" audits such as PCI compliance differ a bit, but in general they should still involve a technical deep dive into your product's infrastructure, conducted by consultants with expertise in multiple tech stacks and overall experience in a variety of frontend and backend frameworks. I'm curious. Do you review every line of code in a customer's codebase? What about the code of every library they import? If you don't review imports, do you leave a big caveat in your report that says their code looks okay, but the libraries could be full of vulnerabilities?

metaverse 11y ago

We went through a SAQ D Service Provider 3.0, and paying for an ASV didn't hurt nearly as much as filling out that 80 page questionnaire... In fact, it reminded us apply some recent CVE's to our system before taking it to production. We used Comodo HackerGuardian which is $250/y, so you don't have to pay $1000s.

akshatpradhan 11y ago
I think my tool can actually help you. You document all your procedures in an organized structure using Github flavored markdown. http://cc-stg2.herokuapp.com/compliancechimp/documents/software-development-process-sdlc http://cc-stg2.herokuapp.com/compliancechimp/documents/softw... We're becoming the true Turbo Tax for compliance.
- tomjen3 11y ago
  You probably have a good tool, but I am not even going to take a look at it when it doesn't have its own url. herokuapp just screams weekend project.
  - akshatpradhan 11y ago
    I do have a real tool. Its on ComplianceChimp.com. But its down right now because my TLS Certs expired a couple days ago and I never recorded my Key rotation procedures. I did think about temporarily disabling SSL, specifically for this HN post, but thought against it. We want to do compliance right. We're fixing the certs now and updating our Key rotation procedures for you all to see in our publicly viewable compliance workbook.
    - ceequof 11y ago
      Not to kick someone while they're down... but a security company screwing up their key rotation is not exactly a good sign.
      - akshatpradhan 11y ago
        There's only so much I can get done with my dwindling runway =(. But you're right, that getting Key Rotation Procedures documented is the 1st thing I should have done. This is good feedback actually because now I know that after scoping the assets in my turbo tax-like tool, the very next thing a person should do is write down their key rotation procedures. Its also easier to write out as a procedure because its such a common yet forgetful practice. We're putting cycles into this right absolutely now.

dsacco 11y ago

I'm sorry that's been your experience. What should have happened is a primarily manual penetration test, administered by security engineers who themselves used to be fully competent software developers. Any automated tooling should have had strict manual verification and should not have been the focus of the test. Furthermore, superfluous results should not have been submitted in the PDF report. Strictly "policy" audits such as PCI compliance differ a bit, but in general they should still involve a technical deep dive into your product's infrastructure, conducted by consultants with expertise in multiple tech stacks and overall experience in a variety of frontend and backend frameworks. The final deliverable ("PDF report") also should have been hand-written, and in language that conveys technical expertise, complete with recommended steps towards remediation of any issues. My employer, Accuvant does this, as well as Matasano (more well known here on HN). As for why it's so expensive...well, I bill out at about $2000 per day. It really comes down to a lot of what people like patio11 and tptacek like to talk about here regarding consulting: 1. This is highly specialized work, with a much smaller population of competent engineers than typical web developers (for example). As such, it naturally receives a higher fee for supply and demand. Now, some people abuse this and run scans like Nessus and call it a day. These are not real infosec firms, they are parasites. 2. More specifically, we ask for it and we receive it, and we do exceedingly well. If people keep paying us five figures a week to perform a penetration test, we're not going to stop asking for it or reduce our prices.

hueving 11y ago
>These are not real infosec firms, they are parasites. The entire consulting penetration testing market is setup to encourage this behavior. There is no way to prove you actually did anything correct. Someone can write a wonderful PDF analysis by hand and still leave the system full of glaring holes. Customers can't tell a system is broken until it gets hacked. >More specifically, we ask for it and we receive it, and we do exceedingly well. If people keep paying us five figures a week to perform a penetration test, we're not going to stop asking for it or reduce our prices. Right, but many times I've seen companies do it because they are desperate to do it for compliance purposes. :/ Essentially there is a non-trivial portion of the market held up by regulatory demand. >Strictly "policy" audits such as PCI compliance differ a bit, but in general they should still involve a technical deep dive into your product's infrastructure, conducted by consultants with expertise in multiple tech stacks and overall experience in a variety of frontend and backend frameworks. I'm curious. Do you review every line of code in a customer's codebase? What about the code of every library they import? If you don't review imports, do you leave a big caveat in your report that says their code looks okay, but the libraries could be full of vulnerabilities?