5 ms·

This has been my experience with similar tools (mostly openvas / greenbone) and companies peddling vulnerability tests for compliance purposes. There doesn't se

This has been my experience with similar tools (mostly openvas / greenbone) and companies peddling vulnerability tests for compliance purposes. There doesn't seem to be a lot of "functional scanning" in this space, like you're talking about. So I agree the false positives are a pain in networks with patched systems but this also seems to be standard behavior.

ejcx 7y ago
Yes it’s unfortunately standard for expensive scanners too. Our goal was to make something easy to run and deploy with similar results to something like openvas or Nessus. I personally think accuracy issues are because security vendors don’t want to show that their expensive software found nothing, so they dial down the accuracy. This works using the same methods as a Nessus, banner grabbing and indexing issues. We are going to keep working on this and look for ways to make it more accurate, but I’d say our results with this versus our expensive scanners was pretty much the same.
- thaumaturgy 7y ago
  Yeah. FWIW those openvas reports really satisfied the C-level itch to have some kind of work product in the form of an official-looking report, and your looks identical. Overall you seem to have built an awesome toolkit to start with. Greenbone is a total pain to get working initially and openvas updates are broken out-of-the-box. If I were still doing this kind of stuff I'd be immediately trying out Flan Scan.
- jascii 7y ago
  "I personally think accuracy issues are because security vendors don’t want to show that their expensive software found nothing" That might be a bit cynical, not sure I'd call openvas expensive.. I think it comes down to the fact that the damage of a single false negative can far outweigh the cost of the false positives. In the end, these scanners are mostly a tool to help document your security efforts, often for compliance reasons, making you think about and document the choices you make from a security perspective.
  - ejcx 7y ago
    Definitely a little cynical =]. I think the same thing about a lot of 3rd party security consulting. OpenVAS is free, but the big issue we've had with it is the complexity of setting it up and maintaining it. We are a security team, and would rather not spend our time managing servers, especially since we aren't the best people to do that at Cloudflare.
    - bostik 7y ago
      Yeah, OpenVAS is a nasty piece of work to set up and operate reliably. Especially headless. Shameless plug: I got it to work. Will be doing a talk about the experience at week's DC4420 meeting.
      - papower 7y ago
        Are you in a position to share that session online or perhaps a blog post on your experience and approach ? I've looked at it (openvas), got something working, but was never happy with it and ended up returning to a simpler/proven nmap base that I could manage better and add complexity if/when needed.
        bostik 7y ago
        I was planning to do a blog post in about a month's time. The DC4420 meetings and/or talks are not recorded (luckily!), but I intend to polish the talk up for a future re-run. On the other hand.. I do have something that might be enough to get you going. The setup we built is open: https://github.com/smarkets/vuln-scanner https://github.com/smarkets/vuln-scanner - go have a look. The glue code has comments on some of the stranger bugs I had to work around. So does the readme. If something isn't clear, feel free to ask.
        papower 7y ago
        Thanks, looks promising. One of my challanges was understanding the the zoo of tests OpenVAS would run and trying to reliably select which ones to apply. Did you, or anyone here, ever spot a way of outputting all the tests (nmap scripts etc.) that a particular run would trigger (but without actually running them)
sigio 7y ago
Yup.... I haven't encountered products yet that actually report useful information on systems that receive regular automated software updates. All you get are loads of false-positives.
- jascii 7y ago
  In some industries you actually have to document and prove that. That's where these tools shine.
tssva 7y ago
Most vulnerability scanners with functional scanning capabilities are run without them. Enterprises are concerned about the effect of functional scanning upon their production systems, is it going to bring something down, and functional scanning often requires a level of system access which enterprises are usually not comfortable giving. Since they are rarely used vendors don't invest heavily in developing them.
- bitminer 7y ago
  My experience in corporate and government is the opposite. They require functional scanning. Always. But they also required expensive functional scammer/scanner subscriptions. I see this basic tool plus some scraping of changelogs (included in suse and redhat binary rpms) for cve numbers should substantially reduce the false positives. For the price of another 50 lines of python. Don't know what other distros or the BSDs have available.
  - tssva 7y ago
    We have had completely opposite experiences then. I have almost never never had a corporate or government entity require a functional scan in a production environment. Some have started off asking for one but in the end I have never had one actually do one. They always end up balking at the need to supply credentials and are uneasy with the risk to the stability of production system inherent in a functional scan. Instead they settle for the non-functional scan from the expensive subscription tools such as Qualys which they use. The majority of time these reports are being produced to fulfill an auditing or FISMA requirement and not considered worth the risk to systems a functional scan could present.

ejcx 7y ago

Yes it’s unfortunately standard for expensive scanners too. Our goal was to make something easy to run and deploy with similar results to something like openvas or Nessus. I personally think accuracy issues are because security vendors don’t want to show that their expensive software found nothing, so they dial down the accuracy. This works using the same methods as a Nessus, banner grabbing and indexing issues. We are going to keep working on this and look for ways to make it more accurate, but I’d say our results with this versus our expensive scanners was pretty much the same.

thaumaturgy 7y ago
Yeah. FWIW those openvas reports really satisfied the C-level itch to have some kind of work product in the form of an official-looking report, and your looks identical. Overall you seem to have built an awesome toolkit to start with. Greenbone is a total pain to get working initially and openvas updates are broken out-of-the-box. If I were still doing this kind of stuff I'd be immediately trying out Flan Scan.
jascii 7y ago
"I personally think accuracy issues are because security vendors don’t want to show that their expensive software found nothing" That might be a bit cynical, not sure I'd call openvas expensive.. I think it comes down to the fact that the damage of a single false negative can far outweigh the cost of the false positives. In the end, these scanners are mostly a tool to help document your security efforts, often for compliance reasons, making you think about and document the choices you make from a security perspective.
- ejcx 7y ago
  Definitely a little cynical =]. I think the same thing about a lot of 3rd party security consulting. OpenVAS is free, but the big issue we've had with it is the complexity of setting it up and maintaining it. We are a security team, and would rather not spend our time managing servers, especially since we aren't the best people to do that at Cloudflare.
  - bostik 7y ago
    Yeah, OpenVAS is a nasty piece of work to set up and operate reliably. Especially headless. Shameless plug: I got it to work. Will be doing a talk about the experience at week's DC4420 meeting.
    - papower 7y ago
      Are you in a position to share that session online or perhaps a blog post on your experience and approach ? I've looked at it (openvas), got something working, but was never happy with it and ended up returning to a simpler/proven nmap base that I could manage better and add complexity if/when needed.
      - bostik 7y ago
        I was planning to do a blog post in about a month's time. The DC4420 meetings and/or talks are not recorded (luckily!), but I intend to polish the talk up for a future re-run. On the other hand.. I do have something that might be enough to get you going. The setup we built is open: https://github.com/smarkets/vuln-scanner https://github.com/smarkets/vuln-scanner - go have a look. The glue code has comments on some of the stranger bugs I had to work around. So does the readme. If something isn't clear, feel free to ask.
        papower 7y ago
        Thanks, looks promising. One of my challanges was understanding the the zoo of tests OpenVAS would run and trying to reliably select which ones to apply. Did you, or anyone here, ever spot a way of outputting all the tests (nmap scripts etc.) that a particular run would trigger (but without actually running them)

sigio 7y ago

Yup.... I haven't encountered products yet that actually report useful information on systems that receive regular automated software updates. All you get are loads of false-positives.

jascii 7y ago
In some industries you actually have to document and prove that. That's where these tools shine.

tssva 7y ago

Most vulnerability scanners with functional scanning capabilities are run without them. Enterprises are concerned about the effect of functional scanning upon their production systems, is it going to bring something down, and functional scanning often requires a level of system access which enterprises are usually not comfortable giving. Since they are rarely used vendors don't invest heavily in developing them.

bitminer 7y ago
My experience in corporate and government is the opposite. They require functional scanning. Always. But they also required expensive functional scammer/scanner subscriptions. I see this basic tool plus some scraping of changelogs (included in suse and redhat binary rpms) for cve numbers should substantially reduce the false positives. For the price of another 50 lines of python. Don't know what other distros or the BSDs have available.
- tssva 7y ago
  We have had completely opposite experiences then. I have almost never never had a corporate or government entity require a functional scan in a production environment. Some have started off asking for one but in the end I have never had one actually do one. They always end up balking at the need to supply credentials and are uneasy with the risk to the stability of production system inherent in a functional scan. Instead they settle for the non-functional scan from the expensive subscription tools such as Qualys which they use. The majority of time these reports are being produced to fulfill an auditing or FISMA requirement and not considered worth the risk to systems a functional scan could present.