14 ms·

The security scanner that cried wolf

104 points by feross 6y ago

franciscop 6y ago
Note: sorry if I sound harsh, but these companies are raising millions to basically add a burden to the backs of hard-working OSS authors instead of helping the ecosystem, it's really shameless IMHO. Positive note on the end! I've seen multiple known OSS authors speaking publicly unfavorably about these security scanners. It seems everyone is offering a "solution" to the vulnerability issue with Open Source, but they do it in a way that at best it's a reminder to update your dependencies (as noted in the article) and at worst it's a burden to OSS authors. Their interests are also very misaligned with everyone else in the field; best security practices are to communicate security issues with the author privately, have you or them make a fix with a reasonable timeline and then publish a fix for everyone (or reveal the issues publicly after X days). That'd help both authors and users, but unfortunately it doesn't help selling the security tools. Instead, due to this misalignment of incentives, the current player's workflow include bad practices like: - First time someone notices a potential vulnerability they will register it in their system, and then if you are lucky you'll get a public issue. Responsive disclosure sounds like "no publicity for us so nah". - Paint features as security vulnerabilities. Example: your "arbitrary command execution tool" can "execute arbitrary commands" or "your JSON reading tool can read JSON". Well, thanks, that's totally useless. - Tell users that your dev libraries, that never end up in production, have critical vulnerabilities. Sure if my CSS compilation tool has one it won't matter since it doesn't end up in the production server (unless it's a vuln. in the output). Edit: for a positive note, these are the current champions IMHO that are helping the ecosystem in this field (besides authors themselves ofc): - Github's free Actions for OSS. This has been amazingly helpful for testing for free and easily, and basically my turning point on believing that Microsoft has evolved for the better. - Hundreds or thousands of volunteer fellow OSS authors disclosing vulnerabilities privately. They are mostly anonymous, but def a huge force. - Companies releasing open source, since they are in a better position to offer big well-rounded libraries that reduce your dependency graph.
- WrtCdEvrydy 6y ago
  It's because the risk of a failure is high. If you end up not flagging something and it causes a vulnerability to become a data leak, you will end up getting blamed as a classic "noone could have seen this coming"
  - aequitas 6y ago
    The risk of a failure is _sometimes_ high and this will lead to something called alert fatigue: https://en.wikipedia.org/wiki/Alarm_fatigue https://en.wikipedia.org/wiki/Alarm_fatigue
    - WrtCdEvrydy 6y ago
      The risk of failure to the scanner company is high. The risk of failure to you is low. That's the difference, they are shifting the burden of security to you... since if you don't patch the "exploit" which may not be an exploit, they can always point back to you in case there is a security breach because you didn't listen to the "magical scanner". It's a clear conflict of interest from twisted incentives. Additionally, if the thing never throws up any exploit warnings, you might wonder why you're paying so much for this fancy alerting thing.
      - aequitas 6y ago
        The value of a scanner with this kind of s/n ratio is very low. Because it would still cost me a lot time (thus money) to investigate every alert every time. What I would pay for is a scanner that is contextually aware and alerts when needed, give or take a few false positives. Not ones that have false positive as their default.
        WrtCdEvrydy 6y ago
        Now you're getting it. Now think about the fact that most auditors want you running a scanner on your network (and it's required for certain legislation like GDPR)
williesleg 6y ago
We used to call them script kiddies, now they're the ISO
jascii 6y ago
I can't help but feel the author is a little misguided about the purpose of security scanners. Security is a dynamic process, and scanners are a tool to help compare the current state of your systems with the policies you have set. Out of the box, the scanners can not be aware of your policies and will by necessity produce false positives.
- vbezhenar 6y ago
  In my experience those scanners were used for software certification. You're paying money, they run those scanners on your system, they spew meaningless warnings, you're doing meaningless actions (hopefully harmless) to mitigate those warnings and you've passed a certification, congratulations. Nobody checks whether those warnings make any sense, nobody checks what exactly did you do to mitigate those warnings, it's security theater. And that was extremely expensive scanner, I checked their website and they're selling it for big money. Probably other people find it more useful.
  - dumpsterdiver 6y ago
    I've heard this sentiment repeated quite a bit by non-technical people who hover around the fringes of the security field (compliance officers and the like), that noone actually cares and we're just checking off boxes. That's not fair for those of us who really do care. As someone in offensive security, I can assure you that there's a huge difference between forgetting to check off a box to pay lip service and leaving a critical server wide open to hackers. The former will simply bounce back until you check it, the latter might put your company in the news or even out of business.
    - jascii 6y ago
      In my experience with compliance (mostly NIST 800-53) auditors will not just want to see scans, but more importantly an assessment of whether found issues were applicable and what actions are being taken to mitigate. Maybe I have just been lucky to work with particularly good auditors though.
  - 6y ago
scjody 6y ago
We've been shipping a Docker-based app to customers for years, and every now and then one of them runs a security scanner on our images. I have yet to see a scan that isn't a disaster of false positives (for the reasons outlined in the article and more!) One of the craziest recent examples was a scan using a tool called Twistlock. Many of our images are built from an upstream image that may have outdated apt dependencies, so one of the first things we do is upgrade them. Twistlock flagged _every instance_ of this because "Package binaries should not be altered" (in other words, between subsequent layers in an image). I am baffled how anyone at Twistlock decided that this was a useful thing for their product to detect, or why any Twistlock customer trusts it given issues like this.
- nonameiguess 6y ago
  It flags that because it could indicate someone got onto your system and injected their own code or changed machine instructions at the binary level, which is a pretty common way to get a remote shell. It is annoying to have to mark false positives, but that's just the nature of the beast when it comes to being thorough about security. More annoying with this check than when you update packages in a container image instead of starting clean is that this same technique is often used to compare hashes of packages managed by an installer versus what is actually on disk, and thus flags every single package in a JIT-compiled language that caches byte code on disk as altered.
  - CameronNemo 6y ago
    I have used Prisma Cloud / twistlock. The tampering detection is only useful for detecting changes to running containers, not for changes to binaries between layers. The latter is just dumb and causes anti-productive false positives like above.
  - acdha 6y ago
    It’s because they’re implementing the feature so they can show a CISO a big scary report and say “good thing you paid us - otherwise you wouldn’t have known!” If they were serious about build errors they could use the built-in features of APT, YUM, etc. to only report binaries which don’t match the canonical distribution’s hashes, as has been standard sysadmin practice for aeons.
dec0dedab0de 6y ago
So the same CVEs show up for debian but not redhat even though they're not being fixed upstream. Is IBM/redhat paying someone here? I don't understand why it would be different.
- raesene9 6y ago
  AFAIK the reason it's different is that Debian/Ubuntu publish lists of unpatched CVEs but I don't believe that RH do. What scanners are often doing is pulling the published Sec-db for a distribution and using that as their data source. So as a result they can only report on things that are in the data source.
  - dec0dedab0de 6y ago
    Maybe I just don't understand CVEs. I thought it would be published at a third party and linked to glibc, not to a specific distro.
    - CameronNemo 6y ago
      Trivy is open source. You can check where they get their data from. But I imagine they use this for Debian: https://security-tracker.debian.org/tracker/ https://security-tracker.debian.org/tracker/
    - raesene9 6y ago
      There is a list of CVEs available via NVD or similar, however the challenge for a scanner is "how do I know if the binaries I'm seeing have been patched" The common way to do that is to lean on the linux distribution's package management system, which is why the way it works varies from distro to distro.
- deleted 6y ago
  [deleted]
carlosf 6y ago
Great read. I believe that's an issue not just with container security scanners, but with automated security in general. One example: I enabled AWS Security Hub to have some sort of security score on my account and although it found a few interesting, it generates way too much noise to be usable. Microsoft also has this sort of security score for Azure and Office 365 environments and they have the same issues.
raesene9 6y ago
This article shows an interesting contrast between how different types of scanning tools address the issue of valid CVEs for which there are no updates. The question is "do you want to know about a valid CVE if there's no patch available"? For high security environments, the answer might be yes (you want to evaluate individual patches to see if they impact your application) For many environments the answer is likely (as is the case with OP) no, you only want to know about issues for which there is an updated package available. Interestingly (well to me :) ) more established scanning tools (e.g. Tenable Nessus) default to "no" on this question and container scanning tools mostly default to "yes". (I did an example comparison https://raesene.github.io/blog/2020/11/22/When_Is_A_Vulnerability_Not_A_Vulnerability/ https://raesene.github.io/blog/2020/11/22/When_Is_A_Vulnerab...) The important part is that your organization makes an informed choice about which works best for them. As a sidenote there are also differences in severities and how vulnerabilities are counted (e.g. if you have multiple issues in a package do you count each one or just count once for that package) which can lead to different numbers of issues being reported by different scanners. https://raesene.github.io/blog/2020/06/21/Container_Vulnerability_Scanning_Fun/ https://raesene.github.io/blog/2020/06/21/Container_Vulnerab...
- WrtCdEvrydy 6y ago
  Here's the question, if you default to "yes", wouldn't you want to throw "There might be CVEs in this image that don't have a CVE assigned" for every container...
  - raesene9 6y ago
    well I'd say there's a difference there, I guess between "no patch but CVE" and "no known CVE" :)
- thaeli 6y ago
  The logic I really want is "hide old unfixed but show me unfixed that's less than X days old" so new stuff surfaces briefly but these zombie vulns aren't constant noise.
  - 6y ago
c7DJTLrn 6y ago
My impression of container scanners has always been that they are security theatre. Something that lets companies say 'look how security conscious we are'. Same for code scanners. If an automated process really could find vulnerabilities to the same fidelity as a human pentester, that would be groundbreaking. In most cases, companies don't want to fork out the cost for real security, so they run these useless scanners instead.
- cube00 6y ago
  I lost all respect for those scanners after I found one was flagging libraries (even the latest) because it had an open ended version range. Upon digging into why it was because "the library has an insecure mode in it" and the warning will remain until the library authors remove that mode. Here's a better idea, how about you flag when I actually use the mode? If I don't use it, don't flag and let me get on with my real job. To top off the scanner didn't support suppressing single vulnerabilities so our options were (1) the build remained "unstable" forever or (2) we ignore the scanning result so we can get back to a stable build.
- brendoelfrendo 6y ago
  I mean, plenty of human pen-testers also use automated tools. It's common knowledge (or at least I thought it was) that these scanners produce contextless output that is going to require analysis in order to be useful. The scanner is just looking for known vulnerabilities; that's a useful tool, because you don't necessarily need to know that a CVE was published to know that it exists in your environment. A human still needs to step in and match that finding to a real risk of exploitation.
collsni 6y ago
This isn't only a problem with docker security scanners it's a problem with linux security scanners in general. There are vulnerabilities out there that just aren't patched and they show up on scans and you need to be able to differentiate what you can mitigate and what you can't.
angry_octet 6y ago
There's no way to specify what you'll be doing with the container, or how the vuln is triggered, so of course it can't provide useful guidance. The whole CVE report mechanism would have to change to allow low FAR scanning.
aidenn0 6y ago
Why am I not surprised that the glibc maintainers dispute a security vulnerability. They might actually be right in this case, but glibc has a long history of breaking programs and not caring, which combined with strongly preferring dynamic linking makes it a mess to deal with.
_wldu 6y ago
This is one reason I switched to Alpine from Debian as a base. It's has literally 0 security issues. The auditors love it and I love the small size and speed. Win win.
- cratermoon 6y ago
  From the article: > this image has every security update put out by Debian But the scanner still flagged it.
  - _wldu 6y ago
    Right, but audit doesn't care about that. They just see the number of reported vulnerabilities and flip out. Make that better! Fix that! Alpine to the rescue. Do you want to spend time in meetings explaining why 100+ CVEs are nonsense or would you rather get Alpine up and running? I know which I would have more fun doing ;) I know, it's not really an issue (from a security perspective) but try explaining that to an audit manager who only wants to get a 'good' report for the board. This is really more of an indication of what is wrong with the security industry.
    - cratermoon 6y ago
      That's fair, but you might run into another wall if the people worrying about the nonsense CVEs are also resistant to Alpine for other reasons.
      - GordonS 6y ago
        IIRC, there is a slight performance penalty with Alpine, compared with glibc - are there any other reasons to be resistant to Alpine?
        cratermoon 6y ago
        Yea, in an "enterprisey" organization you'll have auditors who will note that it doesn't have XYZ SoopEr SeCure Certification v3.141 and there isn't an Alpine LTS Enterprise Edition with White Glove Support.
cratermoon 6y ago
> If everything has security vulnerabilities, nothing has security vulnerabilities This reminds me of the era when everyone thought that the way to prevent users from doing things that permanently delete data and can't be undone was to throw up a big red "Are you sure?" confirmation. After a while users use got habituated to clicking "Yes", which made the warnings useless.
- genewitch 6y ago
  I was attempting to reboot a VM inside of a proxmox "HA" cluster, and somehow i rebooted one of the hypervisors. There was no "are you sure?" for rebooting the metal. I revoked my own "mess with the hyper" credentials that time. I didn't even know i could do that.
  - cratermoon 6y ago
    Oh yes, there are plenty of footguns in the administrator/operations/developer space. That's just one of the reasons that I refuse "oh you're in X team, here's unfettered everything access in case you need it" offers. Just give me the least privilege I need to complete the task at hand.
glsdfgkjsklfj 6y ago
on a related note, most antivirus companies get paid to mark things such as open source printer driver hacks that allow you to use refillable cartridges and what not. They will show up on scans with names like "malware.generic.12345"
tptacek 6y ago
I think this is directionally right about security scanners. But while "no patch available" is a problem, I think it's the wrong problem to think about. One obvious concern about that line of thinking is that an exposure is an exposure regardless of whether there's a patch available. The bigger problem, not just with container scanners but with all sorts of scanners (Node dependency scanners are another good example) is that there's a huge incentive to "enumerate" vulnerabilities with CVEs, and most of these vulnerabilities are meaningless. I spent a couple years at my previous gig triaging security scanner results. Like pretty much everyone who does this professionally, I got so I could do this pretty quickly, and without much though about my actual environment; whole classes of vulnerabilities are pretty much garbage, and most people would be better off if their scanner required a special flag to alert on them. What you really need to know is, given a vulnerability, is it likely to constitute an exposure in my environment? Clearly, a scanner vendor can't do this analysis perfectly. But they can do more than the zero they do now. I started my computer security career at a firm called Secure Networks. We sold a security scanner called Ballista. We competed with a much-better-known firm called ISS, who sold the eponymous ISS scanner. The big debate between us and in the market was how to "count" vulnerabilities; ISS, for instance, had a huge collection of Windows registry best-practices rules, and claimed every one of them as (in effect) a vulnerability, while we tried to claim only what our scanner could actually exploit. We lost that debate, obviously, and the CVE system followed shortly thereafter. I think the general thing to keep in mind is that, for the most part, modern security scanner teams are pretty small. They're driven off vendorsec and CVE feeds because that's automatable. Like ISS vs SNI, they compete with each other, and their figure of merit is often "how many", not "how important" --- a metric they're not staffed to generate anyways. I wouldn't put much stock in their results.
- hundt 6y ago
  Having implemented it for a job, this has been my impression of container scanning as well. Either your container has an OS, in which case you don't use 99% of the software on it and so 99% of the vulnerabilities found do not affect you, or else you have a container with "scratch" as the base image, in which case the scanner has no insight.
- sethev 6y ago
rurban 6y ago
I find the reasoning weird, first coming with the highly disputed allegation that Debian is a "well trusted and secure" distro, and explaining away the complete lack of security process in Debian. The Debian lack of security is legendary. Redhat came up with 0 security issues, whilst Debian came up with 63 illegal/wontfix issues. Just do your homework, Debian!

104 points by feross 6y ago

Note: sorry if I sound harsh, but these companies are raising millions to basically add a burden to the backs of hard-working OSS authors instead of helping the ecosystem, it's really shameless IMHO. Positive note on the end! I've seen multiple known OSS authors speaking publicly unfavorably about these security scanners. It seems everyone is offering a "solution" to the vulnerability issue with Open Source, but they do it in a way that at best it's a reminder to update your dependencies (as noted in the article) and at worst it's a burden to OSS authors. Their interests are also very misaligned with everyone else in the field; best security practices are to communicate security issues with the author privately, have you or them make a fix with a reasonable timeline and then publish a fix for everyone (or reveal the issues publicly after X days). That'd help both authors and users, but unfortunately it doesn't help selling the security tools. Instead, due to this misalignment of incentives, the current player's workflow include bad practices like: - First time someone notices a potential vulnerability they will register it in their system, and then if you are lucky you'll get a public issue. Responsive disclosure sounds like "no publicity for us so nah". - Paint features as security vulnerabilities. Example: your "arbitrary command execution tool" can "execute arbitrary commands" or "your JSON reading tool can read JSON". Well, thanks, that's totally useless. - Tell users that your dev libraries, that never end up in production, have critical vulnerabilities. Sure if my CSS compilation tool has one it won't matter since it doesn't end up in the production server (unless it's a vuln. in the output). Edit: for a positive note, these are the current champions IMHO that are helping the ecosystem in this field (besides authors themselves ofc): - Github's free Actions for OSS. This has been amazingly helpful for testing for free and easily, and basically my turning point on believing that Microsoft has evolved for the better. - Hundreds or thousands of volunteer fellow OSS authors disclosing vulnerabilities privately. They are mostly anonymous, but def a huge force. - Companies releasing open source, since they are in a better position to offer big well-rounded libraries that reduce your dependency graph.

WrtCdEvrydy 6y ago
It's because the risk of a failure is high. If you end up not flagging something and it causes a vulnerability to become a data leak, you will end up getting blamed as a classic "noone could have seen this coming"
- aequitas 6y ago
  The risk of a failure is _sometimes_ high and this will lead to something called alert fatigue: https://en.wikipedia.org/wiki/Alarm_fatigue https://en.wikipedia.org/wiki/Alarm_fatigue
  - WrtCdEvrydy 6y ago
    The risk of failure to the scanner company is high. The risk of failure to you is low. That's the difference, they are shifting the burden of security to you... since if you don't patch the "exploit" which may not be an exploit, they can always point back to you in case there is a security breach because you didn't listen to the "magical scanner". It's a clear conflict of interest from twisted incentives. Additionally, if the thing never throws up any exploit warnings, you might wonder why you're paying so much for this fancy alerting thing.
    - aequitas 6y ago
      The value of a scanner with this kind of s/n ratio is very low. Because it would still cost me a lot time (thus money) to investigate every alert every time. What I would pay for is a scanner that is contextually aware and alerts when needed, give or take a few false positives. Not ones that have false positive as their default.
      - WrtCdEvrydy 6y ago
        Now you're getting it. Now think about the fact that most auditors want you running a scanner on your network (and it's required for certain legislation like GDPR)

williesleg 6y ago

We used to call them script kiddies, now they're the ISO

jascii 6y ago

I can't help but feel the author is a little misguided about the purpose of security scanners. Security is a dynamic process, and scanners are a tool to help compare the current state of your systems with the policies you have set. Out of the box, the scanners can not be aware of your policies and will by necessity produce false positives.

vbezhenar 6y ago
In my experience those scanners were used for software certification. You're paying money, they run those scanners on your system, they spew meaningless warnings, you're doing meaningless actions (hopefully harmless) to mitigate those warnings and you've passed a certification, congratulations. Nobody checks whether those warnings make any sense, nobody checks what exactly did you do to mitigate those warnings, it's security theater. And that was extremely expensive scanner, I checked their website and they're selling it for big money. Probably other people find it more useful.
- dumpsterdiver 6y ago
  I've heard this sentiment repeated quite a bit by non-technical people who hover around the fringes of the security field (compliance officers and the like), that noone actually cares and we're just checking off boxes. That's not fair for those of us who really do care. As someone in offensive security, I can assure you that there's a huge difference between forgetting to check off a box to pay lip service and leaving a critical server wide open to hackers. The former will simply bounce back until you check it, the latter might put your company in the news or even out of business.
  - jascii 6y ago
    In my experience with compliance (mostly NIST 800-53) auditors will not just want to see scans, but more importantly an assessment of whether found issues were applicable and what actions are being taken to mitigate. Maybe I have just been lucky to work with particularly good auditors though.
- 6y ago

scjody 6y ago

We've been shipping a Docker-based app to customers for years, and every now and then one of them runs a security scanner on our images. I have yet to see a scan that isn't a disaster of false positives (for the reasons outlined in the article and more!) One of the craziest recent examples was a scan using a tool called Twistlock. Many of our images are built from an upstream image that may have outdated apt dependencies, so one of the first things we do is upgrade them. Twistlock flagged _every instance_ of this because "Package binaries should not be altered" (in other words, between subsequent layers in an image). I am baffled how anyone at Twistlock decided that this was a useful thing for their product to detect, or why any Twistlock customer trusts it given issues like this.

nonameiguess 6y ago
It flags that because it could indicate someone got onto your system and injected their own code or changed machine instructions at the binary level, which is a pretty common way to get a remote shell. It is annoying to have to mark false positives, but that's just the nature of the beast when it comes to being thorough about security. More annoying with this check than when you update packages in a container image instead of starting clean is that this same technique is often used to compare hashes of packages managed by an installer versus what is actually on disk, and thus flags every single package in a JIT-compiled language that caches byte code on disk as altered.
- CameronNemo 6y ago
  I have used Prisma Cloud / twistlock. The tampering detection is only useful for detecting changes to running containers, not for changes to binaries between layers. The latter is just dumb and causes anti-productive false positives like above.
- acdha 6y ago
  It’s because they’re implementing the feature so they can show a CISO a big scary report and say “good thing you paid us - otherwise you wouldn’t have known!” If they were serious about build errors they could use the built-in features of APT, YUM, etc. to only report binaries which don’t match the canonical distribution’s hashes, as has been standard sysadmin practice for aeons.

dec0dedab0de 6y ago

So the same CVEs show up for debian but not redhat even though they're not being fixed upstream. Is IBM/redhat paying someone here? I don't understand why it would be different.

raesene9 6y ago
AFAIK the reason it's different is that Debian/Ubuntu publish lists of unpatched CVEs but I don't believe that RH do. What scanners are often doing is pulling the published Sec-db for a distribution and using that as their data source. So as a result they can only report on things that are in the data source.
- dec0dedab0de 6y ago
  Maybe I just don't understand CVEs. I thought it would be published at a third party and linked to glibc, not to a specific distro.
  - CameronNemo 6y ago
    Trivy is open source. You can check where they get their data from. But I imagine they use this for Debian: https://security-tracker.debian.org/tracker/ https://security-tracker.debian.org/tracker/
  - raesene9 6y ago
    There is a list of CVEs available via NVD or similar, however the challenge for a scanner is "how do I know if the binaries I'm seeing have been patched" The common way to do that is to lean on the linux distribution's package management system, which is why the way it works varies from distro to distro.
deleted 6y ago
[deleted]

carlosf 6y ago

Great read. I believe that's an issue not just with container security scanners, but with automated security in general. One example: I enabled AWS Security Hub to have some sort of security score on my account and although it found a few interesting, it generates way too much noise to be usable. Microsoft also has this sort of security score for Azure and Office 365 environments and they have the same issues.

raesene9 6y ago

This article shows an interesting contrast between how different types of scanning tools address the issue of valid CVEs for which there are no updates. The question is "do you want to know about a valid CVE if there's no patch available"? For high security environments, the answer might be yes (you want to evaluate individual patches to see if they impact your application) For many environments the answer is likely (as is the case with OP) no, you only want to know about issues for which there is an updated package available. Interestingly (well to me :) ) more established scanning tools (e.g. Tenable Nessus) default to "no" on this question and container scanning tools mostly default to "yes". (I did an example comparison https://raesene.github.io/blog/2020/11/22/When_Is_A_Vulnerability_Not_A_Vulnerability/ https://raesene.github.io/blog/2020/11/22/When_Is_A_Vulnerab...) The important part is that your organization makes an informed choice about which works best for them. As a sidenote there are also differences in severities and how vulnerabilities are counted (e.g. if you have multiple issues in a package do you count each one or just count once for that package) which can lead to different numbers of issues being reported by different scanners. https://raesene.github.io/blog/2020/06/21/Container_Vulnerability_Scanning_Fun/ https://raesene.github.io/blog/2020/06/21/Container_Vulnerab...

WrtCdEvrydy 6y ago
Here's the question, if you default to "yes", wouldn't you want to throw "There might be CVEs in this image that don't have a CVE assigned" for every container...
- raesene9 6y ago
  well I'd say there's a difference there, I guess between "no patch but CVE" and "no known CVE" :)
thaeli 6y ago
The logic I really want is "hide old unfixed but show me unfixed that's less than X days old" so new stuff surfaces briefly but these zombie vulns aren't constant noise.
- 6y ago

c7DJTLrn 6y ago

My impression of container scanners has always been that they are security theatre. Something that lets companies say 'look how security conscious we are'. Same for code scanners. If an automated process really could find vulnerabilities to the same fidelity as a human pentester, that would be groundbreaking. In most cases, companies don't want to fork out the cost for real security, so they run these useless scanners instead.

cube00 6y ago
I lost all respect for those scanners after I found one was flagging libraries (even the latest) because it had an open ended version range. Upon digging into why it was because "the library has an insecure mode in it" and the warning will remain until the library authors remove that mode. Here's a better idea, how about you flag when I actually use the mode? If I don't use it, don't flag and let me get on with my real job. To top off the scanner didn't support suppressing single vulnerabilities so our options were (1) the build remained "unstable" forever or (2) we ignore the scanning result so we can get back to a stable build.
brendoelfrendo 6y ago
I mean, plenty of human pen-testers also use automated tools. It's common knowledge (or at least I thought it was) that these scanners produce contextless output that is going to require analysis in order to be useful. The scanner is just looking for known vulnerabilities; that's a useful tool, because you don't necessarily need to know that a CVE was published to know that it exists in your environment. A human still needs to step in and match that finding to a real risk of exploitation.

collsni 6y ago

This isn't only a problem with docker security scanners it's a problem with linux security scanners in general. There are vulnerabilities out there that just aren't patched and they show up on scans and you need to be able to differentiate what you can mitigate and what you can't.

angry_octet 6y ago

There's no way to specify what you'll be doing with the container, or how the vuln is triggered, so of course it can't provide useful guidance. The whole CVE report mechanism would have to change to allow low FAR scanning.

aidenn0 6y ago

Why am I not surprised that the glibc maintainers dispute a security vulnerability. They might actually be right in this case, but glibc has a long history of breaking programs and not caring, which combined with strongly preferring dynamic linking makes it a mess to deal with.

_wldu 6y ago

This is one reason I switched to Alpine from Debian as a base. It's has literally 0 security issues. The auditors love it and I love the small size and speed. Win win.

cratermoon 6y ago
From the article: > this image has every security update put out by Debian But the scanner still flagged it.
- _wldu 6y ago
  Right, but audit doesn't care about that. They just see the number of reported vulnerabilities and flip out. Make that better! Fix that! Alpine to the rescue. Do you want to spend time in meetings explaining why 100+ CVEs are nonsense or would you rather get Alpine up and running? I know which I would have more fun doing ;) I know, it's not really an issue (from a security perspective) but try explaining that to an audit manager who only wants to get a 'good' report for the board. This is really more of an indication of what is wrong with the security industry.
  - cratermoon 6y ago
    That's fair, but you might run into another wall if the people worrying about the nonsense CVEs are also resistant to Alpine for other reasons.
    - GordonS 6y ago
      IIRC, there is a slight performance penalty with Alpine, compared with glibc - are there any other reasons to be resistant to Alpine?
      - cratermoon 6y ago
        Yea, in an "enterprisey" organization you'll have auditors who will note that it doesn't have XYZ SoopEr SeCure Certification v3.141 and there isn't an Alpine LTS Enterprise Edition with White Glove Support.

cratermoon 6y ago

> If everything has security vulnerabilities, nothing has security vulnerabilities This reminds me of the era when everyone thought that the way to prevent users from doing things that permanently delete data and can't be undone was to throw up a big red "Are you sure?" confirmation. After a while users use got habituated to clicking "Yes", which made the warnings useless.

genewitch 6y ago
I was attempting to reboot a VM inside of a proxmox "HA" cluster, and somehow i rebooted one of the hypervisors. There was no "are you sure?" for rebooting the metal. I revoked my own "mess with the hyper" credentials that time. I didn't even know i could do that.
- cratermoon 6y ago
  Oh yes, there are plenty of footguns in the administrator/operations/developer space. That's just one of the reasons that I refuse "oh you're in X team, here's unfettered everything access in case you need it" offers. Just give me the least privilege I need to complete the task at hand.

glsdfgkjsklfj 6y ago

on a related note, most antivirus companies get paid to mark things such as open source printer driver hacks that allow you to use refillable cartridges and what not. They will show up on scans with names like "malware.generic.12345"

tptacek 6y ago

I think this is directionally right about security scanners. But while "no patch available" is a problem, I think it's the wrong problem to think about. One obvious concern about that line of thinking is that an exposure is an exposure regardless of whether there's a patch available. The bigger problem, not just with container scanners but with all sorts of scanners (Node dependency scanners are another good example) is that there's a huge incentive to "enumerate" vulnerabilities with CVEs, and most of these vulnerabilities are meaningless. I spent a couple years at my previous gig triaging security scanner results. Like pretty much everyone who does this professionally, I got so I could do this pretty quickly, and without much though about my actual environment; whole classes of vulnerabilities are pretty much garbage, and most people would be better off if their scanner required a special flag to alert on them. What you really need to know is, given a vulnerability, is it likely to constitute an exposure in my environment? Clearly, a scanner vendor can't do this analysis perfectly. But they can do more than the zero they do now. I started my computer security career at a firm called Secure Networks. We sold a security scanner called Ballista. We competed with a much-better-known firm called ISS, who sold the eponymous ISS scanner. The big debate between us and in the market was how to "count" vulnerabilities; ISS, for instance, had a huge collection of Windows registry best-practices rules, and claimed every one of them as (in effect) a vulnerability, while we tried to claim only what our scanner could actually exploit. We lost that debate, obviously, and the CVE system followed shortly thereafter. I think the general thing to keep in mind is that, for the most part, modern security scanner teams are pretty small. They're driven off vendorsec and CVE feeds because that's automatable. Like ISS vs SNI, they compete with each other, and their figure of merit is often "how many", not "how important" --- a metric they're not staffed to generate anyways. I wouldn't put much stock in their results.

hundt 6y ago
Having implemented it for a job, this has been my impression of container scanning as well. Either your container has an OS, in which case you don't use 99% of the software on it and so 99% of the vulnerabilities found do not affect you, or else you have a container with "scratch" as the base image, in which case the scanner has no insight.
sethev 6y ago

rurban 6y ago

I find the reasoning weird, first coming with the highly disputed allegation that Debian is a "well trusted and secure" distro, and explaining away the complete lack of security process in Debian. The Debian lack of security is legendary. Redhat came up with 0 security issues, whilst Debian came up with 63 illegal/wontfix issues. Just do your homework, Debian!