4 ms·

Note: sorry if I sound harsh, but these companies are raising millions to basically add a burden to the backs of hard-working OSS authors instead of helping the

Note: sorry if I sound harsh, but these companies are raising millions to basically add a burden to the backs of hard-working OSS authors instead of helping the ecosystem, it's really shameless IMHO. Positive note on the end! I've seen multiple known OSS authors speaking publicly unfavorably about these security scanners. It seems everyone is offering a "solution" to the vulnerability issue with Open Source, but they do it in a way that at best it's a reminder to update your dependencies (as noted in the article) and at worst it's a burden to OSS authors. Their interests are also very misaligned with everyone else in the field; best security practices are to communicate security issues with the author privately, have you or them make a fix with a reasonable timeline and then publish a fix for everyone (or reveal the issues publicly after X days). That'd help both authors and users, but unfortunately it doesn't help selling the security tools. Instead, due to this misalignment of incentives, the current player's workflow include bad practices like: - First time someone notices a potential vulnerability they will register it in their system, and then if you are lucky you'll get a public issue. Responsive disclosure sounds like "no publicity for us so nah". - Paint features as security vulnerabilities. Example: your "arbitrary command execution tool" can "execute arbitrary commands" or "your JSON reading tool can read JSON". Well, thanks, that's totally useless. - Tell users that your dev libraries, that never end up in production, have critical vulnerabilities. Sure if my CSS compilation tool has one it won't matter since it doesn't end up in the production server (unless it's a vuln. in the output). Edit: for a positive note, these are the current champions IMHO that are helping the ecosystem in this field (besides authors themselves ofc): - Github's free Actions for OSS. This has been amazingly helpful for testing for free and easily, and basically my turning point on believing that Microsoft has evolved for the better. - Hundreds or thousands of volunteer fellow OSS authors disclosing vulnerabilities privately. They are mostly anonymous, but def a huge force. - Companies releasing open source, since they are in a better position to offer big well-rounded libraries that reduce your dependency graph.

WrtCdEvrydy 6y ago
It's because the risk of a failure is high. If you end up not flagging something and it causes a vulnerability to become a data leak, you will end up getting blamed as a classic "noone could have seen this coming"
- aequitas 6y ago
  The risk of a failure is _sometimes_ high and this will lead to something called alert fatigue: https://en.wikipedia.org/wiki/Alarm_fatigue https://en.wikipedia.org/wiki/Alarm_fatigue
  - WrtCdEvrydy 6y ago
    The risk of failure to the scanner company is high. The risk of failure to you is low. That's the difference, they are shifting the burden of security to you... since if you don't patch the "exploit" which may not be an exploit, they can always point back to you in case there is a security breach because you didn't listen to the "magical scanner". It's a clear conflict of interest from twisted incentives. Additionally, if the thing never throws up any exploit warnings, you might wonder why you're paying so much for this fancy alerting thing.
    - aequitas 6y ago
      The value of a scanner with this kind of s/n ratio is very low. Because it would still cost me a lot time (thus money) to investigate every alert every time. What I would pay for is a scanner that is contextually aware and alerts when needed, give or take a few false positives. Not ones that have false positive as their default.
      - WrtCdEvrydy 6y ago
        Now you're getting it. Now think about the fact that most auditors want you running a scanner on your network (and it's required for certain legislation like GDPR)
      - px43 6y ago
        Sounds like what you're looking for is a scanner with more false negatives than false positives. One that won't alert you about a certain percentage of actual critical issues that it detects. You're not alone here, which is why things are so insecure. Enterprise security tools care mostly about catering to people like you who can't be bothered to investigate security issues. There are a totally different set of tools for those of us who actually want to find bugs, and the false positive rate (false:true ratio) on the best of those tools is more like 100:1. Often those tools have false positive rates of 1000:1 or even 10000:1, and people have built good machine learning models to sift through the findings automatically and sort them by likelihood of exploitability. It blows me away to hear people complain about the existence of false positives at ratios like 1:1. If only 50% of the findings in your tool are false positives, you are absolutely missing a ton of real bugs that will eventually be found by people who get paid to find real bugs.
px43 6y ago
> Paint features as security vulnerabilities. Example: your "arbitrary command execution tool" can "execute arbitrary commands" This is pretty funny to me. I live on the other side of this, and let me tell you, what you don't often hear, and what we legally can't tell you, is that there are dozens of financial institutions running your "command execution as a service" tool open to the internet with default admin creds because they think it's a simple uptime monitor. I have broken into multiple Fortune 500 companies using these sorts of bugs, and because I actually do want to live in a world where people can trust their technology, I do the right thing and report them. These bugs often get de-prioritized, or closed as WONTFIX because they're only applicable in really obscure situations that would never happen in the real world. My favorite example of this happening is when Homokov was trying to get mass assignment disabled by default, and his issues on GitHub kept getting closed because it was a feature, and only an idiot would screw up mass assignment, so he exploited a mass assignment bug in GitHub to create an issue from 1000 years in the future warning the past about letting mass assignment bugs persist. Since it was from the future, it was always on top of the issues list, serving as a constant reminder to the rails developers that their security posture is a joke.
- franciscop 6y ago
  Well and SQL injection is still #1 security vulnerability AFAIK. Do we mark any script that can run an arbitrary SQL query as "vulnerable"? Or educate the devs to not concatenate strings from untrusted inputs and instead either validate the text or use named parameters? If you want powerful tools these are also dangerous. Sure we should try to make our libraries as safe as possible and pick the safer version everything being the same, but everything is not the same. "there are dozens of financial institutions running your "command execution as a service" tool open to the internet with default admin creds because they think it's a simple uptime monitor" that's 9 out of 10 on the companies' side in my experience (except for the "default admin creds", which we all know should be prompted/created on the first run).
  - cratermoon 6y ago
    > educate the devs to not concatenate strings from untrusted inputs What if the dev is the CTO? https://arstechnica.com/gadgets/2021/03/rookie-coding-mistake-prior-to-gab-hack-came-from-sites-cto/ https://arstechnica.com/gadgets/2021/03/rookie-coding-mistak...

WrtCdEvrydy 6y ago

It's because the risk of a failure is high. If you end up not flagging something and it causes a vulnerability to become a data leak, you will end up getting blamed as a classic "noone could have seen this coming"

aequitas 6y ago
The risk of a failure is _sometimes_ high and this will lead to something called alert fatigue: https://en.wikipedia.org/wiki/Alarm_fatigue https://en.wikipedia.org/wiki/Alarm_fatigue
- WrtCdEvrydy 6y ago
  The risk of failure to the scanner company is high. The risk of failure to you is low. That's the difference, they are shifting the burden of security to you... since if you don't patch the "exploit" which may not be an exploit, they can always point back to you in case there is a security breach because you didn't listen to the "magical scanner". It's a clear conflict of interest from twisted incentives. Additionally, if the thing never throws up any exploit warnings, you might wonder why you're paying so much for this fancy alerting thing.
  - aequitas 6y ago
    The value of a scanner with this kind of s/n ratio is very low. Because it would still cost me a lot time (thus money) to investigate every alert every time. What I would pay for is a scanner that is contextually aware and alerts when needed, give or take a few false positives. Not ones that have false positive as their default.
    - WrtCdEvrydy 6y ago
      Now you're getting it. Now think about the fact that most auditors want you running a scanner on your network (and it's required for certain legislation like GDPR)
    - px43 6y ago
      Sounds like what you're looking for is a scanner with more false negatives than false positives. One that won't alert you about a certain percentage of actual critical issues that it detects. You're not alone here, which is why things are so insecure. Enterprise security tools care mostly about catering to people like you who can't be bothered to investigate security issues. There are a totally different set of tools for those of us who actually want to find bugs, and the false positive rate (false:true ratio) on the best of those tools is more like 100:1. Often those tools have false positive rates of 1000:1 or even 10000:1, and people have built good machine learning models to sift through the findings automatically and sort them by likelihood of exploitability. It blows me away to hear people complain about the existence of false positives at ratios like 1:1. If only 50% of the findings in your tool are false positives, you are absolutely missing a ton of real bugs that will eventually be found by people who get paid to find real bugs.

px43 6y ago

> Paint features as security vulnerabilities. Example: your "arbitrary command execution tool" can "execute arbitrary commands" This is pretty funny to me. I live on the other side of this, and let me tell you, what you don't often hear, and what we legally can't tell you, is that there are dozens of financial institutions running your "command execution as a service" tool open to the internet with default admin creds because they think it's a simple uptime monitor. I have broken into multiple Fortune 500 companies using these sorts of bugs, and because I actually do want to live in a world where people can trust their technology, I do the right thing and report them. These bugs often get de-prioritized, or closed as WONTFIX because they're only applicable in really obscure situations that would never happen in the real world. My favorite example of this happening is when Homokov was trying to get mass assignment disabled by default, and his issues on GitHub kept getting closed because it was a feature, and only an idiot would screw up mass assignment, so he exploited a mass assignment bug in GitHub to create an issue from 1000 years in the future warning the past about letting mass assignment bugs persist. Since it was from the future, it was always on top of the issues list, serving as a constant reminder to the rails developers that their security posture is a joke.

franciscop 6y ago
Well and SQL injection is still #1 security vulnerability AFAIK. Do we mark any script that can run an arbitrary SQL query as "vulnerable"? Or educate the devs to not concatenate strings from untrusted inputs and instead either validate the text or use named parameters? If you want powerful tools these are also dangerous. Sure we should try to make our libraries as safe as possible and pick the safer version everything being the same, but everything is not the same. "there are dozens of financial institutions running your "command execution as a service" tool open to the internet with default admin creds because they think it's a simple uptime monitor" that's 9 out of 10 on the companies' side in my experience (except for the "default admin creds", which we all know should be prompted/created on the first run).
- cratermoon 6y ago
  > educate the devs to not concatenate strings from untrusted inputs What if the dev is the CTO? https://arstechnica.com/gadgets/2021/03/rookie-coding-mistake-prior-to-gab-hack-came-from-sites-cto/ https://arstechnica.com/gadgets/2021/03/rookie-coding-mistak...