4 ms·

I'm curious how you'd account to factors like `npm audit` failing on probably most javascript repos out there on reasonably high CVSS settings due to including

I'm curious how you'd account to factors like `npm audit` failing on probably most javascript repos out there on reasonably high CVSS settings due to including items like https://nvd.nist.gov/vuln/detail/CVE-2021-3807 https://nvd.nist.gov/vuln/detail/CVE-2021-3807 ? And particularly, how to fix that, or Working As Intended?

pornel 4y ago
The security vendors have an ass-covering policy of scoring for the worst possible scenario, without any regard for how implausible it is. If you had a cupboard in your basement storing a can of spray that had a "warning: flammable" sticker with a weak glue: the sticker could fall off, you could have used it without knowing it's flammable, which could lead you to setting your house on fire. Therefore the CVSS score of "weak sticker glue" is exactly the same as "your house is on fire". This maximally inflated score is then combined with a primitive reporting policy that alerts about mere presence of dependencies with an advisory anywhere in your dependency tree. Security vendors aren't checking whether the dependencies are actually affected in this context or not. This is a lazy recipe for labelling everything as critically vulnerable all the time.
- jacques_chester 4y ago
  It's a double-whammy because not only do security vendors have an incentive to overstate scores, so do their researchers. I talk about it a little in the section on comparing scores over several years. One thing I didn't analyse that might be interesting would be to see if there's a consistent gap between security vendor / researcher reported scores in NVD and scores given to customers by their general software vendors.
  - lmeyerov 4y ago
    I keep coming back to something like collaborative filtering Raw context-unaware scores may come from researchers, but 'real' scores are based on deployment context. So a vuln may have diff final scores based on context, and the community needs ways identifying which bucket they fall into. Likewise, there is probably an 80/20 for most libs in terms of use case, so a good bet on default bucket of typical personal profile might help further decrease overhyping for raw scores. CI vs Server vs Browser vs TCB vs ... . Ex: npm audit reports on dev dependencies by default, but a regex complexity attack there is less legit, yet that's where most of the alerts seem to be. Instead of pushing to the user, can push a use-adjusted score to the reporting tool, and user can pick sensitivities for both. Collaborative filtering is one way, and others as well. Without something like thay, scores are largely ungrounded FUD / box checking.

pornel 4y ago

The security vendors have an ass-covering policy of scoring for the worst possible scenario, without any regard for how implausible it is. If you had a cupboard in your basement storing a can of spray that had a "warning: flammable" sticker with a weak glue: the sticker could fall off, you could have used it without knowing it's flammable, which could lead you to setting your house on fire. Therefore the CVSS score of "weak sticker glue" is exactly the same as "your house is on fire". This maximally inflated score is then combined with a primitive reporting policy that alerts about mere presence of dependencies with an advisory anywhere in your dependency tree. Security vendors aren't checking whether the dependencies are actually affected in this context or not. This is a lazy recipe for labelling everything as critically vulnerable all the time.

jacques_chester 4y ago
It's a double-whammy because not only do security vendors have an incentive to overstate scores, so do their researchers. I talk about it a little in the section on comparing scores over several years. One thing I didn't analyse that might be interesting would be to see if there's a consistent gap between security vendor / researcher reported scores in NVD and scores given to customers by their general software vendors.
- lmeyerov 4y ago
  I keep coming back to something like collaborative filtering Raw context-unaware scores may come from researchers, but 'real' scores are based on deployment context. So a vuln may have diff final scores based on context, and the community needs ways identifying which bucket they fall into. Likewise, there is probably an 80/20 for most libs in terms of use case, so a good bet on default bucket of typical personal profile might help further decrease overhyping for raw scores. CI vs Server vs Browser vs TCB vs ... . Ex: npm audit reports on dev dependencies by default, but a regex complexity attack there is less legit, yet that's where most of the alerts seem to be. Instead of pushing to the user, can push a use-adjusted score to the reporting tool, and user can pick sensitivities for both. Collaborative filtering is one way, and others as well. Without something like thay, scores are largely ungrounded FUD / box checking.