10 ms·

The Bogus CVE Problem

260 points by signa11 3y ago

mellosouls 3y ago
The main example in the article was discussed recently: CVE-2020-19909 is everything that is wrong with CVEs(74 comments) https://news.ycombinator.com/item?id=37267940 https://news.ycombinator.com/item?id=37267940 and Bogus CVE Follow-Ups (7 comments) https://news.ycombinator.com/item?id=37394919 https://news.ycombinator.com/item?id=37394919
rany_ 3y ago
If it weren't for CVEs it would be a lot more difficult to convince management to update the codebase and get rid of legacy crap even though I'm fully aware that almost all the CVEs don't apply to our project. It's just better for productivity/QOL. I think it's normal that CVEs don't convey all the nuance and at the end of the day, it's up to you to make an assessment on whether you're vulnerable. Though curl CVE seems like nonsense no matter how you look at it, I could see it might be an issue if delay was being set by untrusted user... but they could also set a very short delay without taking advantage of this flaw. So moot point.
- nikanj 3y ago
  The first paragraph is defending burning down the house every year or two, because that way you can change the curtains without pushback
  - lpapez 3y ago
    Following the analogy further personally I find a controlled house burn preferable to self-combusting hazard curtains.
    - zaphar 3y ago
      When the replacement house has a new random set of N self-combusting components where N>1 I'm not sure it's any safer.
      - lpapez 3y ago
        The important thing is to keep building the house all of the time, gotta pay the bills somehow :)
        TeMPOraL 3y ago
        Hell, SREs will tell you it's better this way - if you're regularly burning your house down and rebuilding it anew, there isn't time for maintenance problems to creep in!
  - remram 3y ago
    What part of the paragraph do you equate to burning a house down? The assignment of a CVE number?
    - lq9AJ8yrfs 3y ago
      Marking a defect as critical in typical large co control governance leads promptly toward re-prioritizing work and sometimes disabling otherwise-online and profit-generating systems. Some buyers insist on this in contracts with technology providers. That feels very much like burning cash, even if it's lost profits and not technically destroying a tangible object.
      - remram 3y ago
        Sounds like your company's governance is made entirely of pyromaniacs, I don't know why you feel a need to deflect the blame.
- bcassedy 3y ago
  That same management is the type to set rules where you don't get to assess if you're vulnerable, you just have to fix all the CVEs
  - HideousKojima 3y ago
    And where if you beg them to allow code review (especially for the code made by your incompetent offshore teams) they say it's too expensive/uses too much developer time, but then they'll pay a subscription for garbage static analysis tools that's enough to cover multiple full time dev salaries. Speaking purely hypothetically, of course.
cookiengineer 3y ago
The personal problems I have with the NVD and CVEs in general are: - CVEs can be disputed, and this is not based on evidence and can be marked as not affected with as little evidence as "EOL" or end-of-life. See 100% of CVEs filed against SAP, for example. All valid exploits, all work, all are used in the wild by ransomware, and all are still disputed by SAP. - CVEs are free-form text, which makes even version strings uncomparable. Most projects down the line don't "sanitize to ASCII" and don't know about this. But there are many CVEs (~1%ish) with Chinese idiom symbols as a "." character replacement. Nobody fixes that, ever. - CVEs and their affected hardware/software matching strings (CPEs) are utterly useless. They are an abstraction for the manual excel spreadsheet world, and literally every linux security tracker implements their own somewhat crappy parser to match against the real upstream projects, their package names and/or the source code origins (e.g. a git repo link). - CPEs are redundant. The linux kernel itself has more than 200 different CPE strings, and none of them are recorrected after they have been filed by the reporting person. What the actual duck. Have fun trying to find out if Arch Linux is affected when the initial CVE was filed against, say, SUSE or Oracle Linux. - Most hardware vendors wait to confirm or deny anything until their hardware is EOL. Smart, right? Nobody knows so it must be secure, right? Looking at you, Cisco, Fortinet and the like. And now you are wondering why exploits from 2016 are still used and shared in Telegram channels of ransomware developers? - There is no mandatory PoC in the CVE database. This could change so much in the ecosystem, especially with the hardforked still-patched feature frozen packages of distributions like Debian. Debian's security tracker is littered with "marked as fixed" but actually "can't fix it because code diverged too much from upstream already" CVEs. And guess what, those are exactly those exploits used in the wild by root/bootkits. - As the article author states, CVEs are never recorrected. This is the biggest issue of all of them. Nobody maintains a sane state of the database, even the new git repositories are buggy as hell and don't validate against their own schema. Nobody fixes those CVEs "upstream" and nobody maintains old CVEs that turn out to affect more upstream projects. Prime example is log4j, because literally almost all java software is affected when they bundle(d) it; which almost all of them do. I complained to this but basically can't or won't be fixed [2], [3], [4] because even the new schemas are experimental (for over 8 years now). Source: am scraping literally all linux security trackers to make a better CVE database for automatic comparison purposes. And it's a super ducked state of affairs. [1] [1] https://github.com/tholian-network/vulnerabilities https://github.com/tholian-network/vulnerabilities [2] https://github.com/CVEProject/cvelistV5/issues/4 https://github.com/CVEProject/cvelistV5/issues/4 [3] https://github.com/CVEProject/cvelistV5/issues/5 https://github.com/CVEProject/cvelistV5/issues/5 [4] https://github.com/CVEProject/cvelistV5/issues/6 https://github.com/CVEProject/cvelistV5/issues/6
nikanj 3y ago
Jackson had this problem a few months back, where someone reported a critical CVE against the project and broke builds all around the planet https://github.com/FasterXML/jackson-databind/issues/3972 https://github.com/FasterXML/jackson-databind/issues/3972 Basically the programmer (not the attacker) had to write code where an object contained itself HashMap<String,Object> map=new HashMap<>(); map.put("recursive",map); After this, Jackson would indeed stack overflow if you asked it to wrap the object to JSON. Then again, half the build-in Java functions (e.g. getting an object hashcode for the map object) also fail for a recursive structure. The issue remains open 3 months later, Mitre still thinks it's hella serious, and people have yet again learned to just ignore their CI warning about CVEs
mholt 3y ago
This resonates. Unfortunately, the CVE database(s) are too noisy to be useful. It could benefit from higher standards and more thorough vetting. (Maybe take some lessons from academia.) A "security researcher" once filed a CVE for a regular bug in Caddy [0], making claims that were totally provably false. It was assigned 7.5... the same as Heartbleed [1] -- yes, the one that leaked almost all the private encryption keys on the Internet back in 2014. When I appealed to NVD for retraction (or whatever they do in this case), I never heard back, despite several emails / form submissions. It is not well-maintained. It is poorly managed. There is virtually no oversight. NVD is a trainwreck. More recently I inadvertently discovered a 0-day RCE in acme.sh [2]. (ACME clients are security-sensitive contexts since they typically deal with private keys and download signed credentials.) Anyway, it was assigned a CVSS 3.x score of * 9.8 * [3] -- I imagine that should be like "cyber-nuclear meltdown" territory, but no, this was actually benign as far as we can tell. Probably deserves more like a 4 or 5 or something. So, again: There is virtually no oversight. NVD is a trainwreck. Anyway, the whole system is broken, and I'm effectively ignoring CVEs now. But if someone just tells me to patch my <whatever>, I'll probably do it, and that's good enough for me. [0]: https://github.com/caddyserver/caddy/issues/4775 https://github.com/caddyserver/caddy/issues/4775 [1]: https://nvd.nist.gov/vuln/detail/cve-2014-0160 https://nvd.nist.gov/vuln/detail/cve-2014-0160 [2]: https://matt.life/writing/the-acme-protocol-in-practice-and-reality https://matt.life/writing/the-acme-protocol-in-practice-and-... [3]: https://nvd.nist.gov/vuln/detail/CVE-2023-38198 https://nvd.nist.gov/vuln/detail/CVE-2023-38198
- jabroni_salad 3y ago
  Honestly if you just stick with CISA's known exploited list then you will be ahead of most operators and spend less time ghost hunting stupid shit. I really have come to despise CVSS (and especially how tenable will put predictions in the CVSS column for officially unrated vulns) and how assessors seemingly so not have to justify their 'findings'. Not exploited, no poc, needs root to use, why is this a High? Come back to me with something good.
  - 1MachineElf 3y ago
    CISA-KEV is a good list, but updates to it can be delayed by days to months.
_8j50 3y ago
I don't know if you all know this but you don't have to fix CVEs. If the software author won't issue a fix, you are supposed to have an exemption process to accept the non-existent risk. In my view, the CVE system is too strict, it needs more noise and low severity vulns that are hard to pull off in the real world. It showd the scrutiny the software has gotten and seeing a lot of such vulns shows the software has good and secure design. Contrary to public sentiment, not having CVEs does not mean the software is secure, it most likely means (and you should presume) it hasn't gotten proper scrutiny. The main issue I have seen is how the scoring can be 9 or 10 without a working exploit. It's a database to help people make decisions not a compliance checklist. You should not misuse the databse alone to measure the how vulnerable you are, you should also be considering risk and impact as well as how fixable it is. But using it as a beating stick is far too easy.
- politelemon 3y ago
  > you don't have to fix CVEs. Goodhart's law is in effect. Large corporates only see that the cve exists and has a large number against it. There is no qualitative reading or investigation done. The number is big, therefore you have to fix it... (only helping perpetuate the issue).
mdmglr 3y ago
The bogus CVE problem has caused delays in my projects because the CIO wants our COTS scanner tool reports to have 0 CVE's or a detailed explanation on why it is not an issue. Also I'm having difficulty communicating: CVSS is not a measure of risk, and that many of the ReDoS vulns are very much dependent on the context.
- worthless-trash 3y ago
  Wouldn't the full score (which includes your environment context) be a measure of risk ?
tptacek 3y ago
So, uh, what's the problem? CVEs are just meant to ensure that we're all talking about the same bug when we're talking about a bug. If you're making a big deal about a CVE existing for a bug you don't care about... that's on you?
- hannob 3y ago
  This is the problem: "Service-level contracts and governmental requirements mean that a critical CVE needs to be addressed in short order, so non-critical bugs that get marked that way can cause real problems." You are right what CVEs are, but there are plenty of people who think CVEs are something different.
  - thephyber 3y ago
    The CVEs aren’t the problem. You are describing the problems related to Vogons who assume all bureaucratic inputs are perfect. All tickets should have an escape hatch where some person with direct experience with the software can report false positives or “not an issue”. Those are different than the purpose of the article, which is describing the variance of quality of CVE data.
  - waihtis 3y ago
    Not to mention the completely trash cyber risk ratings industry. If you score poorly in because of their absurd scoring methods, it will potentially impact revenue streams if you have to deal with vendor risk departments, etc who don't understand any of the underlying mechanics
  - mxmlnkn 3y ago
    Seems like someone has to file a CVE regarding CVE scores leading to DOS attacks on the developers.
  - tptacek 3y ago
    The worst possible response to that problem would be to run the CVE program in a way that ratifies it going forward!
- 3y ago
lars_francke 3y ago
Yes, the CVE database/process is broken. Step 1 would be to not treat vulnerability == CVE. It's a simple change in terminology but we need to make sure to communicate that CVEs are just one, probably the most popular, but just one source of vulnerabilities. Governance and security teams the world over need to learn this. Next step ist to establish good tooling around publishing VEX statements (Vulnerability Exploitability Exchange). That is currently not easy. In the EU we currently have the discussions about the upcoming Cyber Resilience Act which has language in it that says products must be "free of known vulnerabilities" to be allowed to be placed on the European market. We're trying to change that language but it ain't trivial. The current best proposal is saying "known _exploited_ vulnerabilities". Otherwise I'd be able to DoS my competitors by publishing bogus vulnerabilities. For that we have at least two sources: CISA KEV[1] (which just celebrated its thousands entry) and FIRST EPSS[2]. There might be more, these are the ones I'm aware of. We are a software vendor and we want to do the best we can but at the moment it's a box ticking exercise that's not useful to anyone. We want to look at all vulnerabilities but we want to focus on the important ones. We need to be able to say "our product is not affected by this vulnerability" and we need our customers to trust us. Currently, they often trust the CVE/NVD database more which is a huge problem because they are not experts in the specific products. And especially for things like libraries you need to take the context into account in which it is used. Vulnerabilities are reported (using CPE, pURL etc.) at the "library" or "application" level but they really exist at a much more granular level (e.g. a single function is affected and if that's not used there's no problem). I'm convinced that the whole space of vulnerability disclosure and management will change significantly over the next few years. We happened to stumble into this and are now active in trying to shape the Cyber Resilience Act into a form that makes more sense because that will force basically every European company to now start this process. Other countries will follow (yes I know the US already has rules for some sectors but not everything, so do other countries). [1] <https://www.cisa.gov/known-exploited-vulnerabilities-catalog https://www.cisa.gov/known-exploited-vulnerabilities-catalog> [2] <https://www.first.org/epss/model https://www.first.org/epss/model>
- 1MachineElf 3y ago
  Do you have any thoughts on CVSSv4[0]? It appears to incorporate finer-grained and organization-specific scoring to address issues many have with the one size fits all approach currently used for CVEs. [0] https://www.first.org/cvss/v4-0/ https://www.first.org/cvss/v4-0/
eslerm 3y ago
Bogus CNAs worry me more.
hsbauauvhabzb 3y ago
From the article “The report was for a legitimate bug, where the ‑‑retry‑delay option value was being multiplied by 1000 (to milliseconds) without an overflow check. But what it was not was a security bug, Stenberg said; giving insanely large values for the option might result in incorrect delays—far shorter than requested—but it is not a security problem to make multiple requests in a short time span. If it were, "then a browser makes a DOS [denial of service] every time you visit a website — and curl does it when you give it two URLs on the same command line", he said in a followup post.” I agree there’s plenty of junk in the cve database but devils advocate: ‘a system relies on a user supplied sleep to trigger a web request which executes some function which if execute too quickly may cause double transaction execution or similar. Almost any functional bug (and arguably some UI bugs) _can_ have security or real world impacts under some scenario. CVE cannot generically say that any bug does not have security implications, noting that not all security bugs require a threat actor. A data breach is bad, regardless of if it’s a hack or if it’s somebody accidentally leaving a pile of printed PII on a bus, even if most piles of paper left on busses are just trash.
- blueflow 3y ago
  > may cause double transaction execution or similar. From curl(1): --retry‐delay <seconds> Make curl sleep this amount of time before each retry when a transfer has failed with a transient error How would a retry at the wrong time cause a double transaction? I doubt this would be a bug on the curl side.
  - hsbauauvhabzb 3y ago
    How many unix tools have idiosyncratic behaviours to prevent breaking obscure use cases? If there’s not a situation which exists where that bug may be a security issue, I could hack one up just so there is.
    - blueflow 3y ago
      Then your "situation" should get the CVE. It is indirectly relying on transient errors (like DNS resolution) not happening.
junon 3y ago
Can relate. Dealing with "ReDOS" (regex DOS) CVEs used solely to win bounties from certain sites, paired with overly hostile and secretive "researchers", has turned me off from the whole CVE system entirely.
waihtis 3y ago
Outside of regulation, we also have the dumpster file called cyber risk ratings pulling more and more leverage in vendor risk assessments; good luck arguing with an enterprise vendor risk department why having 100 9.8 CVSS score vulnerabilities actually don't impact anything
- kei62sii 3y ago
  I work on this exact problem for a cybersecurity company, and it’s super challenging 1) because it’s like trying to assign a probability to an earthquake (how do you validate your model with such few ground truth events?) and 2) like you mention, the more accurate we make the model, the more pushback there is as people begin to lose trust when it doesn’t match their intuition. What’s a good way to make a convincing case of “trust us, we know this risk assessment doesn’t look intuitive but it’s better than the common XYZ approach because…”? Or is this a losing battle, and we should just allow customers to rank by CVE severities if that’s what they really want to do? Some smaller startups are starting to work toward providing cyber VaR — quantify the probabilistic loss in financial terms over a given time period. This is really the only direction forward in cybersecurity IMO as risk must ultimately be modeled with statistics, not hunches. But I don’t think the industry is ready for it (and neither are the models to be frank).
  - waihtis 3y ago
    There may be nothing novel here for you given you're already working in the problem area, but as established a CVE measure is completely hypothetical on its own; if your customers insist on looking at one variable (CVE), you could always make the case that CVE is incomplete on it's own and should always be supplemented with data such as: a) is it actually exploitable b) is someone exploiting it in the wild c) accessibility of component/asset/application/whatever that carries the CVE (e.g. is it internet-facing or a non-networked subcomponent hidden under 15 layers of defense) d) other exploitability scoring methods, like EPSS e) etc... (sometimes I like to use the analogue of trying to gauge your body's health based on one variable, like your body temperature - it can be high temporarily [in case of a flu] but it doesn't tell you anything about your overall health [as we all get sick occasionally]) The VaR approach is sensible but has to start from much further down in the root of the problem by first creating a value-based catalogue of all your assets, after which you should simulate all prospective attack paths towards all assets (maybe above a certain value threshold to simplify the model a bit), then overlay with attack / exploit probability data (simple example would be a CVE dataset + EPSS scoring) and finally you have something resemblant of an actual data-based risk model you can quantify impacts on. It's quite a large task and I don't think there's a single player doing all areas, you'd have to patchwork together multiple different datasets. Still, it's probably where the industry will end up in the next 10-15 year timespan. (I'm somewhat involved in the area too but just as a threat data provider for larger models.)
xvilka 3y ago
For anyone working with CVE and other advisories there's a cool search engine and feed for those - Vulners[1]. They have integration with various providers and tools, as well as allow to search also for exploits among numerous blogs. [1] https://vulners.com https://vulners.com
nonrandomstring 3y ago
General wisdom is: error < bug < vuln < poc < wild exploit Yet a narrow security view cantered around single faults within single programs misses a lot. Take backdoors, which are neither errors nor bugs, nor may be seen in the wild, but are simply vulnerabilities in waiting, working as expected by design. Or a set of mere errors may, in concert, lead to a systemic and commonly exploited vulnerability. Giving an identifier to a reproducible observation, in some context and time, is a good start. But to be useful it needs to be part of a wider, joined-up taxonomy that helps us understand risk better.
LudwigNagasena 3y ago
That reminds me of bogus `npm` results. You use a regex at build time? Oh, apparently you are exposed to a ReDoS attack even though that regex isn’t available at runtime. Or that time when Python disallowed string to int conversion of long strings because it is a DoS vulnerability. The lack of sanitisation led to DoS vulnerabilities in applications, but the conversion is obviously not a vulnerability in itself. Oh, and now those apps have DoS attack that just halts the application because the conversion throws ValueError. It’s hard to say what the solution to that problem is. Oftentimes security vulnerabilities indeed don’t cause any problems alone and require a chain of other exploits to actually cause trouble. At the same time you don’t want to treat every function that runs in O(n^2) on input as a potential DoS attack. Distinguishing between a simple bug and a vulnerability requires contextual knowledge and expert judgement that simply cannot be built into bureaucratic organisations like MITRE.
- stefan_ 3y ago
  This is my favorite lately. Sure, some insane people use these packages for backend services, but we are just building a big JS blob that's statically shipped to the browser; none of the "vulnerabilities" in them apply. But try getting this across to people who think they need to track security issues they don't understand (not fix them, of course).
- derstander 3y ago
  > Distinguishing between a simple bug and a vulnerability requires contextual knowledge and expert judgement that simply cannot be built into bureaucratic organisations like MITRE. Why can’t it? MITRE runs FFRDCs. Having worked for an FFRDC (although in electrical engineering, not computer security), if you were to tell me that engineers in FFRDCs or UARCs in my domain simply cannot utilize contextual knowledge or expert judgment (or that their knowledge and judgment would be completely overridden by management such that it appears they can’t) then I would wonder if you ever interacted with that kind of organization. Is computer security somehow different to e.g. radar or comms systems engineering?
  - LudwigNagasena 3y ago
baby 3y ago
Everything about CVEs when I used them screamed “old/legacy stuff” from the processes to the tooling.
kbrkbr 3y ago
In my company we have moved to using EPSS (exploit probability scoring system) on top of CVE's. That filters a lot of noise. https://www.first.org/epss/ https://www.first.org/epss/
jcarrano 3y ago
As I mentioned a couple days ago [1], this problem will get much worse in the EU with the upcoming CRA. Either the only software vendors left will be big ones with the capacity and leverage to counter these CVEs, or companies will turn to obscure, bug-ridden software which have no CVEs because no one bothers to research them. [1] https://news.ycombinator.com/item?id=37582808 https://news.ycombinator.com/item?id=37582808
- 112233 3y ago
  expect closed-source analog of GNU, and a license that forbids showing source code, plus tools for obfuscating binaries, masked as "optimizers". there will be ademand for anything that reduces chance of CVE discovery and that reduces open source SBOM entries. open source most likely becomes a liability under CRA
  - jcarrano 3y ago
    Yes, and the next step is that the vendor will sue researchers for posting CVEs which they consider "bogus", accusing them of hurting their business. It's not so much as open source becomes a liability, but rather "well known", whether open or not, becomes a liability. If you pull a random json library from github that does not even appear on any vulnerability database then you will probably be fine.
    - delfinom 3y ago
      Shit, I think the best way to fight this is to file bogus CVEs against propertiary software vendors. You'll see the industry demand CVEs be fixed within a month. You can after-all, file CVEs anonymously. It takes no effort to stand up a fake "research firm", or two, or three doing so. Bonus points if the CVEs are against all the shitty cybersec scan tools.
      - msm_ 3y ago
        You can file CVEs anonymously, but CNA don't have to assign a CVE number if they consider submission bogus. Depending on the CNA they may also verify the submission themselves or contact the vendor before assignment (usually vendor has to confirm the vulnerability before the publication, often CVE publication date is decided together with the vendor if the vulnerability is not already public). Of course there are shitty CNAs that don't care. But honestly CVEs are a useful tool for communication, people (both security people and non-security people) should stop obsessing over them.
tyingq 3y ago
There's lots of common sense missing in how CVEs are used at ground level in a company also. Like, say, I use node.js in my project solely in the build/deploy process, and not at all in the deployed code. Something like using the serverless framework to make python AWS Lambdas. The processes in place often can't make that distinction, so some "high /critical node.js when used as a web server" problem halts all builds, even though there's no node.js in the deployed application, and node.js is only used cli style.
- robszumski 3y ago
  This is why having an accurate scope is key. Easiest way to do that in an hour is to prefer to scan a workload like a container instead of a code repo. Or have a scanner that understands how something is built (and ideally how it runs).
petesergeant 3y ago
> Various CNAs assign CVE numbers for their own products (e.g., Microsoft, Oracle, HP, Red Hat) I wonder if this is part of the problem. The big players aren't hurt by this problem because they get to decide what's a real CVE for themselves, so there's little commercial push for change.
oooyay 3y ago
It's awful that a system meant for the benefit of maintainers and users is now being perverted into such theatrics. In a way, if you want to undermine the concept of security this is one of the ways you can do it. On the user side of this token I experience similar theatrics. Monthly I have to deal with a list of vulnerabilities that are detected within our software. Some are easy fixes, like updating a package manager which equates to just rerunning a pipeline. Others are not, and I found myself recently trying to rebuild an entire container build process because the software is on a biannual release process and they'd used the x/crypto package in Go which contains an SSH client and server. The server was vulnerable and the software we were using made use of the client, but not the server. Regardless our software was flagged with a high vulnerability. The utilities available are smart enough to take apart my Go binary during container inspection but not smart enough to figure out if I actually use the vulnerable thing. The resulting toil and theatrics aren't just a time waster, they're a soul sucking activity laiden with difficult tasks to achieve minor outcomes like, "How do I make sure we miss the possibility of even irrelevant CVEs showing up in the first place" - which is probably not what you want.
rascul 3y ago
Greg Kroah-Hartman (the kernel developer/maintainer) in 2019 gave a presentation about some of the issues with CVEs. https://www.youtube.com/watch?v=HeeoTE9jLjM https://www.youtube.com/watch?v=HeeoTE9jLjM https://github.com/gregkh/presentation-cve-is-dead https://github.com/gregkh/presentation-cve-is-dead
skissane 3y ago
Here’s an example of a bogus CVE in the GNU C Library: https://nvd.nist.gov/vuln/detail/CVE-2023-0687 https://nvd.nist.gov/vuln/detail/CVE-2023-0687 That CVE is correct that the “gmon” component of glibc contains memory corruption bugs. In fact, I managed to find even more than the specific one that CVE is about. But, the claim that this is a security vulnerability is a bit silly. These are profiler functions, which are usually only called in non-production builds with profiling enabled (gcc -pg), and even then only from CRT startup code. It is rather unlikely an attacker can exploit any of these bugs in them without already having the ability to run arbitrary code in the process, at which point they don’t need these functions, they’ve already got all these bugs have to give them.
bcoughlan 3y ago
The solution at my workplace is a bot that opens PRs to bump dependencies and automatically merges if the tests pass. It's taken a lot of workload off devs to meet security targets. But I worry it makes supply-chain attacks more attractive. If an attacker can compromise a package and it's instantly merged into the codebases of thousands of different companies that's a huge danger.
- ashishbijlani 3y ago
  I've been building Packj [1] to detect dummy, malicious, abandoned, typo-squatting, and other "risky" PyPI/NPM/Ruby/PHP/Maven/Rust packages. It carries out static/dynamic/metadata analysis and scans for 40+ attributes such as num funcs/files, spawning of shell, use of SSH keys, network communication, use of decode+eval, etc. to flag risky packages. Packj Github action [2] can alert if a risky dependency is pulled into your build. 1. https://github.com/ossillate-inc/packj https://github.com/ossillate-inc/packj 2. https://github.com/ossillate-inc/packj-github-action https://github.com/ossillate-inc/packj-github-action
Aachen 3y ago
Well that's a poor article. Not used to that from LWN at all. I think this sentence sums up most of its problems: > In fact, the pull request for the fix was attached to the report, but that apparently made little difference in the assessment from NVD. - CVEs are about vulnerabilities, not fixes. If everyone updated their software and all software updated its dependencies, I'm not sure we needed a vulnerability tracking system at all! The fix being available doesn't change the impact or exploitability of the bug while you run the out-of-date software - The text makes it sound like ratings given are opinions (elsewhere it talks about NVD having a "change of heart"). NVD uses a calculation system called CVSS where you input parameters like whether the vulnerable component can be reached over a network, whether the vulnerability results in denial of service or in confidentiality being breached or whatnot. These are objective parameters, and while interpretation mistakes are extremely common¹, ending up with a vastly different score is rather difficult if you have all relevant information to fill it out in the first place (also because mistakes are likely to cancel out). The article acts as though it's their subjective assessment, but rather, the score changes with new facts being presented. (In an ideal world, of course. In practice they typically forget or aren't told the correct info. But apparently here they were told and did update, so that's good right?) The article then goes on to state that curl uses its own subjective system now instead and they gave some bug an entirely different rating. The article does not say anything towards who had the more correct result but then goes with the assumption that curl's own subjective rating is better. Is that really what we want, when complaining that NVD gives bad ratings? More subjectivity, intransparency about how they got to a particular result? Anyway I'm neither loving the CVE system nor CVSS ratings either, but these are the wrong reasons ¹ http://export.arxiv.org/abs/2308.15259 http://export.arxiv.org/abs/2308.15259 recent study into CVSS scoring inconsistencies
userbinator 3y ago
Service-level contracts and governmental requirements mean that a critical CVE needs to be addressed in short order, so non-critical bugs that get marked that way can cause real problems. Bureaucracy destroys common sense once again. If every developer opposes doing anything with bogus CVEs and calling them out for the BS they are, with a detailed explanation of why, then we might get some changes, but unfortunately just mentioning anything "security" has gotten such a paranoic response from most of the population that it's a difficult battle.
donatj 3y ago
I feel like there are a lot of CVE's in the JS community that basically require an attacker to already have access to server side code execution. That seems like it raises some bigger questions.
insanitybit 3y ago
It's funny. I was just learning about RuneWatch, which is how the Runescape community attempts to democratize and distribute the problem of dealing with scammers. The evolution was basically that it started the way CVEs work. You could submit a report and there wasn't really any verification. Of course, this was abused - far worse than the CVE system, naturally, but the same sort of problem. Now they require a lot of evidence and things are much much better and abuse is far more difficult. It seems sort of obvious. If this is a vulnerability, prove it. It almost feels like a CVE should have a "practical" tag. Like: 1. Is this theoretically exploitable? 2. Is this practically exploitable? 3. Is this proven exploitable? This should be a massive contributor to the score. It also means that a score can be raised by having the sec researcher demonstrate an exploit - so disagreements can be resolved very easily. Related, here's a post from Spender on the topic (since the "CVE bad" situation is nothing new) https://grsecurity.net/reports_of_cves_death_greatly_exaggerated https://grsecurity.net/reports_of_cves_death_greatly_exagger...
wnevets 3y ago
CVEs became a target for people's careers and wallet. I'm not surprised it's turning into a mess.
testplzignore 3y ago
My all-time most hated CVE is CVE-2022-24975. This is not a vulnerability - it's a user education issue with not understanding how Git works. The Git documentation not explicitly explaining the dangers of using mirror was rated a * 7.5 *. Due to the policy of having no high CVEs, this ground development at my company to a halt for an entire day until our security team put an exemption in place.
AtNightWeCode 3y ago
It is not just a CVE problem. Many sec standards feels really random and difficult to work with. You can get better scores on some of these sec scans if you open up some ports in the firewall.
memorysafety 3y ago
CVE scoring is parasocial activity. Hence so much drama. Similarly to SemVer, the good-faith grader attempts to convey a sizeable blob of knowledge... by compressing it into a one-dimensional number. No matter the scoring formula, this step is lossy. On the receiving end of this communication, all you can do with the score is add a huge grain of salt to it, then perhaps use to prioritize your review queue. You still must check the details, and work out a judgement tailored to your specific context. There's no other way. There isn't a choice for the grader either, to skip the obscenely lossy scoring step. Just like with release versions, they must do it, as the audience consists of unbounded number of engineers; faithfully doing it saves mountains of time for everyone involved (present and future). Just like with dependency upgrades, it's the consumer's choice to disregard CVE scores (version numbers), vulnDB entries (changelogs), or even existence itself of a vulnerability (upgrade). Likewise, it's their fault if consequences arise. Viewed thusly, can be seen: anecdotes of pointwise drama will continue (even when the bulk of activity chugs along happily, efficiently and quietly) -- because at the core of it, CVE ID's and scores are just that, a communication tool. It mostly can't make strangers exercise care or spend effort more than they're willing to. It can optimise utility of attention that they do pay.

260 points by signa11 3y ago

The main example in the article was discussed recently: CVE-2020-19909 is everything that is wrong with CVEs(74 comments) https://news.ycombinator.com/item?id=37267940 https://news.ycombinator.com/item?id=37267940 and Bogus CVE Follow-Ups (7 comments) https://news.ycombinator.com/item?id=37394919 https://news.ycombinator.com/item?id=37394919

rany_ 3y ago

If it weren't for CVEs it would be a lot more difficult to convince management to update the codebase and get rid of legacy crap even though I'm fully aware that almost all the CVEs don't apply to our project. It's just better for productivity/QOL. I think it's normal that CVEs don't convey all the nuance and at the end of the day, it's up to you to make an assessment on whether you're vulnerable. Though curl CVE seems like nonsense no matter how you look at it, I could see it might be an issue if delay was being set by untrusted user... but they could also set a very short delay without taking advantage of this flaw. So moot point.

nikanj 3y ago
The first paragraph is defending burning down the house every year or two, because that way you can change the curtains without pushback
- lpapez 3y ago
  Following the analogy further personally I find a controlled house burn preferable to self-combusting hazard curtains.
  - zaphar 3y ago
    When the replacement house has a new random set of N self-combusting components where N>1 I'm not sure it's any safer.
    - lpapez 3y ago
      The important thing is to keep building the house all of the time, gotta pay the bills somehow :)
      - TeMPOraL 3y ago
        Hell, SREs will tell you it's better this way - if you're regularly burning your house down and rebuilding it anew, there isn't time for maintenance problems to creep in!
- remram 3y ago
  What part of the paragraph do you equate to burning a house down? The assignment of a CVE number?
  - lq9AJ8yrfs 3y ago
    Marking a defect as critical in typical large co control governance leads promptly toward re-prioritizing work and sometimes disabling otherwise-online and profit-generating systems. Some buyers insist on this in contracts with technology providers. That feels very much like burning cash, even if it's lost profits and not technically destroying a tangible object.
    - remram 3y ago
      Sounds like your company's governance is made entirely of pyromaniacs, I don't know why you feel a need to deflect the blame.
bcassedy 3y ago
That same management is the type to set rules where you don't get to assess if you're vulnerable, you just have to fix all the CVEs
- HideousKojima 3y ago
  And where if you beg them to allow code review (especially for the code made by your incompetent offshore teams) they say it's too expensive/uses too much developer time, but then they'll pay a subscription for garbage static analysis tools that's enough to cover multiple full time dev salaries. Speaking purely hypothetically, of course.

cookiengineer 3y ago

The personal problems I have with the NVD and CVEs in general are: - CVEs can be disputed, and this is not based on evidence and can be marked as not affected with as little evidence as "EOL" or end-of-life. See 100% of CVEs filed against SAP, for example. All valid exploits, all work, all are used in the wild by ransomware, and all are still disputed by SAP. - CVEs are free-form text, which makes even version strings uncomparable. Most projects down the line don't "sanitize to ASCII" and don't know about this. But there are many CVEs (~1%ish) with Chinese idiom symbols as a "." character replacement. Nobody fixes that, ever. - CVEs and their affected hardware/software matching strings (CPEs) are utterly useless. They are an abstraction for the manual excel spreadsheet world, and literally every linux security tracker implements their own somewhat crappy parser to match against the real upstream projects, their package names and/or the source code origins (e.g. a git repo link). - CPEs are redundant. The linux kernel itself has more than 200 different CPE strings, and none of them are recorrected after they have been filed by the reporting person. What the actual duck. Have fun trying to find out if Arch Linux is affected when the initial CVE was filed against, say, SUSE or Oracle Linux. - Most hardware vendors wait to confirm or deny anything until their hardware is EOL. Smart, right? Nobody knows so it must be secure, right? Looking at you, Cisco, Fortinet and the like. And now you are wondering why exploits from 2016 are still used and shared in Telegram channels of ransomware developers? - There is no mandatory PoC in the CVE database. This could change so much in the ecosystem, especially with the hardforked still-patched feature frozen packages of distributions like Debian. Debian's security tracker is littered with "marked as fixed" but actually "can't fix it because code diverged too much from upstream already" CVEs. And guess what, those are exactly those exploits used in the wild by root/bootkits. - As the article author states, CVEs are never recorrected. This is the biggest issue of all of them. Nobody maintains a sane state of the database, even the new git repositories are buggy as hell and don't validate against their own schema. Nobody fixes those CVEs "upstream" and nobody maintains old CVEs that turn out to affect more upstream projects. Prime example is log4j, because literally almost all java software is affected when they bundle(d) it; which almost all of them do. I complained to this but basically can't or won't be fixed [2], [3], [4] because even the new schemas are experimental (for over 8 years now). Source: am scraping literally all linux security trackers to make a better CVE database for automatic comparison purposes. And it's a super ducked state of affairs. [1] [1] https://github.com/tholian-network/vulnerabilities https://github.com/tholian-network/vulnerabilities [2] https://github.com/CVEProject/cvelistV5/issues/4 https://github.com/CVEProject/cvelistV5/issues/4 [3] https://github.com/CVEProject/cvelistV5/issues/5 https://github.com/CVEProject/cvelistV5/issues/5 [4] https://github.com/CVEProject/cvelistV5/issues/6 https://github.com/CVEProject/cvelistV5/issues/6

nikanj 3y ago

Jackson had this problem a few months back, where someone reported a critical CVE against the project and broke builds all around the planet https://github.com/FasterXML/jackson-databind/issues/3972 https://github.com/FasterXML/jackson-databind/issues/3972 Basically the programmer (not the attacker) had to write code where an object contained itself HashMap<String,Object> map=new HashMap<>(); map.put("recursive",map); After this, Jackson would indeed stack overflow if you asked it to wrap the object to JSON. Then again, half the build-in Java functions (e.g. getting an object hashcode for the map object) also fail for a recursive structure. The issue remains open 3 months later, Mitre still thinks it's hella serious, and people have yet again learned to just ignore their CI warning about CVEs

mholt 3y ago

This resonates. Unfortunately, the CVE database(s) are too noisy to be useful. It could benefit from higher standards and more thorough vetting. (Maybe take some lessons from academia.) A "security researcher" once filed a CVE for a regular bug in Caddy [0], making claims that were totally provably false. It was assigned 7.5... the same as Heartbleed [1] -- yes, the one that leaked almost all the private encryption keys on the Internet back in 2014. When I appealed to NVD for retraction (or whatever they do in this case), I never heard back, despite several emails / form submissions. It is not well-maintained. It is poorly managed. There is virtually no oversight. NVD is a trainwreck. More recently I inadvertently discovered a 0-day RCE in acme.sh [2]. (ACME clients are security-sensitive contexts since they typically deal with private keys and download signed credentials.) Anyway, it was assigned a CVSS 3.x score of * 9.8 * [3] -- I imagine that should be like "cyber-nuclear meltdown" territory, but no, this was actually benign as far as we can tell. Probably deserves more like a 4 or 5 or something. So, again: There is virtually no oversight. NVD is a trainwreck. Anyway, the whole system is broken, and I'm effectively ignoring CVEs now. But if someone just tells me to patch my <whatever>, I'll probably do it, and that's good enough for me. [0]: https://github.com/caddyserver/caddy/issues/4775 https://github.com/caddyserver/caddy/issues/4775 [1]: https://nvd.nist.gov/vuln/detail/cve-2014-0160 https://nvd.nist.gov/vuln/detail/cve-2014-0160 [2]: https://matt.life/writing/the-acme-protocol-in-practice-and-reality https://matt.life/writing/the-acme-protocol-in-practice-and-... [3]: https://nvd.nist.gov/vuln/detail/CVE-2023-38198 https://nvd.nist.gov/vuln/detail/CVE-2023-38198

jabroni_salad 3y ago
Honestly if you just stick with CISA's known exploited list then you will be ahead of most operators and spend less time ghost hunting stupid shit. I really have come to despise CVSS (and especially how tenable will put predictions in the CVSS column for officially unrated vulns) and how assessors seemingly so not have to justify their 'findings'. Not exploited, no poc, needs root to use, why is this a High? Come back to me with something good.
- 1MachineElf 3y ago
  CISA-KEV is a good list, but updates to it can be delayed by days to months.

_8j50 3y ago

I don't know if you all know this but you don't have to fix CVEs. If the software author won't issue a fix, you are supposed to have an exemption process to accept the non-existent risk. In my view, the CVE system is too strict, it needs more noise and low severity vulns that are hard to pull off in the real world. It showd the scrutiny the software has gotten and seeing a lot of such vulns shows the software has good and secure design. Contrary to public sentiment, not having CVEs does not mean the software is secure, it most likely means (and you should presume) it hasn't gotten proper scrutiny. The main issue I have seen is how the scoring can be 9 or 10 without a working exploit. It's a database to help people make decisions not a compliance checklist. You should not misuse the databse alone to measure the how vulnerable you are, you should also be considering risk and impact as well as how fixable it is. But using it as a beating stick is far too easy.

politelemon 3y ago
> you don't have to fix CVEs. Goodhart's law is in effect. Large corporates only see that the cve exists and has a large number against it. There is no qualitative reading or investigation done. The number is big, therefore you have to fix it... (only helping perpetuate the issue).

mdmglr 3y ago

The bogus CVE problem has caused delays in my projects because the CIO wants our COTS scanner tool reports to have 0 CVE's or a detailed explanation on why it is not an issue. Also I'm having difficulty communicating: CVSS is not a measure of risk, and that many of the ReDoS vulns are very much dependent on the context.

worthless-trash 3y ago
Wouldn't the full score (which includes your environment context) be a measure of risk ?

tptacek 3y ago

So, uh, what's the problem? CVEs are just meant to ensure that we're all talking about the same bug when we're talking about a bug. If you're making a big deal about a CVE existing for a bug you don't care about... that's on you?

hannob 3y ago
This is the problem: "Service-level contracts and governmental requirements mean that a critical CVE needs to be addressed in short order, so non-critical bugs that get marked that way can cause real problems." You are right what CVEs are, but there are plenty of people who think CVEs are something different.
- thephyber 3y ago
  The CVEs aren’t the problem. You are describing the problems related to Vogons who assume all bureaucratic inputs are perfect. All tickets should have an escape hatch where some person with direct experience with the software can report false positives or “not an issue”. Those are different than the purpose of the article, which is describing the variance of quality of CVE data.
- waihtis 3y ago
  Not to mention the completely trash cyber risk ratings industry. If you score poorly in because of their absurd scoring methods, it will potentially impact revenue streams if you have to deal with vendor risk departments, etc who don't understand any of the underlying mechanics
- mxmlnkn 3y ago
  Seems like someone has to file a CVE regarding CVE scores leading to DOS attacks on the developers.
- tptacek 3y ago
  The worst possible response to that problem would be to run the CVE program in a way that ratifies it going forward!
3y ago

lars_francke 3y ago

Yes, the CVE database/process is broken. Step 1 would be to not treat vulnerability == CVE. It's a simple change in terminology but we need to make sure to communicate that CVEs are just one, probably the most popular, but just one source of vulnerabilities. Governance and security teams the world over need to learn this. Next step ist to establish good tooling around publishing VEX statements (Vulnerability Exploitability Exchange). That is currently not easy. In the EU we currently have the discussions about the upcoming Cyber Resilience Act which has language in it that says products must be "free of known vulnerabilities" to be allowed to be placed on the European market. We're trying to change that language but it ain't trivial. The current best proposal is saying "known _exploited_ vulnerabilities". Otherwise I'd be able to DoS my competitors by publishing bogus vulnerabilities. For that we have at least two sources: CISA KEV[1] (which just celebrated its thousands entry) and FIRST EPSS[2]. There might be more, these are the ones I'm aware of. We are a software vendor and we want to do the best we can but at the moment it's a box ticking exercise that's not useful to anyone. We want to look at all vulnerabilities but we want to focus on the important ones. We need to be able to say "our product is not affected by this vulnerability" and we need our customers to trust us. Currently, they often trust the CVE/NVD database more which is a huge problem because they are not experts in the specific products. And especially for things like libraries you need to take the context into account in which it is used. Vulnerabilities are reported (using CPE, pURL etc.) at the "library" or "application" level but they really exist at a much more granular level (e.g. a single function is affected and if that's not used there's no problem). I'm convinced that the whole space of vulnerability disclosure and management will change significantly over the next few years. We happened to stumble into this and are now active in trying to shape the Cyber Resilience Act into a form that makes more sense because that will force basically every European company to now start this process. Other countries will follow (yes I know the US already has rules for some sectors but not everything, so do other countries). [1] <https://www.cisa.gov/known-exploited-vulnerabilities-catalog https://www.cisa.gov/known-exploited-vulnerabilities-catalog> [2] <https://www.first.org/epss/model https://www.first.org/epss/model>

1MachineElf 3y ago
Do you have any thoughts on CVSSv4[0]? It appears to incorporate finer-grained and organization-specific scoring to address issues many have with the one size fits all approach currently used for CVEs. [0] https://www.first.org/cvss/v4-0/ https://www.first.org/cvss/v4-0/

eslerm 3y ago

Bogus CNAs worry me more.

hsbauauvhabzb 3y ago

From the article “The report was for a legitimate bug, where the ‑‑retry‑delay option value was being multiplied by 1000 (to milliseconds) without an overflow check. But what it was not was a security bug, Stenberg said; giving insanely large values for the option might result in incorrect delays—far shorter than requested—but it is not a security problem to make multiple requests in a short time span. If it were, "then a browser makes a DOS [denial of service] every time you visit a website — and curl does it when you give it two URLs on the same command line", he said in a followup post.” I agree there’s plenty of junk in the cve database but devils advocate: ‘a system relies on a user supplied sleep to trigger a web request which executes some function which if execute too quickly may cause double transaction execution or similar. Almost any functional bug (and arguably some UI bugs) _can_ have security or real world impacts under some scenario. CVE cannot generically say that any bug does not have security implications, noting that not all security bugs require a threat actor. A data breach is bad, regardless of if it’s a hack or if it’s somebody accidentally leaving a pile of printed PII on a bus, even if most piles of paper left on busses are just trash.

blueflow 3y ago
> may cause double transaction execution or similar. From curl(1): --retry‐delay <seconds> Make curl sleep this amount of time before each retry when a transfer has failed with a transient error How would a retry at the wrong time cause a double transaction? I doubt this would be a bug on the curl side.
- hsbauauvhabzb 3y ago
  How many unix tools have idiosyncratic behaviours to prevent breaking obscure use cases? If there’s not a situation which exists where that bug may be a security issue, I could hack one up just so there is.
  - blueflow 3y ago
    Then your "situation" should get the CVE. It is indirectly relying on transient errors (like DNS resolution) not happening.

junon 3y ago

Can relate. Dealing with "ReDOS" (regex DOS) CVEs used solely to win bounties from certain sites, paired with overly hostile and secretive "researchers", has turned me off from the whole CVE system entirely.

waihtis 3y ago

Outside of regulation, we also have the dumpster file called cyber risk ratings pulling more and more leverage in vendor risk assessments; good luck arguing with an enterprise vendor risk department why having 100 9.8 CVSS score vulnerabilities actually don't impact anything

kei62sii 3y ago
I work on this exact problem for a cybersecurity company, and it’s super challenging 1) because it’s like trying to assign a probability to an earthquake (how do you validate your model with such few ground truth events?) and 2) like you mention, the more accurate we make the model, the more pushback there is as people begin to lose trust when it doesn’t match their intuition. What’s a good way to make a convincing case of “trust us, we know this risk assessment doesn’t look intuitive but it’s better than the common XYZ approach because…”? Or is this a losing battle, and we should just allow customers to rank by CVE severities if that’s what they really want to do? Some smaller startups are starting to work toward providing cyber VaR — quantify the probabilistic loss in financial terms over a given time period. This is really the only direction forward in cybersecurity IMO as risk must ultimately be modeled with statistics, not hunches. But I don’t think the industry is ready for it (and neither are the models to be frank).
- waihtis 3y ago
  There may be nothing novel here for you given you're already working in the problem area, but as established a CVE measure is completely hypothetical on its own; if your customers insist on looking at one variable (CVE), you could always make the case that CVE is incomplete on it's own and should always be supplemented with data such as: a) is it actually exploitable b) is someone exploiting it in the wild c) accessibility of component/asset/application/whatever that carries the CVE (e.g. is it internet-facing or a non-networked subcomponent hidden under 15 layers of defense) d) other exploitability scoring methods, like EPSS e) etc... (sometimes I like to use the analogue of trying to gauge your body's health based on one variable, like your body temperature - it can be high temporarily [in case of a flu] but it doesn't tell you anything about your overall health [as we all get sick occasionally]) The VaR approach is sensible but has to start from much further down in the root of the problem by first creating a value-based catalogue of all your assets, after which you should simulate all prospective attack paths towards all assets (maybe above a certain value threshold to simplify the model a bit), then overlay with attack / exploit probability data (simple example would be a CVE dataset + EPSS scoring) and finally you have something resemblant of an actual data-based risk model you can quantify impacts on. It's quite a large task and I don't think there's a single player doing all areas, you'd have to patchwork together multiple different datasets. Still, it's probably where the industry will end up in the next 10-15 year timespan. (I'm somewhat involved in the area too but just as a threat data provider for larger models.)

xvilka 3y ago

For anyone working with CVE and other advisories there's a cool search engine and feed for those - Vulners[1]. They have integration with various providers and tools, as well as allow to search also for exploits among numerous blogs. [1] https://vulners.com https://vulners.com

nonrandomstring 3y ago

General wisdom is: error < bug < vuln < poc < wild exploit Yet a narrow security view cantered around single faults within single programs misses a lot. Take backdoors, which are neither errors nor bugs, nor may be seen in the wild, but are simply vulnerabilities in waiting, working as expected by design. Or a set of mere errors may, in concert, lead to a systemic and commonly exploited vulnerability. Giving an identifier to a reproducible observation, in some context and time, is a good start. But to be useful it needs to be part of a wider, joined-up taxonomy that helps us understand risk better.

LudwigNagasena 3y ago

That reminds me of bogus `npm` results. You use a regex at build time? Oh, apparently you are exposed to a ReDoS attack even though that regex isn’t available at runtime. Or that time when Python disallowed string to int conversion of long strings because it is a DoS vulnerability. The lack of sanitisation led to DoS vulnerabilities in applications, but the conversion is obviously not a vulnerability in itself. Oh, and now those apps have DoS attack that just halts the application because the conversion throws ValueError. It’s hard to say what the solution to that problem is. Oftentimes security vulnerabilities indeed don’t cause any problems alone and require a chain of other exploits to actually cause trouble. At the same time you don’t want to treat every function that runs in O(n^2) on input as a potential DoS attack. Distinguishing between a simple bug and a vulnerability requires contextual knowledge and expert judgement that simply cannot be built into bureaucratic organisations like MITRE.

stefan_ 3y ago
This is my favorite lately. Sure, some insane people use these packages for backend services, but we are just building a big JS blob that's statically shipped to the browser; none of the "vulnerabilities" in them apply. But try getting this across to people who think they need to track security issues they don't understand (not fix them, of course).
derstander 3y ago
> Distinguishing between a simple bug and a vulnerability requires contextual knowledge and expert judgement that simply cannot be built into bureaucratic organisations like MITRE. Why can’t it? MITRE runs FFRDCs. Having worked for an FFRDC (although in electrical engineering, not computer security), if you were to tell me that engineers in FFRDCs or UARCs in my domain simply cannot utilize contextual knowledge or expert judgment (or that their knowledge and judgment would be completely overridden by management such that it appears they can’t) then I would wonder if you ever interacted with that kind of organization. Is computer security somehow different to e.g. radar or comms systems engineering?
- LudwigNagasena 3y ago

baby 3y ago

Everything about CVEs when I used them screamed “old/legacy stuff” from the processes to the tooling.

kbrkbr 3y ago

In my company we have moved to using EPSS (exploit probability scoring system) on top of CVE's. That filters a lot of noise. https://www.first.org/epss/ https://www.first.org/epss/

jcarrano 3y ago

As I mentioned a couple days ago [1], this problem will get much worse in the EU with the upcoming CRA. Either the only software vendors left will be big ones with the capacity and leverage to counter these CVEs, or companies will turn to obscure, bug-ridden software which have no CVEs because no one bothers to research them. [1] https://news.ycombinator.com/item?id=37582808 https://news.ycombinator.com/item?id=37582808

112233 3y ago
expect closed-source analog of GNU, and a license that forbids showing source code, plus tools for obfuscating binaries, masked as "optimizers". there will be ademand for anything that reduces chance of CVE discovery and that reduces open source SBOM entries. open source most likely becomes a liability under CRA
- jcarrano 3y ago
  Yes, and the next step is that the vendor will sue researchers for posting CVEs which they consider "bogus", accusing them of hurting their business. It's not so much as open source becomes a liability, but rather "well known", whether open or not, becomes a liability. If you pull a random json library from github that does not even appear on any vulnerability database then you will probably be fine.
  - delfinom 3y ago
    Shit, I think the best way to fight this is to file bogus CVEs against propertiary software vendors. You'll see the industry demand CVEs be fixed within a month. You can after-all, file CVEs anonymously. It takes no effort to stand up a fake "research firm", or two, or three doing so. Bonus points if the CVEs are against all the shitty cybersec scan tools.
    - msm_ 3y ago
      You can file CVEs anonymously, but CNA don't have to assign a CVE number if they consider submission bogus. Depending on the CNA they may also verify the submission themselves or contact the vendor before assignment (usually vendor has to confirm the vulnerability before the publication, often CVE publication date is decided together with the vendor if the vulnerability is not already public). Of course there are shitty CNAs that don't care. But honestly CVEs are a useful tool for communication, people (both security people and non-security people) should stop obsessing over them.

tyingq 3y ago

There's lots of common sense missing in how CVEs are used at ground level in a company also. Like, say, I use node.js in my project solely in the build/deploy process, and not at all in the deployed code. Something like using the serverless framework to make python AWS Lambdas. The processes in place often can't make that distinction, so some "high /critical node.js when used as a web server" problem halts all builds, even though there's no node.js in the deployed application, and node.js is only used cli style.

robszumski 3y ago
This is why having an accurate scope is key. Easiest way to do that in an hour is to prefer to scan a workload like a container instead of a code repo. Or have a scanner that understands how something is built (and ideally how it runs).

petesergeant 3y ago

> Various CNAs assign CVE numbers for their own products (e.g., Microsoft, Oracle, HP, Red Hat) I wonder if this is part of the problem. The big players aren't hurt by this problem because they get to decide what's a real CVE for themselves, so there's little commercial push for change.

oooyay 3y ago

It's awful that a system meant for the benefit of maintainers and users is now being perverted into such theatrics. In a way, if you want to undermine the concept of security this is one of the ways you can do it. On the user side of this token I experience similar theatrics. Monthly I have to deal with a list of vulnerabilities that are detected within our software. Some are easy fixes, like updating a package manager which equates to just rerunning a pipeline. Others are not, and I found myself recently trying to rebuild an entire container build process because the software is on a biannual release process and they'd used the x/crypto package in Go which contains an SSH client and server. The server was vulnerable and the software we were using made use of the client, but not the server. Regardless our software was flagged with a high vulnerability. The utilities available are smart enough to take apart my Go binary during container inspection but not smart enough to figure out if I actually use the vulnerable thing. The resulting toil and theatrics aren't just a time waster, they're a soul sucking activity laiden with difficult tasks to achieve minor outcomes like, "How do I make sure we miss the possibility of even irrelevant CVEs showing up in the first place" - which is probably not what you want.

rascul 3y ago

Greg Kroah-Hartman (the kernel developer/maintainer) in 2019 gave a presentation about some of the issues with CVEs. https://www.youtube.com/watch?v=HeeoTE9jLjM https://www.youtube.com/watch?v=HeeoTE9jLjM https://github.com/gregkh/presentation-cve-is-dead https://github.com/gregkh/presentation-cve-is-dead

skissane 3y ago

Here’s an example of a bogus CVE in the GNU C Library: https://nvd.nist.gov/vuln/detail/CVE-2023-0687 https://nvd.nist.gov/vuln/detail/CVE-2023-0687 That CVE is correct that the “gmon” component of glibc contains memory corruption bugs. In fact, I managed to find even more than the specific one that CVE is about. But, the claim that this is a security vulnerability is a bit silly. These are profiler functions, which are usually only called in non-production builds with profiling enabled (gcc -pg), and even then only from CRT startup code. It is rather unlikely an attacker can exploit any of these bugs in them without already having the ability to run arbitrary code in the process, at which point they don’t need these functions, they’ve already got all these bugs have to give them.

bcoughlan 3y ago

The solution at my workplace is a bot that opens PRs to bump dependencies and automatically merges if the tests pass. It's taken a lot of workload off devs to meet security targets. But I worry it makes supply-chain attacks more attractive. If an attacker can compromise a package and it's instantly merged into the codebases of thousands of different companies that's a huge danger.

ashishbijlani 3y ago
I've been building Packj [1] to detect dummy, malicious, abandoned, typo-squatting, and other "risky" PyPI/NPM/Ruby/PHP/Maven/Rust packages. It carries out static/dynamic/metadata analysis and scans for 40+ attributes such as num funcs/files, spawning of shell, use of SSH keys, network communication, use of decode+eval, etc. to flag risky packages. Packj Github action [2] can alert if a risky dependency is pulled into your build. 1. https://github.com/ossillate-inc/packj https://github.com/ossillate-inc/packj 2. https://github.com/ossillate-inc/packj-github-action https://github.com/ossillate-inc/packj-github-action

Aachen 3y ago

Well that's a poor article. Not used to that from LWN at all. I think this sentence sums up most of its problems: > In fact, the pull request for the fix was attached to the report, but that apparently made little difference in the assessment from NVD. - CVEs are about vulnerabilities, not fixes. If everyone updated their software and all software updated its dependencies, I'm not sure we needed a vulnerability tracking system at all! The fix being available doesn't change the impact or exploitability of the bug while you run the out-of-date software - The text makes it sound like ratings given are opinions (elsewhere it talks about NVD having a "change of heart"). NVD uses a calculation system called CVSS where you input parameters like whether the vulnerable component can be reached over a network, whether the vulnerability results in denial of service or in confidentiality being breached or whatnot. These are objective parameters, and while interpretation mistakes are extremely common¹, ending up with a vastly different score is rather difficult if you have all relevant information to fill it out in the first place (also because mistakes are likely to cancel out). The article acts as though it's their subjective assessment, but rather, the score changes with new facts being presented. (In an ideal world, of course. In practice they typically forget or aren't told the correct info. But apparently here they were told and did update, so that's good right?) The article then goes on to state that curl uses its own subjective system now instead and they gave some bug an entirely different rating. The article does not say anything towards who had the more correct result but then goes with the assumption that curl's own subjective rating is better. Is that really what we want, when complaining that NVD gives bad ratings? More subjectivity, intransparency about how they got to a particular result? Anyway I'm neither loving the CVE system nor CVSS ratings either, but these are the wrong reasons ¹ http://export.arxiv.org/abs/2308.15259 http://export.arxiv.org/abs/2308.15259 recent study into CVSS scoring inconsistencies

userbinator 3y ago

Service-level contracts and governmental requirements mean that a critical CVE needs to be addressed in short order, so non-critical bugs that get marked that way can cause real problems. Bureaucracy destroys common sense once again. If every developer opposes doing anything with bogus CVEs and calling them out for the BS they are, with a detailed explanation of why, then we might get some changes, but unfortunately just mentioning anything "security" has gotten such a paranoic response from most of the population that it's a difficult battle.

donatj 3y ago

I feel like there are a lot of CVE's in the JS community that basically require an attacker to already have access to server side code execution. That seems like it raises some bigger questions.

insanitybit 3y ago

It's funny. I was just learning about RuneWatch, which is how the Runescape community attempts to democratize and distribute the problem of dealing with scammers. The evolution was basically that it started the way CVEs work. You could submit a report and there wasn't really any verification. Of course, this was abused - far worse than the CVE system, naturally, but the same sort of problem. Now they require a lot of evidence and things are much much better and abuse is far more difficult. It seems sort of obvious. If this is a vulnerability, prove it. It almost feels like a CVE should have a "practical" tag. Like: 1. Is this theoretically exploitable? 2. Is this practically exploitable? 3. Is this proven exploitable? This should be a massive contributor to the score. It also means that a score can be raised by having the sec researcher demonstrate an exploit - so disagreements can be resolved very easily. Related, here's a post from Spender on the topic (since the "CVE bad" situation is nothing new) https://grsecurity.net/reports_of_cves_death_greatly_exaggerated https://grsecurity.net/reports_of_cves_death_greatly_exagger...

wnevets 3y ago

CVEs became a target for people's careers and wallet. I'm not surprised it's turning into a mess.

testplzignore 3y ago

My all-time most hated CVE is CVE-2022-24975. This is not a vulnerability - it's a user education issue with not understanding how Git works. The Git documentation not explicitly explaining the dangers of using mirror was rated a * 7.5 *. Due to the policy of having no high CVEs, this ground development at my company to a halt for an entire day until our security team put an exemption in place.

AtNightWeCode 3y ago

It is not just a CVE problem. Many sec standards feels really random and difficult to work with. You can get better scores on some of these sec scans if you open up some ports in the firewall.

memorysafety 3y ago

CVE scoring is parasocial activity. Hence so much drama. Similarly to SemVer, the good-faith grader attempts to convey a sizeable blob of knowledge... by compressing it into a one-dimensional number. No matter the scoring formula, this step is lossy. On the receiving end of this communication, all you can do with the score is add a huge grain of salt to it, then perhaps use to prioritize your review queue. You still must check the details, and work out a judgement tailored to your specific context. There's no other way. There isn't a choice for the grader either, to skip the obscenely lossy scoring step. Just like with release versions, they must do it, as the audience consists of unbounded number of engineers; faithfully doing it saves mountains of time for everyone involved (present and future). Just like with dependency upgrades, it's the consumer's choice to disregard CVE scores (version numbers), vulnDB entries (changelogs), or even existence itself of a vulnerability (upgrade). Likewise, it's their fault if consequences arise. Viewed thusly, can be seen: anecdotes of pointwise drama will continue (even when the bulk of activity chugs along happily, efficiently and quietly) -- because at the core of it, CVE ID's and scores are just that, a communication tool. It mostly can't make strangers exercise care or spend effort more than they're willing to. It can optimise utility of attention that they do pay.