4 ms·

Curl Project and Go Security Teams Reject CVSS as Broken

40 points by feross 2y ago

grajaganDev 2y ago
Assigning severity to vulnerabilities is a very difficult problem. This is exacerbated by the fact that most modern exploits are chains of multiple vulnerabilities. I think a vulnerability should be scored based on the context where the most damage could be done.
- CharlesW 2y ago
  > I think a vulnerability should be scored based on the context where the most damage could be done. As I read it, the point of the article is that CVSS makes sense in the case where you know that, but is near-useless when you don't.
  - duskwuff 2y ago
    And in practice, it means CVSS is only really meaningful on software which exists as a complete system, like a web application or a device - questions like "is user interaction required to exploit this vulnerability" are usually unanswerable for individual software components.
deleted 2y ago
[deleted]
grajaganDev 2y ago
I always take CVSS scores with a grain of salt and assume that they could be very wrong in either direction.
- kstrauser 2y ago
  The final score is kind of a die roll, but I appreciate having at least a common vocabulary to discuss these. It also means that a given vendor can’t completely downplay they score. “Sure, it’s network accessible without authentication using wget, but we consider that a low.”
  - Avamander 2y ago
    Unfortunately now vendors can just say "we don't believe in assigning a score because the scoring system is broken" and that seems fine to some.
  - blablabla123 2y ago
    I agree, also I see this mostly as a tool. Sure there are more reports that don't necessarily have the highest quality. But just omitting those doesn't seem the correct approach.
xyzzy123 2y ago
Nobody in the supply chain seems incentivised to get this right. Our feeds at work are swamped with low quality CVEs and the commercial vendors we pay for "intelligence" just leech off community feeds and add zero value. We get told there are CVSS 9s relating to kernel vulns due to kernel header packages in containers, we get 9s for "RCE" and when you go look it's applicable with extremely niche settings or compile flags that no OS distribution in existence actually enables. Nobody at the "supply chain" tool vendors seem to take 2 seconds to sanity check ANYTHING - let alone do the kind of high effort thinking that the curl or go projects are looking for. Sometimes you go look and there's no bug at all - you go look at the github issue thread and the reporter misunderstood what they were looking at :/ I guess more "findings" = "more value" in demos when naive customers look at the output. The way I see people dealing with this is: * Make it easy to build and deploy regularly. Try to do that. * Don't look at individual issues at all unless your hand is forced. Investigating low signal CVEs is net negative value when you factor in opportunity cost (i.e. the other things maintainers could be doing to make the software better or more secure). * Keep an eye on whether issues are forever trending up, i.e. the golden signal is "is this asset being maintained at all". It's easy for things to drop off the maintenance radar. * Pay for "overlays" that add actual signal (of exploitability) to the crap flood. Do investigate when a "high signal" vuln pops up on your dashboard. If you have a dedicated security team it seems reasonable for them to handle this bit[1]. [1] 90% of the time if you have a few staff who hang out on the right forums they will hear about the genuine "CVSS 10s" at the same time your security team does, paying for vuln intel mainly serves to formalise things a bit vs having to handwave when someone asks you what your process for detecting "critical vulnerabilities" is.
RainyDayTmrw 2y ago
At my day job, we are required by auditors to have automation that reports on CVEs and CVSS scores. Then, a non-technical human comes to bug us about them. I think there's something to be said about how, if something can be misused, someone will come along to misuse it, so perhaps we should all be more mindful in designing misuse resistance.
sc68cal 2y ago
Ha my HN comment is in the article. My 5 seconds of internet fame have come at last

40 points by feross 2y ago

Assigning severity to vulnerabilities is a very difficult problem. This is exacerbated by the fact that most modern exploits are chains of multiple vulnerabilities. I think a vulnerability should be scored based on the context where the most damage could be done.

CharlesW 2y ago
> I think a vulnerability should be scored based on the context where the most damage could be done. As I read it, the point of the article is that CVSS makes sense in the case where you know that, but is near-useless when you don't.
- duskwuff 2y ago
  And in practice, it means CVSS is only really meaningful on software which exists as a complete system, like a web application or a device - questions like "is user interaction required to exploit this vulnerability" are usually unanswerable for individual software components.

deleted 2y ago

[deleted]

grajaganDev 2y ago

I always take CVSS scores with a grain of salt and assume that they could be very wrong in either direction.

kstrauser 2y ago
The final score is kind of a die roll, but I appreciate having at least a common vocabulary to discuss these. It also means that a given vendor can’t completely downplay they score. “Sure, it’s network accessible without authentication using wget, but we consider that a low.”
- Avamander 2y ago
  Unfortunately now vendors can just say "we don't believe in assigning a score because the scoring system is broken" and that seems fine to some.
- blablabla123 2y ago
  I agree, also I see this mostly as a tool. Sure there are more reports that don't necessarily have the highest quality. But just omitting those doesn't seem the correct approach.

xyzzy123 2y ago

Nobody in the supply chain seems incentivised to get this right. Our feeds at work are swamped with low quality CVEs and the commercial vendors we pay for "intelligence" just leech off community feeds and add zero value. We get told there are CVSS 9s relating to kernel vulns due to kernel header packages in containers, we get 9s for "RCE" and when you go look it's applicable with extremely niche settings or compile flags that no OS distribution in existence actually enables. Nobody at the "supply chain" tool vendors seem to take 2 seconds to sanity check ANYTHING - let alone do the kind of high effort thinking that the curl or go projects are looking for. Sometimes you go look and there's no bug at all - you go look at the github issue thread and the reporter misunderstood what they were looking at :/ I guess more "findings" = "more value" in demos when naive customers look at the output. The way I see people dealing with this is: * Make it easy to build and deploy regularly. Try to do that. * Don't look at individual issues at all unless your hand is forced. Investigating low signal CVEs is net negative value when you factor in opportunity cost (i.e. the other things maintainers could be doing to make the software better or more secure). * Keep an eye on whether issues are forever trending up, i.e. the golden signal is "is this asset being maintained at all". It's easy for things to drop off the maintenance radar. * Pay for "overlays" that add actual signal (of exploitability) to the crap flood. Do investigate when a "high signal" vuln pops up on your dashboard. If you have a dedicated security team it seems reasonable for them to handle this bit[1]. [1] 90% of the time if you have a few staff who hang out on the right forums they will hear about the genuine "CVSS 10s" at the same time your security team does, paying for vuln intel mainly serves to formalise things a bit vs having to handwave when someone asks you what your process for detecting "critical vulnerabilities" is.

RainyDayTmrw 2y ago

At my day job, we are required by auditors to have automation that reports on CVEs and CVSS scores. Then, a non-technical human comes to bug us about them. I think there's something to be said about how, if something can be misused, someone will come along to misuse it, so perhaps we should all be more mindful in designing misuse resistance.

sc68cal 2y ago

Ha my HN comment is in the article. My 5 seconds of internet fame have come at last