4 ms·

>> Well, empirically, no, that's not the case Rust itself has had more CVEs in the past 18 months than Postfix (written in C) has had in the past 18 years [1].

>> Well, empirically, no, that's not the case Rust itself has had more CVEs in the past 18 months than Postfix (written in C) has had in the past 18 years [1]. So it must be more nuanced than you're claiming. EDIT: Postfix is somewhere in the region of 1,000x to 10,000x more popular than rust - it's hard to compare a compiler to a mail transfer agent but the MTA is in the more risky use case and yet fares better despite having more sysadmins deploying it than there are rust developers. [1] https://www.cvedetails.com/vulnerability-list/vendor_id-19029/product_id-48677/Rust-lang-Rust.html https://www.cvedetails.com/vulnerability-list/vendor_id-1902... https://www.cvedetails.com/vulnerability-list/vendor_id-8450/Postfix.html https://www.cvedetails.com/vulnerability-list/vendor_id-8450...

scaredginger 4y ago
I mean, you're not wrong, but cherry-picking two examples that aren't really comparable isn't a good argument
- CraigJPerry 4y ago
  Agreed. It's hard to find examples of successful large Rust projects to compare against. The compiler and maybe deno are about it i think? This is not what you'd expect for an alleged developer rocket ship after 10+ years. It doesn't compare favourably with C++ at 10 years old, or Java, or Go, or...
  - sanxiyn 4y ago
    Largest Rust systems aren't public, like both Dropbox server and client. Millions of lines of Rust are shipping in Firefox, so we could count CVEs in Firefox attributable to Rust. I agree this is annoying, but someone should do this. I point out Rust in Firefox is an order of magnitude (10x) larger than rustc or deno.
    - casey2 4y ago
      It's not even millions yet, and Firefox along with Mozilla is already dying. When will the cargo cult end.
      - sanxiyn 4y ago
        Please... I ran line counter on mozilla/gecko-dev just before writing that. There are three millions. Go clone it and count yourself.
  - scaredginger 4y ago
    It sounds like you just have a bone to pick here, but I would like to point out that C++ came out of Bell Labs, Java had massive corporate backing including a huge marketing campaign and Go has Google's backing. Each of your examples had huge advantages unrelated to any technical benefit. A closer comparison might be Python, which took a long time to reach its potential, and I see Rust on the same trajectory, with high-profile teams now adopting it, like AWS.
lucasyvas 4y ago
It's definitely more nuanced. That said, I'd rather only have to fix memory unsafety in the language than potentially every program written with it. It reduces the audit surface area to just "unsafe" and Rust itself.
rhn_mk1 4y ago
That data point doesn't say much before accounting for some confounding variables: - the complexity of the code base - the attack surface - the amount of effort to find the CVEs
- CraigJPerry 4y ago
  Sure - and i wouldn't put too much weight on my examples, i picked them as a direct refutation to the GP's claim "Well, empirically, no, that's not the case". That said - 2 & 3 in your list seem like they'd lend further weight to Postifx rather than Rust in this scenario. Postfix is a network service forced to accept any and all input from remote attackers. Postfix has been around longer and has marketed itself as secure, that's meant it's attracted the attention of capable attackers keen to demonstrate their prowess. I think the rust code base is more complex than postfix though. Postfix's architecture borders on a work of art.
burntsushi 4y ago
You're comparing a programming language's standard library with an application. In the former case, most of the CVEs are pointing out bugs in 'unsafe' implementations of std APIs that result in UB where they should not. In the latter case, the CVEs are mostly pointing out actual exploits that permit privilege escalation. If you applied the standard by which CVEs are created for Rust to C---an apples to apples comparison---then most APIs in libc would have CVEs filed for them and they would be marked as 'wontfix'. It's not at all a direct refutation of the GP's claim.
dralley 4y ago
A lot of Rust CVEs are filed for things that C/C++ software would just consider normal bugs, or not bugs at all. For example: https://blog.rust-lang.org/2022/01/20/cve-2022-21658.html https://blog.rust-lang.org/2022/01/20/cve-2022-21658.html The C++ stdlib is affected by the same issue, but they don't consider it a security bug, because the API doesn't claim to be safe of race conditions. Rust doesn't explicitly claim that either, but the Rust community has a higher bar for correctness around these things, so it gets a CVE. Prior discussion here: https://news.ycombinator.com/item?id=30461036 https://news.ycombinator.com/item?id=30461036 https://www.reddit.com/r/cpp/comments/s8ok0h/possible_toctou_vulnerabilities_in/hti8jyt/ https://www.reddit.com/r/cpp/comments/s8ok0h/possible_toctou...
pcwalton 4y ago
Completely uncomparable. This is a great example of why CVE counts are a useless metric.
howinteresting 4y ago
To be clear, if you're using number of CVEs to determine whether some software is more secure than other software then you are very directly incentivizing people to not file CVEs. Your actions are directly harming the ecosystem.
Jweb_Guru 4y ago
I'm sorry, but you don't understand how Rust's safety guarantees work. There would be multiple CVEs filed against almost every single library in libc and C++'s standard library if we used the same standards as Rust for those two. We don't, because it would obviously be pointless to do so, because C and C++ libraries generally don't have explicit safety guarantees, and those that do usually put an extremely high correctness burden on the caller. AFAIK, virtually all of Rust's CVEs found to date would go away (assuming the rest of the code remained the same) if Rust libraries just declared all the APIs in question "unsafe," which is basically what C and C++ do. The Rust standard library, your "random" example here, would have 0 CVEs by this metric, and so would pretty much every Rust library (the vast majority of Rust CVEs are filed against libraries). In other words--a library can't be blamed for being insecure if it never makes any security promises in the first place. Rust libraries make stronger and more explicit guarantees, and the code is encapsulated in a much smaller region of the program, so it is easier to prove that they are in violation of their contract; but it emphatically does not mean that it results in more end to end vulnerabilities. Claiming that Rust simply isn't used in enough, or critical enough, places yet to have real exploits like C programs do is pretty naive, as it's started to be used in places like Android's bluetooth stack. In fact, in practice there's even more of a gap than this--most C and C++ programs don't report CVEs even for reliably reproducible instance of UB unless you can demonstrate a full end to end attack chain, and many won't even fix or prioritize them if they consider it "benign" UB (what this means is anyone's guess!). Even projects like Chrome that actually care about security often won't upgrade UB bugs to CVEs if they believe (erroneously or not) that other hardening mechanism in the program will prevent it from being used as part of an RCE chain. Applying the same standards people typically do to C and C++ projects (an end to end exploit chain), I'm not sure whether there would be a single known Rust CVE in any widely used program. Basically: the standard of evidence for a CVE in your average C or C++ program is astronomically high, while in Rust it is extremely low. Any attempt to decide which language is practically safer by counting CVEs (which is in any case bad methodology) needs to have awareness of this fact if it's trying to actually measure anything real.
pclmulqdq 4y ago
There are standard criteria for what can become a CVE, and I guess some Rust users are very zealous with bug reporting. At first glance, it looks like a lot of these issues are minor, but not normal, bugs to see in a programming language. I guess abnormality rises to that level.

scaredginger 4y ago

I mean, you're not wrong, but cherry-picking two examples that aren't really comparable isn't a good argument

CraigJPerry 4y ago
Agreed. It's hard to find examples of successful large Rust projects to compare against. The compiler and maybe deno are about it i think? This is not what you'd expect for an alleged developer rocket ship after 10+ years. It doesn't compare favourably with C++ at 10 years old, or Java, or Go, or...
- sanxiyn 4y ago
  Largest Rust systems aren't public, like both Dropbox server and client. Millions of lines of Rust are shipping in Firefox, so we could count CVEs in Firefox attributable to Rust. I agree this is annoying, but someone should do this. I point out Rust in Firefox is an order of magnitude (10x) larger than rustc or deno.
  - casey2 4y ago
    It's not even millions yet, and Firefox along with Mozilla is already dying. When will the cargo cult end.
    - sanxiyn 4y ago
      Please... I ran line counter on mozilla/gecko-dev just before writing that. There are three millions. Go clone it and count yourself.
- scaredginger 4y ago
  It sounds like you just have a bone to pick here, but I would like to point out that C++ came out of Bell Labs, Java had massive corporate backing including a huge marketing campaign and Go has Google's backing. Each of your examples had huge advantages unrelated to any technical benefit. A closer comparison might be Python, which took a long time to reach its potential, and I see Rust on the same trajectory, with high-profile teams now adopting it, like AWS.

lucasyvas 4y ago

It's definitely more nuanced. That said, I'd rather only have to fix memory unsafety in the language than potentially every program written with it. It reduces the audit surface area to just "unsafe" and Rust itself.

rhn_mk1 4y ago

That data point doesn't say much before accounting for some confounding variables: - the complexity of the code base - the attack surface - the amount of effort to find the CVEs

CraigJPerry 4y ago
Sure - and i wouldn't put too much weight on my examples, i picked them as a direct refutation to the GP's claim "Well, empirically, no, that's not the case". That said - 2 & 3 in your list seem like they'd lend further weight to Postifx rather than Rust in this scenario. Postfix is a network service forced to accept any and all input from remote attackers. Postfix has been around longer and has marketed itself as secure, that's meant it's attracted the attention of capable attackers keen to demonstrate their prowess. I think the rust code base is more complex than postfix though. Postfix's architecture borders on a work of art.

burntsushi 4y ago

You're comparing a programming language's standard library with an application. In the former case, most of the CVEs are pointing out bugs in 'unsafe' implementations of std APIs that result in UB where they should not. In the latter case, the CVEs are mostly pointing out actual exploits that permit privilege escalation. If you applied the standard by which CVEs are created for Rust to C---an apples to apples comparison---then most APIs in libc would have CVEs filed for them and they would be marked as 'wontfix'. It's not at all a direct refutation of the GP's claim.

dralley 4y ago

A lot of Rust CVEs are filed for things that C/C++ software would just consider normal bugs, or not bugs at all. For example: https://blog.rust-lang.org/2022/01/20/cve-2022-21658.html https://blog.rust-lang.org/2022/01/20/cve-2022-21658.html The C++ stdlib is affected by the same issue, but they don't consider it a security bug, because the API doesn't claim to be safe of race conditions. Rust doesn't explicitly claim that either, but the Rust community has a higher bar for correctness around these things, so it gets a CVE. Prior discussion here: https://news.ycombinator.com/item?id=30461036 https://news.ycombinator.com/item?id=30461036 https://www.reddit.com/r/cpp/comments/s8ok0h/possible_toctou_vulnerabilities_in/hti8jyt/ https://www.reddit.com/r/cpp/comments/s8ok0h/possible_toctou...

pcwalton 4y ago

Completely uncomparable. This is a great example of why CVE counts are a useless metric.

howinteresting 4y ago

To be clear, if you're using number of CVEs to determine whether some software is more secure than other software then you are very directly incentivizing people to not file CVEs. Your actions are directly harming the ecosystem.

Jweb_Guru 4y ago

I'm sorry, but you don't understand how Rust's safety guarantees work. There would be multiple CVEs filed against almost every single library in libc and C++'s standard library if we used the same standards as Rust for those two. We don't, because it would obviously be pointless to do so, because C and C++ libraries generally don't have explicit safety guarantees, and those that do usually put an extremely high correctness burden on the caller. AFAIK, virtually all of Rust's CVEs found to date would go away (assuming the rest of the code remained the same) if Rust libraries just declared all the APIs in question "unsafe," which is basically what C and C++ do. The Rust standard library, your "random" example here, would have 0 CVEs by this metric, and so would pretty much every Rust library (the vast majority of Rust CVEs are filed against libraries). In other words--a library can't be blamed for being insecure if it never makes any security promises in the first place. Rust libraries make stronger and more explicit guarantees, and the code is encapsulated in a much smaller region of the program, so it is easier to prove that they are in violation of their contract; but it emphatically does not mean that it results in more end to end vulnerabilities. Claiming that Rust simply isn't used in enough, or critical enough, places yet to have real exploits like C programs do is pretty naive, as it's started to be used in places like Android's bluetooth stack. In fact, in practice there's even more of a gap than this--most C and C++ programs don't report CVEs even for reliably reproducible instance of UB unless you can demonstrate a full end to end attack chain, and many won't even fix or prioritize them if they consider it "benign" UB (what this means is anyone's guess!). Even projects like Chrome that actually care about security often won't upgrade UB bugs to CVEs if they believe (erroneously or not) that other hardening mechanism in the program will prevent it from being used as part of an RCE chain. Applying the same standards people typically do to C and C++ projects (an end to end exploit chain), I'm not sure whether there would be a single known Rust CVE in any widely used program. Basically: the standard of evidence for a CVE in your average C or C++ program is astronomically high, while in Rust it is extremely low. Any attempt to decide which language is practically safer by counting CVEs (which is in any case bad methodology) needs to have awareness of this fact if it's trying to actually measure anything real.

pclmulqdq 4y ago

There are standard criteria for what can become a CVE, and I guess some Rust users are very zealous with bug reporting. At first glance, it looks like a lot of these issues are minor, but not normal, bugs to see in a programming language. I guess abnormality rises to that level.