10 ms·

>After all, I have worked for decades to make it possible to write better, safer, and more efficient C++. In particular, the work on the C++ Core Guidelines spe

by carlmr 4y ago

>After all, I have worked for decades to make it possible to write better, safer, and more efficient C++. In particular, the work on the C++ Core Guidelines specifically aims at delivering statically guaranteed type-safe and resource-safe C++ for people who need that without disrupting code bases that can manage without such strong guarantees or introducing additional tool chains. After all statistics just isn't on his side. Even the top engineers hired by Google create around 1 memory error every 1000 lines. And they didn't have one yet in 1.5 million lines of Rust [1] This is with all of the analysis tools at their disposal and in use for C++. And most companies can't afford the same top talent that Google can. I've been working in safety critical software for a long time now. C and C++ will never be safe because their safety model relies on human infallibility. Ada and Rust IMO have the most potential here. [1] https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html https://security.googleblog.com/2022/12/memory-safe-language...

menio 4y ago
Thanks for sharing this. Wow, its a really suprising statistics to see 0 vuln found in 1.5m loc, even with Rust.
- carlmr 4y ago
  I personally expect there are still some unfound memory errors in there, but it's such a stark contrast to C++ it's amazing. However memory errors in C++ account for 70% of vulnerabilities. I'd expect 30% to thus remain. Edit: meant to write memory errors not vulns.
  - nindalf 4y ago
    It's new code, there must be some logic errors, off-by-one errors, assumptions not expressed in the code etc. But that's not saying much, all new code has issues. The good thing is that since this code was written in a memory safe language, at least that eliminates an entire class of bugs.
- pdw 4y ago
  Zero memory errors, not zero vulnerabilities. In high-quality C and C++ projects, about 70% of vulnerabilities are caused by memory errors. (https://alexgaynor.net/2020/may/27/science-on-memory-unsafety-and-security/ https://alexgaynor.net/2020/may/27/science-on-memory-unsafet...)
- GuB-42 4y ago
  Not zero vulnerabilities, zero memory errors. Memory errors make a good share of vulnerabilities, but not all vulnerabilities are memory errors. And not all memory errors are vulnerabilities, sometimes it is just an inexploitable bug. It is to be expected that there is no bug in the very thing that Rust is supposed to prevent. The same can be said of Java, JS or even PHP. What surprises me a little though is that there is zero memory error even in unsafe Rust. There has to be some unsafe code somewhere, for low level data structures and things like that.
  - carlmr 4y ago
    >What surprises me a little though is that there is zero memory error even in unsafe Rust. There has to be some unsafe code somewhere, for low level data structures and things like that. Even unsafe Rust has more strict typing and borrow checking which C++ doesn't have. It does allow raw pointers, but it's still safer than C++. The other thing is that if say 1% of your loc is unsafe Rust, it's much easier to put this under more scrutiny than the whole C++ codebase. And 1% is a lot if you ask me. I almost never need unsafe at all. >There has to be some unsafe code somewhere, for low level data structures and things like that. In most cases you can just use the standard library code for these. They do contain unsafe of course, but it's in one of the most battle tested places.
TigerTeamX 4y ago
I just cringed when I saw C++ and safety in one sentence. This is good statistics to showcase gut feeling, thanks.
- 72deluxe 4y ago
  I presume you're using an operating system and browser written in Rust? If not, it would seem that the C/C++ based operating systems and browsers that you are use are perfectly safe to use, no??
  - carlmr 4y ago
    Those OSes and browsers do show memory safety issues time and time again. They are some of the most hardened codebases, and still fail in light of the issues with C and C++.
    - TigerTeamX 4y ago
      Exactly.
    - 72deluxe 4y ago
      The languages are not at fault. The fallible users of them are. This is identical to me writing a garbage sentence in English and then proclaiming that English as a language is wrong and needs to be replaced because it's perfectly possible to write something grammatically incorrect in it. Surely you can see the difference but are conflating use of the language with the language itself.
  - UncleMeat 4y ago
    Published last friday - https://seclists.org/oss-sec/2023/q1/20 https://seclists.org/oss-sec/2023/q1/20 Oodles of vulns in linux that are made possible by C's lack of safety guarantees.
    - simplotek 4y ago
      > Oodles of vulns in linux that are made possible by C's lack of safety guarantees. I'm sorry, isn't your link about how a user found a memory error in the Linux codebase? It seems way farfetched to jump to your conclusion from that starting point. I mean, doesn't Linux have over 12 million lines of code right now?
FpUser 4y ago
>"1 memory error every 1000 lines" One memory error per 1000 LOC? I think they probably either use old style C++ or something else weird and ignore all static and runtime memory safety tooling to get to this error rate in their releases. >"And they didn't have one yet in 1.5 million lines of Rust" I assume that those 1.5 "million lines" do not contain any unsafe features Rust allows for. Am I correct here?
- Yoric 4y ago
  > I assume that those 1.5 "million lines" do not contain any unsafe features Rust allows for. Am I correct here? I would assume that there are a few `unsafe` lines of code in these 1.5 MLoC. Which still makes it easier to trust than 1.5MLoC of C++ in which (by Rust definitions of `unsafe`) every line makes use of `unsafe` features. (insert usual caveats about tradeoffs, etc.)
- zapnuk 4y ago
  > One memory error per 1000 LOC? From his link: "Historical vulnerability density is greater than 1/kLOC (1 vulnerability per thousand lines of code) in many of Android’s C/C++ components (e.g. media, Bluetooth, NFC, etc). Based on this historical vulnerability density, it’s likely that using Rust has already prevented hundreds of vulnerabilities from reaching production."
  - FpUser 4y ago
    As already said this error rate for released code seems way too high. No idea what they're doing internally but they definitely doing something wrong. Probably sticking to old style with lots of explicit memory management.
    - pjmlp 4y ago
      Yeah, but that is the thing, most C++ shops write code exactly like that, regardless of what a few iluminated souls write about modern C++ and static analysis best pratices. Not even Bjarne's continuous preaching seems to affect current practices, as most folks don't do social activities regarding their day job programming languages.
pvitz 4y ago
I am not sure if the comparison is correct. Basically, they are looking at the Android code base and calculate 1 vulnerability / kLoc for C++. This doesn't tell us anything about the distribution over time. It could be that Google limited itself to certain C++ features in the beginning of Android and had or has a lot of vulnerabilities from this era, whereas newer code using smart pointers etc. doesn't suffer from this issues (as much). It would be more interesting to compare only the code of C++ and Rust written in the same timeframe. My expectation is that C++ wouldn't look as bad as stated in the article.
- UncleMeat 4y ago
  People have investigated this too. Smart pointers, even the ones created specifically for Chromium, have not eliminated these problems. You can happily find CVEs whose root cause is a UAF on an object managed by unique_ptr in the Chromium codebase.
  - pvitz 4y ago
    I would be really interested in seeing such a case where there was no extraction of a raw pointer or similar involved. Could you please point me to such a CVE?
    - UncleMeat 4y ago
      Of course there is extraction of a raw pointer or reference. But that's a perfectly normal way of writing C++. An object that owns something hands a non-owning pointer to another thing with the assumption that the owning object will outlive the thing that uses the pointer. But people make mistakes and fuck up lifetimes and then you have a UAF. If you never call .get() on a unique_ptr then you'll be okay. But that's often not a workable design.
      - simplotek 4y ago
        > Of course there is extraction of a raw pointer or reference. But that's a perfectly normal way of writing C++. No, it isn't. That's flagged by default settings of most static code analysis tools, and it requires explicitly going against the whole point of explicitly assigning ownership of an object.
nine_k 4y ago
Any improvements to C++'s safety are welcome, because there are huge and important existing C++ code bases. Using C++ for a greenfield project today should require some very good justification though.
- carlmr 4y ago
  Exactly, For small-ish tools RiiR works, but not for large proven-in-use codebases. While a rewrite would probably squash memory errors you would often get logic errors because you forgot to port some condition. I'm not against improvements to C++ safety, I just think making C++ a safe language has provably failed.
  - zozbot234 4y ago
    Real-world RiiR efforts generally start with smallish, security-critical modules within larger code bases. It's not a full rewrite from scratch, you can replace C/C++ with Rust even at the level of a single function.
    - carlmr 4y ago
      That I'm completely on board with, you need to start somewhere.
  - nine_k 4y ago
    Not rewriting, but starting anew. And usually you start (relatively) small. I see fewer and fewer reasons to start a new C++ project, except when the project makes heavy use of existing C++ code base anyway as large C++ libraries.
- simplotek 4y ago
  > Using C++ for a greenfield project today should require some very good justification though. Having to maintain legacy C++ applications is all the justification you need to have to understand that staring greenfield projects in C++ is quite frequently and by far the best option on the table It makes absolutely no sense at all to multiply the tech stacks that a team needs to work on, let alone master, just because naive platitudes have been said about any novel tech stack. You do not make any application better or more secure if you get clueless developers onboarding onto new technologies and forcing them to maintain N+1 code bases and 2*n bindings. This irrational dislike of C++ ignores the fact that adopting the latest C++ release and toggling static code analysis in your project, which nowadays comes for free, already gets any project way ahead of where naive retractors believe C++ is.
DrBazza 4y ago
Is that 1 in 1000 for all C++ written since 1985, or is it "like for like" and all C++ written since 2010 when Rust first appeared or later when Rust stabilised? Chrome was 2008. No idea if the code base has been completely "modernised" or not.
blub 4y ago
If you want to use statistics to support your assertion that C++ hasn’t gotten better, safer and more efficient over the decades then your statistics must compare different versions of C++, not C++ and Rust. The blog you linked to is a PR piece. A real study would go in-depth and look at the quality of the C and C++, particularities of the affected subsystems (such as a requirement to interact directly with HW, etc), split findings by subsystem and components and also link snippets of source code to CVEs. Hopefully Google will publish such a study someday instead of a few pieces of data and graphs. But that’s not the major issue. You’re taking as fact that Google’s Android C++ programmers are particularly interested in safety or are significantly more competent than average. Based on my occasional perusal of Android C++ code in a professional context I cannot confirm either assumption: Android C++ is not modern (not even C++11 modern) and is very pointer heavy. It’s an eclectic mix of C functions and C data types with C++ constructs. Take a look at some random fixes for CVEs that I took from the December security bulletin: * https://android.googlesource.com/platform/frameworks/minikin/+/b215af1ecb2d5e9cec23444978fccc72d3821c98%5E%21/#F0 https://android.googlesource.com/platform/frameworks/minikin... * https://android.googlesource.com/platform/packages/modules/Bluetooth/+/6b4acc4d439bf6e66c520819de068eb486724e05%5E%21/#F0 https://android.googlesource.com/platform/packages/modules/B... * https://android.googlesource.com/platform/packages/modules/Bluetooth/+/644f250acd25ef47950c39349eea6fbfbdd41c14%5E%21/#F0 https://android.googlesource.com/platform/packages/modules/B... That looks like C with classes to me. Perhaps it’s for the best if projects such as Android or Chrome, etc are rewritten in Rust. Catching stuff after the fact with sanitizers, fuzzing, etc will never work if the code is ignoring or can’t use most of the advances that Stroustrup was talking about.
- pjmlp 4y ago
  Google's code bases (and Microsoft as well), are a good example of even ISO C++ members, major contributors to C++ compiler ecosystem and static analysis tooling, still fail to achieve the security standards that Bjarne Stroustrup and other security minded folks keep advocating for. Mostly because in the end they aren't doing what gets preached in spite of the in-house experts. So how much hope have the regular C++ devs that don't take part in ISO C++ and compiler development tasks?

by carlmr 4y ago

menio 4y ago

Thanks for sharing this. Wow, its a really suprising statistics to see 0 vuln found in 1.5m loc, even with Rust.

carlmr 4y ago
I personally expect there are still some unfound memory errors in there, but it's such a stark contrast to C++ it's amazing. However memory errors in C++ account for 70% of vulnerabilities. I'd expect 30% to thus remain. Edit: meant to write memory errors not vulns.
- nindalf 4y ago
  It's new code, there must be some logic errors, off-by-one errors, assumptions not expressed in the code etc. But that's not saying much, all new code has issues. The good thing is that since this code was written in a memory safe language, at least that eliminates an entire class of bugs.
pdw 4y ago
Zero memory errors, not zero vulnerabilities. In high-quality C and C++ projects, about 70% of vulnerabilities are caused by memory errors. (https://alexgaynor.net/2020/may/27/science-on-memory-unsafety-and-security/ https://alexgaynor.net/2020/may/27/science-on-memory-unsafet...)
GuB-42 4y ago
Not zero vulnerabilities, zero memory errors. Memory errors make a good share of vulnerabilities, but not all vulnerabilities are memory errors. And not all memory errors are vulnerabilities, sometimes it is just an inexploitable bug. It is to be expected that there is no bug in the very thing that Rust is supposed to prevent. The same can be said of Java, JS or even PHP. What surprises me a little though is that there is zero memory error even in unsafe Rust. There has to be some unsafe code somewhere, for low level data structures and things like that.
- carlmr 4y ago
  >What surprises me a little though is that there is zero memory error even in unsafe Rust. There has to be some unsafe code somewhere, for low level data structures and things like that. Even unsafe Rust has more strict typing and borrow checking which C++ doesn't have. It does allow raw pointers, but it's still safer than C++. The other thing is that if say 1% of your loc is unsafe Rust, it's much easier to put this under more scrutiny than the whole C++ codebase. And 1% is a lot if you ask me. I almost never need unsafe at all. >There has to be some unsafe code somewhere, for low level data structures and things like that. In most cases you can just use the standard library code for these. They do contain unsafe of course, but it's in one of the most battle tested places.

TigerTeamX 4y ago

I just cringed when I saw C++ and safety in one sentence. This is good statistics to showcase gut feeling, thanks.

72deluxe 4y ago
I presume you're using an operating system and browser written in Rust? If not, it would seem that the C/C++ based operating systems and browsers that you are use are perfectly safe to use, no??
- carlmr 4y ago
  Those OSes and browsers do show memory safety issues time and time again. They are some of the most hardened codebases, and still fail in light of the issues with C and C++.
  - TigerTeamX 4y ago
    Exactly.
  - 72deluxe 4y ago
    The languages are not at fault. The fallible users of them are. This is identical to me writing a garbage sentence in English and then proclaiming that English as a language is wrong and needs to be replaced because it's perfectly possible to write something grammatically incorrect in it. Surely you can see the difference but are conflating use of the language with the language itself.
- UncleMeat 4y ago
  Published last friday - https://seclists.org/oss-sec/2023/q1/20 https://seclists.org/oss-sec/2023/q1/20 Oodles of vulns in linux that are made possible by C's lack of safety guarantees.
  - simplotek 4y ago
    > Oodles of vulns in linux that are made possible by C's lack of safety guarantees. I'm sorry, isn't your link about how a user found a memory error in the Linux codebase? It seems way farfetched to jump to your conclusion from that starting point. I mean, doesn't Linux have over 12 million lines of code right now?

FpUser 4y ago

>"1 memory error every 1000 lines" One memory error per 1000 LOC? I think they probably either use old style C++ or something else weird and ignore all static and runtime memory safety tooling to get to this error rate in their releases. >"And they didn't have one yet in 1.5 million lines of Rust" I assume that those 1.5 "million lines" do not contain any unsafe features Rust allows for. Am I correct here?

Yoric 4y ago
> I assume that those 1.5 "million lines" do not contain any unsafe features Rust allows for. Am I correct here? I would assume that there are a few `unsafe` lines of code in these 1.5 MLoC. Which still makes it easier to trust than 1.5MLoC of C++ in which (by Rust definitions of `unsafe`) every line makes use of `unsafe` features. (insert usual caveats about tradeoffs, etc.)
zapnuk 4y ago
> One memory error per 1000 LOC? From his link: "Historical vulnerability density is greater than 1/kLOC (1 vulnerability per thousand lines of code) in many of Android’s C/C++ components (e.g. media, Bluetooth, NFC, etc). Based on this historical vulnerability density, it’s likely that using Rust has already prevented hundreds of vulnerabilities from reaching production."
- FpUser 4y ago
  As already said this error rate for released code seems way too high. No idea what they're doing internally but they definitely doing something wrong. Probably sticking to old style with lots of explicit memory management.
  - pjmlp 4y ago
    Yeah, but that is the thing, most C++ shops write code exactly like that, regardless of what a few iluminated souls write about modern C++ and static analysis best pratices. Not even Bjarne's continuous preaching seems to affect current practices, as most folks don't do social activities regarding their day job programming languages.

pvitz 4y ago

I am not sure if the comparison is correct. Basically, they are looking at the Android code base and calculate 1 vulnerability / kLoc for C++. This doesn't tell us anything about the distribution over time. It could be that Google limited itself to certain C++ features in the beginning of Android and had or has a lot of vulnerabilities from this era, whereas newer code using smart pointers etc. doesn't suffer from this issues (as much). It would be more interesting to compare only the code of C++ and Rust written in the same timeframe. My expectation is that C++ wouldn't look as bad as stated in the article.

UncleMeat 4y ago
People have investigated this too. Smart pointers, even the ones created specifically for Chromium, have not eliminated these problems. You can happily find CVEs whose root cause is a UAF on an object managed by unique_ptr in the Chromium codebase.
- pvitz 4y ago
  I would be really interested in seeing such a case where there was no extraction of a raw pointer or similar involved. Could you please point me to such a CVE?
  - UncleMeat 4y ago
    Of course there is extraction of a raw pointer or reference. But that's a perfectly normal way of writing C++. An object that owns something hands a non-owning pointer to another thing with the assumption that the owning object will outlive the thing that uses the pointer. But people make mistakes and fuck up lifetimes and then you have a UAF. If you never call .get() on a unique_ptr then you'll be okay. But that's often not a workable design.
    - simplotek 4y ago
      > Of course there is extraction of a raw pointer or reference. But that's a perfectly normal way of writing C++. No, it isn't. That's flagged by default settings of most static code analysis tools, and it requires explicitly going against the whole point of explicitly assigning ownership of an object.

nine_k 4y ago

Any improvements to C++'s safety are welcome, because there are huge and important existing C++ code bases. Using C++ for a greenfield project today should require some very good justification though.

carlmr 4y ago
Exactly, For small-ish tools RiiR works, but not for large proven-in-use codebases. While a rewrite would probably squash memory errors you would often get logic errors because you forgot to port some condition. I'm not against improvements to C++ safety, I just think making C++ a safe language has provably failed.
- zozbot234 4y ago
  Real-world RiiR efforts generally start with smallish, security-critical modules within larger code bases. It's not a full rewrite from scratch, you can replace C/C++ with Rust even at the level of a single function.
  - carlmr 4y ago
    That I'm completely on board with, you need to start somewhere.
- nine_k 4y ago
  Not rewriting, but starting anew. And usually you start (relatively) small. I see fewer and fewer reasons to start a new C++ project, except when the project makes heavy use of existing C++ code base anyway as large C++ libraries.
simplotek 4y ago
> Using C++ for a greenfield project today should require some very good justification though. Having to maintain legacy C++ applications is all the justification you need to have to understand that staring greenfield projects in C++ is quite frequently and by far the best option on the table It makes absolutely no sense at all to multiply the tech stacks that a team needs to work on, let alone master, just because naive platitudes have been said about any novel tech stack. You do not make any application better or more secure if you get clueless developers onboarding onto new technologies and forcing them to maintain N+1 code bases and 2*n bindings. This irrational dislike of C++ ignores the fact that adopting the latest C++ release and toggling static code analysis in your project, which nowadays comes for free, already gets any project way ahead of where naive retractors believe C++ is.

DrBazza 4y ago

Is that 1 in 1000 for all C++ written since 1985, or is it "like for like" and all C++ written since 2010 when Rust first appeared or later when Rust stabilised? Chrome was 2008. No idea if the code base has been completely "modernised" or not.

blub 4y ago

If you want to use statistics to support your assertion that C++ hasn’t gotten better, safer and more efficient over the decades then your statistics must compare different versions of C++, not C++ and Rust. The blog you linked to is a PR piece. A real study would go in-depth and look at the quality of the C and C++, particularities of the affected subsystems (such as a requirement to interact directly with HW, etc), split findings by subsystem and components and also link snippets of source code to CVEs. Hopefully Google will publish such a study someday instead of a few pieces of data and graphs. But that’s not the major issue. You’re taking as fact that Google’s Android C++ programmers are particularly interested in safety or are significantly more competent than average. Based on my occasional perusal of Android C++ code in a professional context I cannot confirm either assumption: Android C++ is not modern (not even C++11 modern) and is very pointer heavy. It’s an eclectic mix of C functions and C data types with C++ constructs. Take a look at some random fixes for CVEs that I took from the December security bulletin: * https://android.googlesource.com/platform/frameworks/minikin/+/b215af1ecb2d5e9cec23444978fccc72d3821c98%5E%21/#F0 https://android.googlesource.com/platform/frameworks/minikin... * https://android.googlesource.com/platform/packages/modules/Bluetooth/+/6b4acc4d439bf6e66c520819de068eb486724e05%5E%21/#F0 https://android.googlesource.com/platform/packages/modules/B... * https://android.googlesource.com/platform/packages/modules/Bluetooth/+/644f250acd25ef47950c39349eea6fbfbdd41c14%5E%21/#F0 https://android.googlesource.com/platform/packages/modules/B... That looks like C with classes to me. Perhaps it’s for the best if projects such as Android or Chrome, etc are rewritten in Rust. Catching stuff after the fact with sanitizers, fuzzing, etc will never work if the code is ignoring or can’t use most of the advances that Stroustrup was talking about.

pjmlp 4y ago
Google's code bases (and Microsoft as well), are a good example of even ISO C++ members, major contributors to C++ compiler ecosystem and static analysis tooling, still fail to achieve the security standards that Bjarne Stroustrup and other security minded folks keep advocating for. Mostly because in the end they aren't doing what gets preached in spite of the in-house experts. So how much hope have the regular C++ devs that don't take part in ISO C++ and compiler development tasks?