5 ms·

> Sometimes it seems like Rust afficionados think that buffer overflows etc. are the main kind of bugs. Have you looked at the CVE stats of the last decade rec

by llogiq 7y ago

> Sometimes it seems like Rust afficionados think that buffer overflows etc. are the main kind of bugs. Have you looked at the CVE stats of the last decade recently? Memory errors make up around 3/4 of that. Even if Rust would make the last quartile harder (which in my experience it doesn't), it could still be worth it for many applications where you cannot afford full verification, but want to avoid your users being pwned.

nsajko 7y ago
I would bet 99% of that 3/4 (or at least the part written in C) could have been caught with testing, fuzzing, and sanitizers we have today.
- littlestymaar 7y ago
  Too bad neither Microsoft, Google or Apple ever heard about those tools…
  - z3phyr 7y ago
    Is there a stat on how much % of CVE directly pertains to Microsoft, Google or Apple products?
    - testvox 7y ago
      You could easily calculate such a stat from here. Microsoft Apple and Google are all in the top 5, with IBM and Oracle being the other two. Adobe used to be on top but with the death of flash they have been slipping. I checked out the breakdown of memory corruption/overflow bugs and its well over 50% of CVEs for MS and Apple. Google is much better with less then a quarter of their CVEs being memory related. https://www.cvedetails.com/top-50-vendors.php https://www.cvedetails.com/top-50-vendors.php Microsoft 6508/116769 = 5.5% Apple 4502/116769 = 3.8% Google 4110/116769 = 3.5% total (6508 + 4502 + 4110)/116769 = 13%
      - bluejekyll 7y ago
        Microsoft's own research on this area claims it's closer to 70%: https://www.zdnet.com/article/microsoft-70-percent-of-all-security-bugs-are-memory-safety-issues/ https://www.zdnet.com/article/microsoft-70-percent-of-all-se...
        littlestymaar 7y ago
        You're not talking about the same thing: the ggp asked how many CVE were dedicated to Apple, Microsoft or Google products (a question that doesn't make much sense here, but the gp still went on the calculation). You are talking about which proportions of thoses big corps CVE are memory-related (which is the right question to ask in this context, but not the one asked…).
  - pjmlp 7y ago
    Or even Linux, with its strict patch process, still managed to have 68% of 2018's kernel CVEs, related to memory corruption issues due to C's use.
    - deleted 7y ago
      [deleted]
- pjmlp 7y ago
  Last JetBrains questionnaire proves, once again, how little most devs care about them. "Which of the following tools do you or your team use for guideline enforcement or other code quality/analysis?" https://www.jetbrains.com/lp/devecosystem-2019/cpp/ https://www.jetbrains.com/lp/devecosystem-2019/cpp/
- deleted 7y ago
  [deleted]
- MaxBarraclough 7y ago
  There's a lot of value in those tools, but it's better to entirely eliminate the family of bugs 'by construction'.
jandrewrogers 7y ago
They were talking about bugs and correctness generally. CVEs are an extremely biased population of such things, and most types of bugs and incorrectness will never show up in a CVE. Keeping planes from crashing and bank accounts correct matter too. Rust solves a subset of memory safety problems but it is not a programming language for high assurance applications and in that context enables many other types of unsafe behavior.
- wbl 7y ago
  How many programs have you worked on that were high assurance programs?
  - jandrewrogers 7y ago
    Core infrastructure software like database kernels and protocol stacks should be implemented at least in part to high assurance standards. I've verified parts of database engines and other critical code many times with good effect, finding subtle bugs we never would have discovered in the wild and with (as expected) no defects discovered later. Most systems programming languages don't make it simple and many people don't do it but it is definitely valuable and worth doing when the economics make sense.
    - FreeFull 7y ago
      So you end up stuck with languages like Ada, where the language can prove the correctness of your code (or rather, that your code follows the specification)?
      - phkahler 7y ago
        Seems like I recently read that the Ada folks might want to borrow some concepts from Rust. To me that says both languages are working toward similar goals.
      - jandrewrogers 7y ago
        Currently modern C++ plus a ton of specialized tooling that covers much of the ground of Ada, just not built into the language. It is a balancing act to keep development from becoming unwieldy and the 80/20 rule applies. Code that is easy to verify also tends to be slow, and that is not a tradeoff you can make for some applications. No one is running something as complex as a database kernel through an end-to-end theorem prover. Design verification scales well (model checkers and similar), implementation not so much due to practical limits on what you can prove and accumulated complexity/bugs in the specification, and verification of code generation is very limited (I use the LLVM stack). Nonetheless, this gets you to a very low defect rate and it isn't like this code is being written from scratch every time. Even with a fully verified toolchain there will still be bugs. I once had a customer find a rare bug in a database engine that was ultimately caused by slight differences in behavior between microarchitectures running the same binary.

by llogiq 7y ago

nsajko 7y ago

I would bet 99% of that 3/4 (or at least the part written in C) could have been caught with testing, fuzzing, and sanitizers we have today.

littlestymaar 7y ago
Too bad neither Microsoft, Google or Apple ever heard about those tools…
- z3phyr 7y ago
  Is there a stat on how much % of CVE directly pertains to Microsoft, Google or Apple products?
  - testvox 7y ago
    You could easily calculate such a stat from here. Microsoft Apple and Google are all in the top 5, with IBM and Oracle being the other two. Adobe used to be on top but with the death of flash they have been slipping. I checked out the breakdown of memory corruption/overflow bugs and its well over 50% of CVEs for MS and Apple. Google is much better with less then a quarter of their CVEs being memory related. https://www.cvedetails.com/top-50-vendors.php https://www.cvedetails.com/top-50-vendors.php Microsoft 6508/116769 = 5.5% Apple 4502/116769 = 3.8% Google 4110/116769 = 3.5% total (6508 + 4502 + 4110)/116769 = 13%
    - bluejekyll 7y ago
      Microsoft's own research on this area claims it's closer to 70%: https://www.zdnet.com/article/microsoft-70-percent-of-all-security-bugs-are-memory-safety-issues/ https://www.zdnet.com/article/microsoft-70-percent-of-all-se...
      - littlestymaar 7y ago
        You're not talking about the same thing: the ggp asked how many CVE were dedicated to Apple, Microsoft or Google products (a question that doesn't make much sense here, but the gp still went on the calculation). You are talking about which proportions of thoses big corps CVE are memory-related (which is the right question to ask in this context, but not the one asked…).
- pjmlp 7y ago
  Or even Linux, with its strict patch process, still managed to have 68% of 2018's kernel CVEs, related to memory corruption issues due to C's use.
  - deleted 7y ago
    [deleted]
pjmlp 7y ago
Last JetBrains questionnaire proves, once again, how little most devs care about them. "Which of the following tools do you or your team use for guideline enforcement or other code quality/analysis?" https://www.jetbrains.com/lp/devecosystem-2019/cpp/ https://www.jetbrains.com/lp/devecosystem-2019/cpp/
deleted 7y ago
[deleted]
MaxBarraclough 7y ago
There's a lot of value in those tools, but it's better to entirely eliminate the family of bugs 'by construction'.

jandrewrogers 7y ago

They were talking about bugs and correctness generally. CVEs are an extremely biased population of such things, and most types of bugs and incorrectness will never show up in a CVE. Keeping planes from crashing and bank accounts correct matter too. Rust solves a subset of memory safety problems but it is not a programming language for high assurance applications and in that context enables many other types of unsafe behavior.

wbl 7y ago
How many programs have you worked on that were high assurance programs?
- jandrewrogers 7y ago
  Core infrastructure software like database kernels and protocol stacks should be implemented at least in part to high assurance standards. I've verified parts of database engines and other critical code many times with good effect, finding subtle bugs we never would have discovered in the wild and with (as expected) no defects discovered later. Most systems programming languages don't make it simple and many people don't do it but it is definitely valuable and worth doing when the economics make sense.
  - FreeFull 7y ago
    So you end up stuck with languages like Ada, where the language can prove the correctness of your code (or rather, that your code follows the specification)?
    - phkahler 7y ago
      Seems like I recently read that the Ada folks might want to borrow some concepts from Rust. To me that says both languages are working toward similar goals.
    - jandrewrogers 7y ago
      Currently modern C++ plus a ton of specialized tooling that covers much of the ground of Ada, just not built into the language. It is a balancing act to keep development from becoming unwieldy and the 80/20 rule applies. Code that is easy to verify also tends to be slow, and that is not a tradeoff you can make for some applications. No one is running something as complex as a database kernel through an end-to-end theorem prover. Design verification scales well (model checkers and similar), implementation not so much due to practical limits on what you can prove and accumulated complexity/bugs in the specification, and verification of code generation is very limited (I use the LLVM stack). Nonetheless, this gets you to a very low defect rate and it isn't like this code is being written from scratch every time. Even with a fully verified toolchain there will still be bugs. I once had a customer find a rare bug in a database engine that was ultimately caused by slight differences in behavior between microarchitectures running the same binary.