4 ms·

I have to wonder if Rust is putting pressure on C/++ to have more static analysis (while at the same time blazing trails in what's possible in that space, and w

by _bxg1 7y ago

I have to wonder if Rust is putting pressure on C/++ to have more static analysis (while at the same time blazing trails in what's possible in that space, and what's possible in terms of error message helpfulness). I think it's a great idea to start baking these things into the compiler, even if it will never be 100% free of false-negatives because of the limitations of what the language can express and guarantee. Still seems like a great way to eliminate a lot of common problems, as a default across the ecosystem instead of as an extra step.

thekhatribharat 7y ago
I think you mean "false-positives". Most static analysis tools are "conservative": will throw more warnings than they can prove. But assuming they're "sound" (and don't "give up" because of time/space constraints), there'd be no "false-negatives".
- UncleMeat 7y ago
  Vanishingly few (maybe even zero) industrial static analysis tools are sound. FPs are too common, especially for something like C++ where common code patterns can only be soundly handled by just forcing everything to Top. Good luck with your sound interprocedural pointer analysis as soon as somebody makes an indirect call.
  - thekhatribharat 7y ago
    I think you mean industrial static analysis tools aren't complete. Sound means no FNs, Complete means no FPs.
    - UncleMeat 7y ago
      No I meant what I said. Virtually no tool is sound (zero false negatives, lots of false positives) because in practice false positives just make people stop using tools when they are too common.
      - mgerdts 7y ago
        At my previous job, we used an internally developed static analysis tool. While false positives were a problem, the team responsible for the tool took bug reports and fixed these issues. There was a way to silence false positives as well as a mode to only see new (potential) problems. Once code was clean, the false positive rate was low enough to not really be a problem. The bar of zero false negatives is not useful for me when the alternative is to rely solely human code analysis and tests. One thing I learned from using this tool is that static analyzers need to be able to configured to know about side effects of functions that are in libraries. For example, if a library function will free memory, it is important that there's a way to tell the analyzer about this behavior to avoid false positives about leaks. Proper design, careful coding, static analysis, code review, and various forms of testing all have roles to play in delivering reliable code.
        UncleMeat 7y ago
        But you can't fix all false positives with bug reports. Its fundamental if you want soundness. And for lots of properties (like I said, pointer analysis in C++) the only scalable and sound techniques basically havoc as soon as interesting things start to happen. The community even has a term (soundiness) to describe how everybody throws out certain kinds of soundness (indirect calls through function pointers, reflection in java, tons of features in javascript, etc). Requiring the user to add piles of annotations still sucks, even if you are smart and do something like abductive inference to identify minimally required annotations. Even something as simple as the checker framework in Java has trouble getting people to write correct annotations and has a few corner cases (what is the nullness property of a field in code reachable from a constructor, for example). Nobody does this stuff correctly for something like dataflow annotations on methods. I've got a PhD in this and develop a static analysis tool for my job. I've never seen a productionized, fully sound tool for general purpose static analysis of general programs that solves problems more difficult that slightly augmented type checking.
      - thekhatribharat 7y ago
        So if industrial static analysis tools aren't sound, which means they produce false negatives (ie. they can pass programs with logical errors they're meant to detect), how do you use them to assure code quality?
  - pjmlp 7y ago
    Which is why some OS vendors just gave up and are going with hardware pointer validation nowadays. If all goes according to plan, I guess in a couple of decades there won't be any modern computer not making use of it.
pjmlp 7y ago
Internet and ramping project costs due to CVE fixes are what is putting pressure into adopting static analysis and hardware memory tagging, not Rust. https://msrc-blog.microsoft.com/2019/07/16/a-proactive-approach-to-more-secure-code/ https://msrc-blog.microsoft.com/2019/07/16/a-proactive-appro... https://security.googleblog.com/2019/08/adopting-arm-memory-tagging-extension.html https://security.googleblog.com/2019/08/adopting-arm-memory-... Lint exists since 1979.
- saagarjha 7y ago
  Hardware memory tagging is not a panacea, it just catches egregious memory errors.
  - pjmlp 7y ago
    Indeed, yet it does a better job than the mythical developer that never gets pointer arithmetic wrong.
    - saagarjha 7y ago
      Rust’ll outdo both.
      - pjmlp 7y ago
        As much as I would like to nuke C and any derivative that has copy-paste compatibility with it (C++, Objective-C,...), no one is going to rewrite Linux, BSD, OpenGL, Vulkan, DirectX, Windows,macOS, iOS, Android,.... in Rust. So I do welcome hardware memory tagging, even if isn't perfect. Apparently Google thinks the same way, given the Android is getting hardware memory tagging as requirement on ARM, with C and C++ still having more visibility than Rust on Fuchsia. Just like Microsoft is doing by pursuing research in Project Verona and CHERI CPU, Checked C (hiring now for the team), Visual C++ Core Guidelines static analyser, focus on .NET, instead of going full speed with Rust. Similar story can be told about Apple's ongoing changes.
rurban 7y ago
It's clang, not rust. And clang's analyzer has different UI concept via web, which is far superior. And for the screen valgrind has a far superior solution. I don't see the advantage of gcc's analyzer yet. Far too verbose. and the most important errors, like wrong optimizer decisions based on their interpretation of UB code are still silenced.
- TwoBit 7y ago
  I'd love to see a compiler warn me that it's doing something potentially unexpected due to UB considerations.
  - saagarjha 7y ago
    Luckily, compiler authors have gotten the message and are starting to improve on this situation slowly.
  - ali_m 7y ago
    clang has UBSan, which adds runtime checks for detecting various kinds of undefined behaviour: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
    - codys 7y ago
      gcc also has ubsan since gcc version 4.9. Many of the sanitizers are shared by the clang & gcc projects. Some functionality, like emitting errors on unsigned overflow, is confined to clang.
    - rurban 7y ago
      That's not helping. I'm talking about wrong decisions, made during compile-time optimizations, like assuming dead code or a value being NULL, and then ripping apart the written code. Without warning. Or the famous optimized away memset call. Which is a security issue. At least a warning would be in order. Or at least an analyzer warning.
  - masklinn 7y ago

by _bxg1 7y ago

thekhatribharat 7y ago

I think you mean "false-positives". Most static analysis tools are "conservative": will throw more warnings than they can prove. But assuming they're "sound" (and don't "give up" because of time/space constraints), there'd be no "false-negatives".

UncleMeat 7y ago
Vanishingly few (maybe even zero) industrial static analysis tools are sound. FPs are too common, especially for something like C++ where common code patterns can only be soundly handled by just forcing everything to Top. Good luck with your sound interprocedural pointer analysis as soon as somebody makes an indirect call.
- thekhatribharat 7y ago
  I think you mean industrial static analysis tools aren't complete. Sound means no FNs, Complete means no FPs.
  - UncleMeat 7y ago
    No I meant what I said. Virtually no tool is sound (zero false negatives, lots of false positives) because in practice false positives just make people stop using tools when they are too common.
    - mgerdts 7y ago
      At my previous job, we used an internally developed static analysis tool. While false positives were a problem, the team responsible for the tool took bug reports and fixed these issues. There was a way to silence false positives as well as a mode to only see new (potential) problems. Once code was clean, the false positive rate was low enough to not really be a problem. The bar of zero false negatives is not useful for me when the alternative is to rely solely human code analysis and tests. One thing I learned from using this tool is that static analyzers need to be able to configured to know about side effects of functions that are in libraries. For example, if a library function will free memory, it is important that there's a way to tell the analyzer about this behavior to avoid false positives about leaks. Proper design, careful coding, static analysis, code review, and various forms of testing all have roles to play in delivering reliable code.
      - UncleMeat 7y ago
        But you can't fix all false positives with bug reports. Its fundamental if you want soundness. And for lots of properties (like I said, pointer analysis in C++) the only scalable and sound techniques basically havoc as soon as interesting things start to happen. The community even has a term (soundiness) to describe how everybody throws out certain kinds of soundness (indirect calls through function pointers, reflection in java, tons of features in javascript, etc). Requiring the user to add piles of annotations still sucks, even if you are smart and do something like abductive inference to identify minimally required annotations. Even something as simple as the checker framework in Java has trouble getting people to write correct annotations and has a few corner cases (what is the nullness property of a field in code reachable from a constructor, for example). Nobody does this stuff correctly for something like dataflow annotations on methods. I've got a PhD in this and develop a static analysis tool for my job. I've never seen a productionized, fully sound tool for general purpose static analysis of general programs that solves problems more difficult that slightly augmented type checking.
    - thekhatribharat 7y ago
      So if industrial static analysis tools aren't sound, which means they produce false negatives (ie. they can pass programs with logical errors they're meant to detect), how do you use them to assure code quality?
- pjmlp 7y ago
  Which is why some OS vendors just gave up and are going with hardware pointer validation nowadays. If all goes according to plan, I guess in a couple of decades there won't be any modern computer not making use of it.

pjmlp 7y ago

Internet and ramping project costs due to CVE fixes are what is putting pressure into adopting static analysis and hardware memory tagging, not Rust. https://msrc-blog.microsoft.com/2019/07/16/a-proactive-approach-to-more-secure-code/ https://msrc-blog.microsoft.com/2019/07/16/a-proactive-appro... https://security.googleblog.com/2019/08/adopting-arm-memory-tagging-extension.html https://security.googleblog.com/2019/08/adopting-arm-memory-... Lint exists since 1979.

saagarjha 7y ago
Hardware memory tagging is not a panacea, it just catches egregious memory errors.
- pjmlp 7y ago
  Indeed, yet it does a better job than the mythical developer that never gets pointer arithmetic wrong.
  - saagarjha 7y ago
    Rust’ll outdo both.
    - pjmlp 7y ago
      As much as I would like to nuke C and any derivative that has copy-paste compatibility with it (C++, Objective-C,...), no one is going to rewrite Linux, BSD, OpenGL, Vulkan, DirectX, Windows,macOS, iOS, Android,.... in Rust. So I do welcome hardware memory tagging, even if isn't perfect. Apparently Google thinks the same way, given the Android is getting hardware memory tagging as requirement on ARM, with C and C++ still having more visibility than Rust on Fuchsia. Just like Microsoft is doing by pursuing research in Project Verona and CHERI CPU, Checked C (hiring now for the team), Visual C++ Core Guidelines static analyser, focus on .NET, instead of going full speed with Rust. Similar story can be told about Apple's ongoing changes.

rurban 7y ago

It's clang, not rust. And clang's analyzer has different UI concept via web, which is far superior. And for the screen valgrind has a far superior solution. I don't see the advantage of gcc's analyzer yet. Far too verbose. and the most important errors, like wrong optimizer decisions based on their interpretation of UB code are still silenced.

TwoBit 7y ago
I'd love to see a compiler warn me that it's doing something potentially unexpected due to UB considerations.
- saagarjha 7y ago
  Luckily, compiler authors have gotten the message and are starting to improve on this situation slowly.
- ali_m 7y ago
  clang has UBSan, which adds runtime checks for detecting various kinds of undefined behaviour: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
  - codys 7y ago
    gcc also has ubsan since gcc version 4.9. Many of the sanitizers are shared by the clang & gcc projects. Some functionality, like emitting errors on unsigned overflow, is confined to clang.
  - rurban 7y ago
    That's not helping. I'm talking about wrong decisions, made during compile-time optimizations, like assuming dead code or a value being NULL, and then ripping apart the written code. Without warning. Or the famous optimized away memset call. Which is a security issue. At least a warning would be in order. Or at least an analyzer warning.
- masklinn 7y ago