2 ms·

Speaking of MathWorks, I had the misfortune of having to use one of their less famous offerings (Polyspace) and it was borderline unusable with basic functional

Speaking of MathWorks, I had the misfortune of having to use one of their less famous offerings (Polyspace) and it was borderline unusable with basic functionality failing out of the box. I believe our license was $50k/year and most of the bugs simply were simply left unfixed after we reported them. This seems to be par for the course for commercial static analysis tools. The Clang family of tools, on the other hand, has served us quite well but unfortunately they don't perform some of the checks that we need to do for regulatory reasons.

dhekir 6y ago
Would you mind telling some of the kinds of checks the Clang tools do not perform? I wonder if they are mostly syntactic or if they need dataflow analyses. (That said, Polyspace has 2 tools now, Bug Finder and Code Prover, whose applications are somewhat different.)
- platinumrad 6y ago
  Clang does not check MISRA rules or calculate other metrics that are considered important in embedded industries. The vast majority of MISRA rules are either purely syntactic (governing things as trivial as variable names) or obvious to the point of uselessness (like "don't violate the One Definition Rule"). The few actually useful ones require dataflow analysis but frankly Polyspace doesn't appear to do a very good job at it. My impression was that it achieved a 0% false negative rate by way of a 99.99% false positive rate. However, we never reported any of these false positives as bugs as the reports likely would have been rejected outright. The bugs we reported were all far more clear, such as the tool refusing to compile perfectly valid code, or glitches in the user interface. I'm honestly not privy to most of the politics and no longer work in this area, but my impression was that even if Clang were to implement these checks, it would not be taken seriously by auditors as it doesn't come with a price tag or tick other boxes like generating nice PDF reports with violation counts and so on.

dhekir 6y ago

Would you mind telling some of the kinds of checks the Clang tools do not perform? I wonder if they are mostly syntactic or if they need dataflow analyses. (That said, Polyspace has 2 tools now, Bug Finder and Code Prover, whose applications are somewhat different.)

platinumrad 6y ago
Clang does not check MISRA rules or calculate other metrics that are considered important in embedded industries. The vast majority of MISRA rules are either purely syntactic (governing things as trivial as variable names) or obvious to the point of uselessness (like "don't violate the One Definition Rule"). The few actually useful ones require dataflow analysis but frankly Polyspace doesn't appear to do a very good job at it. My impression was that it achieved a 0% false negative rate by way of a 99.99% false positive rate. However, we never reported any of these false positives as bugs as the reports likely would have been rejected outright. The bugs we reported were all far more clear, such as the tool refusing to compile perfectly valid code, or glitches in the user interface. I'm honestly not privy to most of the politics and no longer work in this area, but my impression was that even if Clang were to implement these checks, it would not be taken seriously by auditors as it doesn't come with a price tag or tick other boxes like generating nice PDF reports with violation counts and so on.