4 ms·

At my previous job, we used an internally developed static analysis tool. While false positives were a problem, the team responsible for the tool took bug repo

At my previous job, we used an internally developed static analysis tool. While false positives were a problem, the team responsible for the tool took bug reports and fixed these issues. There was a way to silence false positives as well as a mode to only see new (potential) problems. Once code was clean, the false positive rate was low enough to not really be a problem. The bar of zero false negatives is not useful for me when the alternative is to rely solely human code analysis and tests. One thing I learned from using this tool is that static analyzers need to be able to configured to know about side effects of functions that are in libraries. For example, if a library function will free memory, it is important that there's a way to tell the analyzer about this behavior to avoid false positives about leaks. Proper design, careful coding, static analysis, code review, and various forms of testing all have roles to play in delivering reliable code.

UncleMeat 7y ago
But you can't fix all false positives with bug reports. Its fundamental if you want soundness. And for lots of properties (like I said, pointer analysis in C++) the only scalable and sound techniques basically havoc as soon as interesting things start to happen. The community even has a term (soundiness) to describe how everybody throws out certain kinds of soundness (indirect calls through function pointers, reflection in java, tons of features in javascript, etc). Requiring the user to add piles of annotations still sucks, even if you are smart and do something like abductive inference to identify minimally required annotations. Even something as simple as the checker framework in Java has trouble getting people to write correct annotations and has a few corner cases (what is the nullness property of a field in code reachable from a constructor, for example). Nobody does this stuff correctly for something like dataflow annotations on methods. I've got a PhD in this and develop a static analysis tool for my job. I've never seen a productionized, fully sound tool for general purpose static analysis of general programs that solves problems more difficult that slightly augmented type checking.

UncleMeat 7y ago

But you can't fix all false positives with bug reports. Its fundamental if you want soundness. And for lots of properties (like I said, pointer analysis in C++) the only scalable and sound techniques basically havoc as soon as interesting things start to happen. The community even has a term (soundiness) to describe how everybody throws out certain kinds of soundness (indirect calls through function pointers, reflection in java, tons of features in javascript, etc). Requiring the user to add piles of annotations still sucks, even if you are smart and do something like abductive inference to identify minimally required annotations. Even something as simple as the checker framework in Java has trouble getting people to write correct annotations and has a few corner cases (what is the nullness property of a field in code reachable from a constructor, for example). Nobody does this stuff correctly for something like dataflow annotations on methods. I've got a PhD in this and develop a static analysis tool for my job. I've never seen a productionized, fully sound tool for general purpose static analysis of general programs that solves problems more difficult that slightly augmented type checking.