4 ms·

The number of errors seem like pylint giving me errors. Some of them were like a is not a valid variable name. Maybe the numbers are like that. (Pylint that is

The number of errors seem like pylint giving me errors. Some of them were like a is not a valid variable name. Maybe the numbers are like that. (Pylint that is not configured sanely/with default config really gives you thousands of errors and warnings for a fairly large code base and some of them were not errors but coding style.) The numbers here may also be something like that. Nice blog post, may even make developers adopt something like cppcheck but most guys won't buy this.

loeg 9y ago
It's a lot like that. PVS-Studio ran against FreeBSD and produced a ton of false positives.
- AndreyKarpov 9y ago
  The reason for this is described in the article "How to find 56 potential vulnerabilities in FreeBSD code in one evening": https://www.viva64.com/en/b/0496/#ID0ENNAC https://www.viva64.com/en/b/0496/#ID0ENNAC . There it is told that this is not scary and the analyzer settings can be customized.
- jotux 9y ago
  Running any sufficiently advanced static analyzer on any large codebase will generate a ton of false positives.
  - loeg 9y ago
    So what? Qualitatively, it feels like PVS-Studio's false positive rate is higher than Coverity. And far higher than a modern free compiler's -Wall. To describe the 27,000 detections as errors is a little misleading, and I think that's what the grandparent post is discussing.
    - AndreyKarpov 9y ago
      This is not the way you think and it is not nice to mislead people. It all depends on the project. I've heard people who said that Coverity gives many false positives and Cppcheck gives few. I've heard the opposite, that it is impossible to use Cppcheck because of the huge number of false positives, but Coverity is doing great. I heard that Coverity gives more false positives than PVS-Studio. And vice versa. And so on and so forth. What is the reason for such differences? Projects have their certain styles of writing and different kinds of macros. These are macros and peculiarities of style that become the main source of false positives. This is why the first impression of using the analyzers of code depends on luck, and not on the coolness of the analyzer. If the analyzer doesn’t like a self-made my_assert() it will issue 10000 false positives. So there is no point in talking abstractly about the number of false positives. Yes, you can not get lucky and there can be a lot of false positives. However, the static code analyzers allow you to configure them. In articles I have showed many times that the simplest configuration of the analyzer can greatly reduce the number of false positives. Example: https://www.viva64.com/en/b/0496/#ID0ENNAC https://www.viva64.com/en/b/0496/#ID0ENNAC
  - irundebian 9y ago
    If you write good code, there won't be that many FPs.
szatkus 9y ago
I switched to flake8. Pylint's default config is crazy.
- pebers 9y ago
  It's not just the config; Pylint assumes it's possible to statically parse code and make some simple assumptions about things like class / module members, which only works for the most straightforward Python code. It issues constant warnings for many third-party packages - the one that broke the back of it for us was protobuf, in which it complains on every access to any one of the generated classes. In the end we moved to flake8 as well, which finds probably 90% of the issues that Pylint did but at < 1% of the false positive rate.

loeg 9y ago

It's a lot like that. PVS-Studio ran against FreeBSD and produced a ton of false positives.

AndreyKarpov 9y ago
The reason for this is described in the article "How to find 56 potential vulnerabilities in FreeBSD code in one evening": https://www.viva64.com/en/b/0496/#ID0ENNAC https://www.viva64.com/en/b/0496/#ID0ENNAC . There it is told that this is not scary and the analyzer settings can be customized.
jotux 9y ago
Running any sufficiently advanced static analyzer on any large codebase will generate a ton of false positives.
- loeg 9y ago
  So what? Qualitatively, it feels like PVS-Studio's false positive rate is higher than Coverity. And far higher than a modern free compiler's -Wall. To describe the 27,000 detections as errors is a little misleading, and I think that's what the grandparent post is discussing.
  - AndreyKarpov 9y ago
    This is not the way you think and it is not nice to mislead people. It all depends on the project. I've heard people who said that Coverity gives many false positives and Cppcheck gives few. I've heard the opposite, that it is impossible to use Cppcheck because of the huge number of false positives, but Coverity is doing great. I heard that Coverity gives more false positives than PVS-Studio. And vice versa. And so on and so forth. What is the reason for such differences? Projects have their certain styles of writing and different kinds of macros. These are macros and peculiarities of style that become the main source of false positives. This is why the first impression of using the analyzers of code depends on luck, and not on the coolness of the analyzer. If the analyzer doesn’t like a self-made my_assert() it will issue 10000 false positives. So there is no point in talking abstractly about the number of false positives. Yes, you can not get lucky and there can be a lot of false positives. However, the static code analyzers allow you to configure them. In articles I have showed many times that the simplest configuration of the analyzer can greatly reduce the number of false positives. Example: https://www.viva64.com/en/b/0496/#ID0ENNAC https://www.viva64.com/en/b/0496/#ID0ENNAC
- irundebian 9y ago
  If you write good code, there won't be that many FPs.

szatkus 9y ago

I switched to flake8. Pylint's default config is crazy.

pebers 9y ago
It's not just the config; Pylint assumes it's possible to statically parse code and make some simple assumptions about things like class / module members, which only works for the most straightforward Python code. It issues constant warnings for many third-party packages - the one that broke the back of it for us was protobuf, in which it complains on every access to any one of the generated classes. In the end we moved to flake8 as well, which finds probably 90% of the issues that Pylint did but at < 1% of the false positive rate.