3 ms·

I don't know how Coverity works - is it possible that surrounding it with more complex code will cause it to miss the bug because it doesn't have enough informa

I don't know how Coverity works - is it possible that surrounding it with more complex code will cause it to miss the bug because it doesn't have enough information to not constantly raise false positives?

fournm 13y ago
I'm not familiar with Coverity, but Sonar (for the Java world) is very similar in concept and will fail to detect things if there are enough issues before it in a file... Which I would HOPE is not the case here.
- ajanuary 13y ago
  From my experience with static analysis tools it's very easy to write perfectly valid code that the tool doesn't like, so it wouldn't surprise me too much. Of course, the argument can often be made that if it isn't clear enough for the tool to find, it's not clear enough for a person to understand quickly.
  - phazmatis 13y ago
    So you refactor until the noise goes away. Why throw out the signal with the noise? Modern software engineering theory 101.
    - ajanuary 13y ago
      I absolutely agree it would be beneficial for static analysis tools to regularly be run on openssl. My stance (which I made clearer elsewhere in the thread) is more along the lines of: if it's not being done by the core maintainers, but just by concerned third parties, it's very easy to lose the signal in noise you don't have the ability to refactor away (because of time, difficulty getting it merged upstream etc.) So I'm not surprised that given the context it was missed by people running static analysis over it. That context is wrong, and it should have changed a long time ago, but under that context I can see it getting missed [1]. [1] By interpreting static analysis results, not necessarily by the code author and reviewer.
notacoward 13y ago
To put it briefly: no. A little less briefly, then. ;) It is absolutely possible to create code that these tools in general, and Coverity in particular, will have difficulty analyzing . . . but you really have to work at it. Seriously, these guys are good. It's a bit like the "arms race" in building vs. breaking crypto. These guys have been there, they've seen all the moves, they know all the countermoves. Sure, if you load up your code with runtime-assigned function pointers and code that only executes if the last five iterations of a loop each went through specific code paths themselves, then that's going to cause some problems, but most programmers are unlikely to "win" that battle. However, this particular bug looks like it's in the absolute easiest category. Any static analyzer should have caught it. As others have pointed out, the real problem is false positives. If it was caught, but the report was buried in hundreds or thousands of crappy reports about things that actually aren't problems, then it might as well not have been caught. That's why the pros at this spend as much time writing code to eliminate false positives as they do writing code to find new things. In every project I've worked on that used static analysis, the weak link in the chain has been between reporting and remedy, not in the analysis itself.

fournm 13y ago

I'm not familiar with Coverity, but Sonar (for the Java world) is very similar in concept and will fail to detect things if there are enough issues before it in a file... Which I would HOPE is not the case here.

ajanuary 13y ago
From my experience with static analysis tools it's very easy to write perfectly valid code that the tool doesn't like, so it wouldn't surprise me too much. Of course, the argument can often be made that if it isn't clear enough for the tool to find, it's not clear enough for a person to understand quickly.
- phazmatis 13y ago
  So you refactor until the noise goes away. Why throw out the signal with the noise? Modern software engineering theory 101.
  - ajanuary 13y ago
    I absolutely agree it would be beneficial for static analysis tools to regularly be run on openssl. My stance (which I made clearer elsewhere in the thread) is more along the lines of: if it's not being done by the core maintainers, but just by concerned third parties, it's very easy to lose the signal in noise you don't have the ability to refactor away (because of time, difficulty getting it merged upstream etc.) So I'm not surprised that given the context it was missed by people running static analysis over it. That context is wrong, and it should have changed a long time ago, but under that context I can see it getting missed [1]. [1] By interpreting static analysis results, not necessarily by the code author and reviewer.

notacoward 13y ago

To put it briefly: no. A little less briefly, then. ;) It is absolutely possible to create code that these tools in general, and Coverity in particular, will have difficulty analyzing . . . but you really have to work at it. Seriously, these guys are good. It's a bit like the "arms race" in building vs. breaking crypto. These guys have been there, they've seen all the moves, they know all the countermoves. Sure, if you load up your code with runtime-assigned function pointers and code that only executes if the last five iterations of a loop each went through specific code paths themselves, then that's going to cause some problems, but most programmers are unlikely to "win" that battle. However, this particular bug looks like it's in the absolute easiest category. Any static analyzer should have caught it. As others have pointed out, the real problem is false positives. If it was caught, but the report was buried in hundreds or thousands of crappy reports about things that actually aren't problems, then it might as well not have been caught. That's why the pros at this spend as much time writing code to eliminate false positives as they do writing code to find new things. In every project I've worked on that used static analysis, the weak link in the chain has been between reporting and remedy, not in the analysis itself.