5 ms·

Coverity Scan Update

49 points by fcambus 8y ago

kstrauser 8y ago
I wonder who the hosting provider was. I'm not seeing much in the news about one that "unexpectedly ceased operations", just the expected background news of scattered outages.
- westi 8y ago
  Based on DNS history for the domain it looks like it was https://twitter.com/nephoscale https://twitter.com/nephoscale | http://nephoscale.com/ http://nephoscale.com/
  - kstrauser 8y ago
    Wild! I wonder why they picked a host that I'd literally never heard of before this moment? Not that I claim encyclopedic knowledge of virtual hosting providers, but still.
    - dsl 8y ago
      Coverity was an acquisition. The hosting company was probably ran by a friend of the founders (both seem to be based in the bay area).
  - jordanthoms 8y ago
    Weird... seems they just disappeared with no announcement at all?
sunyc 8y ago
I honestly thought it is gone! All links are dead, and synopsis.com’s big Corp style website isn’t helping one bit.
joshstrange 8y ago
> Coverity Scan is a free static code analysis tool for Java, C, C++, C# and JavaScript. It analyzes every line of code and potential execution path and produces a list of potential code defects. There we go, I had no clue what this even was. Do a lot of people here use it?
- radicalbyte 8y ago
  Coverity have one of the best static analyzers for C++ and CSharp. Not a surprise considering they have had* ex-Microsoft compiler engineers such as Eric Lippert working for them. I understand from speaking to C++ engineers who have extensive experience in embedded / industrial applications that Coverity is used extensively there. Personally I've never seen the need to apply it to CSharp projects because the language is naturally safer than C++ and you get a lot of "bang for the buck" by using the tools built into Visual Studio and from JetBrains. * He works at Facebook now :facepalm:
- sanxiyn 8y ago
  "A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World" (2010) is a classic popular level account of Coverity. It is very funny, in an unintentional way. Highly recommended reading. https://cacm.acm.org/magazines/2010/2/69354-a-few-billion-lines-of-code-later/fulltext https://cacm.acm.org/magazines/2010/2/69354-a-few-billion-li...
- forgottenpass 8y ago
  We pay for Coverity for our closed source products and are quite happy with it. But you have to have a project committed to staying on top of static analysis to really get the value out of it. Our product domain means we do, and for all our static analysis tools we have a fix-or-document rule where we have to address every warning before a release. Coverity is great for us.
- KindOne 8y ago
  I have access for two opensource programs using it (bitchx and znc). The webui is a bit like a full blown IDE. It has found a few bugs (some are CVE's) in various projects. https://github.com/znc/znc/search?p=1&q=Coverity&type=Commits https://github.com/znc/znc/search?p=1&q=Coverity&type=Commit... https://github.com/bitchx/bitchx/search?p=1&q=Coverity&type=Commits https://github.com/bitchx/bitchx/search?p=1&q=Coverity&type=...
walterbell 8y ago
Has anyone tried LGTM / Semmle QL for automated code review? They claim 100K OSS projects are using the service. It's a bit hard to find technical information on the product, but they have found CVEs in mainstream products, including iOS. https://lgtm.com https://lgtm.com & https://semmle.com/ql https://semmle.com/ql
- spatulon 8y ago
  I work on C/C++ analysis at Semmle, and am happy to answer any questions you might have. (We also support C#, Java, JavaScript, Python... and Cobol!) We have a few high-profile projects using our automated code review (marketing people tell me I'm not allowed to call it 'PR integration' any more). One example is on the AMP Project, where we caught a regex injection vulnerability in a PR before a human looked at it: https://github.com/ampproject/amphtml/pull/13060 https://github.com/ampproject/amphtml/pull/13060 Our default analysis has found a few other vulnerabilities (remote buffer overflows due to misuse of snprintf in rsyslog and Icecast spring to mind) but, honestly, I think our strength lies in the fact that you can write custom queries that find bugs specific to a single codebase's foibles. For example, my first ever CVE was for a vulnerability in ChakraCore. Google Project Zero found the original bug - type confusion caused by failure to check a flag indicating that the last element of a list should be cast to a different type - but we wrote a query to verify that code accessing that particular list always checked the flag. So when some new code got introduced with the same bug, we noticed as soon as we re-ran the query on the new commit.
  - walterbell 8y ago
    Is there a public language reference on the query language? The Help section of the website only has intro examples. How long does it take for a new developer to learn to write non-trivial queries? Any technical comparisons with other static analysis tools? It would be useful to have a public database of QL sample queries alongside matching OSS code snippets. In theory, this could be constructed from Github, but presumably LGTM already has that historical viewpoint.
    - 8y ago
sanxiyn 8y ago
Coverity is really good. It is a pity some of its advances, effective in practice but not really "publishable", will forever remain as proprietary secret. Source: I worked on static code analysis product and we extensively black-box tested Coverity.
- tacostakohashi 8y ago
  What kind of advances are you thinking of? As best I can tell, most of the warnings are either things that can be figured out for a single translation unit and (some) compilers will eventually incorporate as warnings, or things that can only be figured out by analyzing / linking across many translation units - which a compiler can't do, but the actual "advance" is simple enough if you have all the function definitions to hand.
  - sanxiyn 8y ago
    Most advances are about filtering false positives. If you do "simple" interprocedural analysis and dump the result you will find lots of bugs together with lots of false positives and you can't sell that. In other words, all the advances are in warnings you don't see. Yes, all Coverity warnings you see, are simple. I agree. Quoting from https://cacm.acm.org/magazines/2010/2/69354-a-few-billion-lines-of-code-later/fulltext https://cacm.acm.org/magazines/2010/2/69354-a-few-billion-li... > Since the analysis that suppresses false positives is invisible (it removes error messages rather than generates them) its sophistication has scaled far beyond what our research system did. On the other hand, the commercial Coverity product, despite its improvements, lags behind the research system in some ways because it had to drop checkers or techniques that demand too much sophistication on the part of the user.
    - jetru 8y ago
      :) Cool that you recognize that. This is correct. Coverity does a bunch of specific analyses designed to eliminate False positives. This includes analysis to determine which data states are not possible for a given code path, and doesn't report those specific issues. This also works using data across function calls. For C/C++/C#, Coverity has by far the lowest false positive rates, which make it probably the best in class for those languages. Disclosure: I used to work at Coverity.
danielhochman 8y ago
Coverity Scan regularly goes down for hours or days. In February of 2018 it was down for over a month with no word or ETA on when it would be fixed. I hadn't thought about it since then (we discontinued use), but researching it now they released a statement saying that it was hacked. There was not a single status update during the outage. https://www.theregister.co.uk/2018/03/19/coverity_scan_cryptomining/ https://www.theregister.co.uk/2018/03/19/coverity_scan_crypt...
rurban 8y ago
Wouldn't it be great if professional websites will someday get to the level of non-professional websites? E.g. by giving this announcement page a proper title: "Coverity Scan Outage". Update is a change, this is an outage.

49 points by fcambus 8y ago

I wonder who the hosting provider was. I'm not seeing much in the news about one that "unexpectedly ceased operations", just the expected background news of scattered outages.

westi 8y ago
Based on DNS history for the domain it looks like it was https://twitter.com/nephoscale https://twitter.com/nephoscale | http://nephoscale.com/ http://nephoscale.com/
- kstrauser 8y ago
  Wild! I wonder why they picked a host that I'd literally never heard of before this moment? Not that I claim encyclopedic knowledge of virtual hosting providers, but still.
  - dsl 8y ago
    Coverity was an acquisition. The hosting company was probably ran by a friend of the founders (both seem to be based in the bay area).
- jordanthoms 8y ago
  Weird... seems they just disappeared with no announcement at all?

sunyc 8y ago

I honestly thought it is gone! All links are dead, and synopsis.com’s big Corp style website isn’t helping one bit.

joshstrange 8y ago

> Coverity Scan is a free static code analysis tool for Java, C, C++, C# and JavaScript. It analyzes every line of code and potential execution path and produces a list of potential code defects. There we go, I had no clue what this even was. Do a lot of people here use it?

radicalbyte 8y ago
Coverity have one of the best static analyzers for C++ and CSharp. Not a surprise considering they have had* ex-Microsoft compiler engineers such as Eric Lippert working for them. I understand from speaking to C++ engineers who have extensive experience in embedded / industrial applications that Coverity is used extensively there. Personally I've never seen the need to apply it to CSharp projects because the language is naturally safer than C++ and you get a lot of "bang for the buck" by using the tools built into Visual Studio and from JetBrains. * He works at Facebook now :facepalm:
sanxiyn 8y ago
"A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World" (2010) is a classic popular level account of Coverity. It is very funny, in an unintentional way. Highly recommended reading. https://cacm.acm.org/magazines/2010/2/69354-a-few-billion-lines-of-code-later/fulltext https://cacm.acm.org/magazines/2010/2/69354-a-few-billion-li...
forgottenpass 8y ago
We pay for Coverity for our closed source products and are quite happy with it. But you have to have a project committed to staying on top of static analysis to really get the value out of it. Our product domain means we do, and for all our static analysis tools we have a fix-or-document rule where we have to address every warning before a release. Coverity is great for us.
KindOne 8y ago
I have access for two opensource programs using it (bitchx and znc). The webui is a bit like a full blown IDE. It has found a few bugs (some are CVE's) in various projects. https://github.com/znc/znc/search?p=1&q=Coverity&type=Commits https://github.com/znc/znc/search?p=1&q=Coverity&type=Commit... https://github.com/bitchx/bitchx/search?p=1&q=Coverity&type=Commits https://github.com/bitchx/bitchx/search?p=1&q=Coverity&type=...

walterbell 8y ago

Has anyone tried LGTM / Semmle QL for automated code review? They claim 100K OSS projects are using the service. It's a bit hard to find technical information on the product, but they have found CVEs in mainstream products, including iOS. https://lgtm.com https://lgtm.com & https://semmle.com/ql https://semmle.com/ql

spatulon 8y ago
I work on C/C++ analysis at Semmle, and am happy to answer any questions you might have. (We also support C#, Java, JavaScript, Python... and Cobol!) We have a few high-profile projects using our automated code review (marketing people tell me I'm not allowed to call it 'PR integration' any more). One example is on the AMP Project, where we caught a regex injection vulnerability in a PR before a human looked at it: https://github.com/ampproject/amphtml/pull/13060 https://github.com/ampproject/amphtml/pull/13060 Our default analysis has found a few other vulnerabilities (remote buffer overflows due to misuse of snprintf in rsyslog and Icecast spring to mind) but, honestly, I think our strength lies in the fact that you can write custom queries that find bugs specific to a single codebase's foibles. For example, my first ever CVE was for a vulnerability in ChakraCore. Google Project Zero found the original bug - type confusion caused by failure to check a flag indicating that the last element of a list should be cast to a different type - but we wrote a query to verify that code accessing that particular list always checked the flag. So when some new code got introduced with the same bug, we noticed as soon as we re-ran the query on the new commit.
- walterbell 8y ago
  Is there a public language reference on the query language? The Help section of the website only has intro examples. How long does it take for a new developer to learn to write non-trivial queries? Any technical comparisons with other static analysis tools? It would be useful to have a public database of QL sample queries alongside matching OSS code snippets. In theory, this could be constructed from Github, but presumably LGTM already has that historical viewpoint.
  - 8y ago

sanxiyn 8y ago

Coverity is really good. It is a pity some of its advances, effective in practice but not really "publishable", will forever remain as proprietary secret. Source: I worked on static code analysis product and we extensively black-box tested Coverity.

tacostakohashi 8y ago
What kind of advances are you thinking of? As best I can tell, most of the warnings are either things that can be figured out for a single translation unit and (some) compilers will eventually incorporate as warnings, or things that can only be figured out by analyzing / linking across many translation units - which a compiler can't do, but the actual "advance" is simple enough if you have all the function definitions to hand.
- sanxiyn 8y ago
  Most advances are about filtering false positives. If you do "simple" interprocedural analysis and dump the result you will find lots of bugs together with lots of false positives and you can't sell that. In other words, all the advances are in warnings you don't see. Yes, all Coverity warnings you see, are simple. I agree. Quoting from https://cacm.acm.org/magazines/2010/2/69354-a-few-billion-lines-of-code-later/fulltext https://cacm.acm.org/magazines/2010/2/69354-a-few-billion-li... > Since the analysis that suppresses false positives is invisible (it removes error messages rather than generates them) its sophistication has scaled far beyond what our research system did. On the other hand, the commercial Coverity product, despite its improvements, lags behind the research system in some ways because it had to drop checkers or techniques that demand too much sophistication on the part of the user.
  - jetru 8y ago
    :) Cool that you recognize that. This is correct. Coverity does a bunch of specific analyses designed to eliminate False positives. This includes analysis to determine which data states are not possible for a given code path, and doesn't report those specific issues. This also works using data across function calls. For C/C++/C#, Coverity has by far the lowest false positive rates, which make it probably the best in class for those languages. Disclosure: I used to work at Coverity.

danielhochman 8y ago

Coverity Scan regularly goes down for hours or days. In February of 2018 it was down for over a month with no word or ETA on when it would be fixed. I hadn't thought about it since then (we discontinued use), but researching it now they released a statement saying that it was hacked. There was not a single status update during the outage. https://www.theregister.co.uk/2018/03/19/coverity_scan_cryptomining/ https://www.theregister.co.uk/2018/03/19/coverity_scan_crypt...

rurban 8y ago

Wouldn't it be great if professional websites will someday get to the level of non-professional websites? E.g. by giving this announcement page a proper title: "Coverity Scan Outage". Update is a change, this is an outage.