6 ms·
A couple things: - Competitors with Coverity are CodeSonar[1] and Klocwork[2]. I've not seen Klocwork output, but CodeSonar and Coverity are in the same area
by pnathan 14y ago
A couple things:
- Competitors with Coverity are CodeSonar[1] and Klocwork[2]. I've not seen Klocwork output, but CodeSonar and Coverity are in the same area of quality, with differing strengths. I can not recommend static analysis highly enough if you have a C/C++/Java/C# database. It's very expensive (well into five figures according to Carmack), but how expensive is a bug? What if you have your entire codebase checked daily for bugs? Consider the effect on your quality culture. :-)
- The fact that you are paying "well into five figures" for a tool that essentially covers up design deficiencies in your language should start sounding alarm bells in your head. The proposition more or less goes, "To have reliable C++ code in certain areas, you need a static analyzer; to gain that same advantage in Haskell costs you nothing more than GHC". Of course Haskell doesn't have certain C/C++ capabilities; but it's worth meditating on for your next application, particularly if bugs are more important than performance. N.b- I don't know the ML family enough to say one way or the other in this regard. :-)
[1] http://www.grammatech.com http://www.grammatech.com
[2] http://www.klocwork.com http://www.klocwork.com
- stonemetal 14y agoIf you don't mind MS, their code analysis tools come in the express(free) version of VS 2012.
- pnathan 14y agoPersonally, I mind MS. But that's neither here nor there. My understanding is that their tools don't do "whole program analysis". Is that incorrect (I'd love to hear that I am!)?
- stonemetal 14y agoI can't say I know enough about it to say either way, in the article Carmack says: Technically, the Microsoft tool only performs local analysis, so it should be inferior to Coverity’s global analysis, but enabling it poured out mountains of errors, far more than Coverity reported. True, there were lots of false positives, but there was also a lot of scary, scary stuff.
- eru 14y agoAccording to a very insightful, but perhaps now slightly old article (http://cacm.acm.org/magazines/2010/2/69354-a-few-billion-lines-of-code-later/fulltext http://cacm.acm.org/magazines/2010/2/69354-a-few-billion-lin...) Coverity tries--for political reasons--very hard to keep false positives down, very much at the expense of false negatives.
- pnathan 14y agoHaving seen Coverity's results, I would agree with that article. Particularly cogent are the social problems and how they play out.
- YokoZar 14y agoI'm pretty sure Coverity has multiple "tiers" of analysis, with the initial tier being the subset they're most certain of. Once you fix most of the obvious, almost-no-false-positive type stuff, then they're ready to show you the next tier.
- timtadh 14y agoOn the Java end (all though these tools also do C/C++ and other languages maybe even .NET but their speciality from my understanding is Java) is Fortify [now owned by HP] and IBM AppScan Source [previous Ounce from Ounce Labs]. Both produce high quality output in their domain. Fortify is widely use in the AppSec world.
- johnyzee 14y agoWhat do they give you that you don't get with open source tools like Clover, Findbugs and Checkstyle?
- timtadh 14y agoAccuracy. Detail. They have better analysis engines, which do more complex types of program analysis. The static program analysis field is incredibly deep it takes a lot of engineering to make a world class analysis engine. Just like it takes a lot of effort to make a world class optimising complier. (eg. the Intel C Compiler still out performs GCC despite monumental effort on GCC's side. It takes a long time to build this stuff) They also have better rules sets from what I understand than the open source tools.
- mzl 14y agoWhile the Intel compiler may be better in some/many cases, it is not universal. In my experience (a couple of years ago, C++ code, high-performance requirements), the Intel compiler was quite a bit worse than GCC for code with a lot of branches and abstraction layers. The Microsoft compiler was way better than both. Around 10% faster run-time IIRC.
- raverbashing 14y agoWhat are the main issues with Java code that these tools find?
- btilly 14y agoThe fact that you are paying "well into five figures" says more about the sales process than anything else. Any time you have dedicated sales staff selling stuff to organizations with a sales process that needs to get buy-in from executives, your prices have to go way up to cover that cost. (Doubly so because the sales process is expensive for you, and your price point lowers your success rate.) Unfortunately many organizations have a rule that every purchase over $X has to be approved by management. Thus there are 3 price points that you can find proprietary software sold at: 1. Small enough that a developer can charge it to a corporate credit card. 2. Relatively modest recurring payments, individually small enough to pass on a corporate credit card. 3. Starting well into 5 figures.
- Resident_Geek 14y agoMore detail here: http://www.joelonsoftware.com/articles/CamelsandRubberDuckies.html http://www.joelonsoftware.com/articles/CamelsandRubberDuckie...
- jerf 14y ago"To have reliable C++ code in certain areas, you need a static analyzer; to gain that same advantage in Haskell costs you nothing more than GHC" It is certainly true that many of the static errors that Coverity and friends can find aren't possible in Haskell because it won't compile, but I'd like to see a lot more public experience with Haskell and larger projects to see what the result would be for other problems. A lot of what we now know about how C++ should be written is the result of decades of hard experience, and who knows what we might find in Haskell with as much experience.
- pnathan 14y agoThe bugs that static analysis tools tend to turn up are ones that Haskell catches out of the gate in general. What Haskell brings to the table in terms of unique flavors of bug, I don't know. I would love to read articles about 1MLoC+ codebases in Haskell and how they play out.
- dons 14y agoHere are slides from a recent talk I gave on the 1MLoC+ Haskell code base we have at StanChart. http://code.haskell.org/~dons/talks/padl-keynote-2012-01-24.pdf http://code.haskell.org/~dons/talks/padl-keynote-2012-01-24.... The guys on our team have between 4 and 25 years (yes, not a typo) experience with Haskell. P.S. We are hiring in London, Singapore and Tianjin. --- A couple of days ago was CUFP, by the way, which includes a series of talks from different business users. http://cufp.org/videos/keyword/2269 http://cufp.org/videos/keyword/2269
- eru 14y agoAt Citrix we only have a couple dozen thousand lines of Haskell. But it's also a real world use.
- seanmcdirmid 14y agoTianjin? I only know of one guy in my entire lab in Beijing who even knows Haskell.
- deleted 14y ago
- stiff 14y agoCarmack himself adressed your second point pretty well in another post: http://www.altdevblogaday.com/2012/04/26/functional-programming-in-c/ http://www.altdevblogaday.com/2012/04/26/functional-programm...
- eru 14y agoAnd if you don't want to jump into Haskell's strangeness right away, there's always OCaml. It's fast enough for finance, so it should be fast enough for you. (I prefer Haskell, it's much more beautiful. But that's more of a religious issue. OCaml is still way better than, say, C++.)
- prunebeads 14y agoFor having extensively programmed using C++ and OCaml, I could not agree more. The issues that most people focus on however are : portability to the platforms they're concerned about, performances on these platforms, richness of the ecosystem. Game programmers cannot consider Haskell or OCaml as real alternatives to C++ for these reasons. Yet. That said, people learning to use these languages become better programmers overall. They get a better sense of how to structure, validate their data, and avoid a lot of pitfalls. Have look at the answer made by a representative of one of the products mentioned by J.Carmack - she goes through some of the reflexes she acquired while using her product, and there's definitely an overlap with the way people code using Haskell or ML. It's not an accident that C++ is moving toward incorporating features from functional languages. Just for these reasons alone people should learn to code with languages like that.
- eru 14y agoYes. Though, as with lots of projects, the core that has to be written in a C or C++ can be relatively small. Haskell's FFI is quite powerful, and it should be possible to use it in a way similar to how people embed Lua.
- Nimi 14y agoI wonder if there are languages who have the reliability of Haskell's type system, but are suitable for C/C++ stuff (that probably means, at the very least, they're strictly evaluated). ATS came up as a possible candidate, does anyone have real experience with it? BTW, I once considered this (this being a language with Haskell's strengths, suitable for systems programming) a possible idea for a thesis, but at least for now it's put on hold. If anyone has any comments, I'd really love to hear them.
- doublec 14y agoI've used ATS a fair bit in real programs. The back end of my bitcoin mining pool is written in ATS and handles a reasonable load. I've also used ATS in other projects and am happy to answer questions you might have. Contact details at the bottom of the website linked in my profile.
- eru 14y agoDo you know about OCaml? It shares lots of Haskell's type system, but it's strict. There's also lots of work being put into Haskell right now, to make it more suitable as a low level language---some by direct work on Haskell, some by making it easy to embed DSLs that compile to carefully restricted low level code.
- Nimi 14y agoSounds cool, can you provide some pointers to that work? 10x BTW, I wonder if they allow the programmer to better control when and how much memory is allocated, that seems to me like the biggest obstacle for using Haskell for low-level stuff.
- eru 14y agoICFP 2012 had quite a lot of talks about DSLs. Just go to e.g. youtube and search for something like "icfp 2012 domain specific language".