9 ms·
Static Analysis Tools for C
- Alifatisk 3y agoThe link was supposed to go to the section C https://github.com/analysis-tools-dev/static-analysis#c https://github.com/analysis-tools-dev/static-analysis#c
- jjgreen 3y agoThe whole collection is rather interesting though -- thanks!
- canucker2016 3y agoReaders should also peruse the 'Multiple languages' section, many of the big names, Coverity, Klocwork et al. are listed there. see https://github.com/analysis-tools-dev/static-analysis#multiple-languages-1 https://github.com/analysis-tools-dev/static-analysis#multip...
- rwmj 3y agoI get that this is a list of static analyzers, but for C runtime checkers are useful to know about too. Valgrind, clang ASAN, american-fuzzy-lop, glibc _FORTIFY_SOURCE are all useful tools in the toolkit.
- Alifatisk 3y agoI might actually use Valgrind in the future! Looks helpful as hell. https://valgrind.org/docs/manual/quick-start.html https://valgrind.org/docs/manual/quick-start.html
- rwmj 3y agovalgrind is really essential. Just about every C project that I have written has a "make check-valgrind" rule which runs the same set of tests as "make check", but under valgrind. Passing both is a requirement for release (although for various reasons to do with cost, only "make check" is done in CI). valgrind mainly finds memory leaks, buffer overflows and use of unintialized memory. It even works relatively well with some non-C languages. Example: https://gitlab.com/nbdkit/nbdkit/-/blob/9f96d53dbbbf67ae5ed0acb2ef8887709c47d3cc/tests/Makefile.am#L159-L161 https://gitlab.com/nbdkit/nbdkit/-/blob/9f96d53dbbbf67ae5ed0...
- gpderetta 3y agoValgrind was quite essential, but these days the various sanitizers are both easier to integrate, faster and catch more issues.
- adrianN 3y agoDo they catch more issues? I thought valgrind was the gold standard. I would be happy to be proven wrong!
- PhilipRoman 3y agoIts a tradeoff, they both specialize for slightly different situations. Sanitizers have intricate knowledge of what is undefined behavior at a source level. For example signed integer overflow, etc. Some of these are simply not possible to determine at runtime, as a valid C program is allowed to generate the same machine code as one containing UB. Valgrind, on the other hand, cares purely about the machine code (with a few exceptions like malloc, free, etc.). It can peek into your libraries even if they aren't compiled with sanitizer flags. It is especially good for finding uninitialized memory usage - IIRC it can track this data even on the level of individual bits in a register.
- hannob 3y agoA combination of testing with ASAN and MSAN is pretty much always better than valgrind if you have source code. Though MSAN use can be somewhat challenging, as you need to have all dependencies built with MSAN, too.
- rwmj 3y agoIt's different. The reason valgrind might be better is that it tests production binaries. Having to recompile everything with ASAN is difficult in traditional 'make && make check' workflows, or if you want to build and distribute tested binaries (although can work in CI if set up in advance, and we do use it sometimes there). ASAN + fuzzing is a great combination.
- williamcotton 3y agoI would recommend getting familiar with making a suppressions file! Valgrind is not perfect and it can be a little noisy with “possible” leaks, depending on what you’re doing!
- jbaber 3y agoAs others have said, it's indisposable. It's useful to learn how to suppress false positives (e.g. from boost) so you can see when an actual leak appears.
- gavinhoward 3y agoYes, absolutely. Static analyzers are for when your test suite is subpar. If you're obsessive, like the SQLite guys [1], runtime checkers are better. But you have to be obsessive about an excessive test suite. [1]: https://sqlite.org/faq.html#q17 https://sqlite.org/faq.html#q17
- AlotOfReading 3y agoYeah, no. They're different things. Formal methods, including static analyzers, can prove the absence of issues in all possible conditions under some assumptions. Tests verify that implementation, under the tested conditions, once. You can't replace testing with formal methods, but formal methods are more powerful when they work. That sqlite hasn't found them useful is just a reflection of how absurdly far out they are on the testing spectrum and how complicated the problem domain they're operating in is (very).
- gavinhoward 3y agoDon't get me wrong; I'm a fan of formal methods, when they can work. But you're also weakening your own point; you admit that if you test enough, they don't work as well, and you also admit that they work less well the more complicated things get. The other problem is showing that the artifact you proved is the same artifact that you are using. The enormous amount of effort put in by the seL4 people is proof that that is non-trivial, and seL4 is tiny compared to SQLite. So when you say, "formal methods are more powerful when they work," you are right, but "when they work" is carrying a lot of weight.
- nicoburns 3y agoStronger type systems can get you a lot of the benefits of formal methods with very little work. A SQLite level test suite is probably still better, but very few projects have the time or budget to do that.
- gavinhoward 3y ago
- ChrisRR 3y agoGiven how much C is used in embedded environments, runtime checking is often not an option
- mumblemumble 3y agoI would guess that even when you're targeting embedded environments you can still use runtime checks to help find problems during any part of the development lifecycle where the code isn't running on the target hardware? That's roughly what I do for certain non-embedded work with high performance needs. There's a lot of contract checking and assertions that are too heavyweight for production, so they aren't included in deployment builds. But I certainly wouldn't want to not test at all just because I can't test in production.
- ChrisRR 3y agoYou can, but there's not many tools that you can use to just throw everything at the code like with desktop software. Even then it's often very dependant on the MCU and compilers you're using
- Gibbon1 3y agoI try to do runtime checks for my embedded code. Because as I like to say there isn't anyone to press F5 when something barfs. Bigger issue for me is what to do when a run time check fails. Sometimes there is nothing to be done but to log the error and reset. Other times, you log the error, clean up and keep going. Which why I dislike rust's run time checks which panic without letting the program try to recover.
- mumblemumble 3y agoThey keep the dynamic analysis tools in a separate repository: https://github.com/analysis-tools-dev/dynamic-analysis https://github.com/analysis-tools-dev/dynamic-analysis Both repos link each other close to the tops of their respective readmes. Annoyingly, though, the webstite seems to only include the static tools.
- mre 3y agoMaintainer here. Thought we fixed that. I'll look into it.
- jart 3y agoI can't believe a markdown file has three corporate sponsors.
- executesorder66 3y agoLink aggregation as a Service is the new hype topic taking the place of AI.
- appplication 3y agoFortunately SEO on alphabetically sorted lists is a solved problem. That’s why in announcing my new service, Aardvark. Using its metaphorical tongue, it licks up any bugs in your code.
- mananaysiempre 3y agoThis kind of phonebook optimization seems to have driven somebody to name their company that; and then of course there’s the old-time game company foursome of Activision, Accolade, Acclaim, and Absolute Entertainment. Still, none of them holds a candle to that Russian accounting staple, 1С.
- stevekemp 3y agoIs it "made with :heart:"? I only enjoy alphabetical lists of tools, ideally contained in orphaned repositories which have a bunch of outstanding pull-requests, and which are secretly designed to lure clicks to their sponsors if they've got red-heart emojis and reference the price of a cup of coffee. AI overlords never drink coffee, so they can't appreciate what it means to sponsor such lists.
- mumblemumble 3y agoThey're also publishing https://analysis-tools.dev/ https://analysis-tools.dev/ , which is a dynamic site that I would assume does have some hosting costs. The wording in that banner implies that the sponsors were helping to fund that and not just the markdown file.
- 3y ago
- fusslo 3y agoDoes anyone use any of these tools as part of their professional life? I would love to know which tool you use and how it's used.
- Alifatisk 3y agoSomeone mentioned Infer (by Meta) some years ago, now when checking it out, it looks to be dead. Otherwise, I've used flawfinder for some of my projects.
- retrocryptid 3y agoWhen I write my autobiography, an entire chapter will be named "/* FlawFinder: ignore */".
- sumtechguy 3y agoI used to use a program that some devs were convinced had tons of false positives. Until you actually look at it. Usually something silly was going on to either cause it to not see what you were doing (rare) or there really was an issue (not rare). The 'oh that is a false positive' usually clues me into there is a whole class of bugs that this project has that need some deep looking into. Now I am not saying that happened to you but sometimes they are worth looking at. I saw one yesterday the dev did not understand why it would randomly crash. His destruction loop was backwards and was using pointers that may or may not still be good. Sometimes you cannot control the lib. So you have to 'ignore' but that only gives you time to mitigate.
- flohofwoe 3y agoNot professionaly, because currently I mostly write Typescript in my job, but I run all my (hobby) C and C++ code through clang static analyzer (exceptionally simple in Xcode because it's fully integrated, but also in command line builds - I haven't bothered with setting up a CI for this yet though, still on the todo list). For command line builds I'm planning to migrate to clang-tidy though (contrary to popular belief clang-tidy is not just a linter, it also runs the clang static analyzer checks). Visual Studio also comes with their own static analyzer, but this doesn't seem to be as thorough as Clang's. Also: static analyzers are just one aspect, the other is runtime sanitizers (e.g. clang's ASAN, UBSAN and TSAN).
- retrocryptid 3y agoso... there's no lisp or forth. i guess that means you can't write security bugs in lisp, right?
- eesmith 3y agoClojure is a Lisp.
- Verdex 3y agoI was going to say that this probably isn't fair because they do get quite a few languages and nobody is going to have the bandwidth to get all languages... but then *they* say "...for all programming languages...". I mean, there are a LOT of programming languages out there. Was it really going to hurt them to say something like "almost all" or "many" or even "all commonly used". Ultimately, this is a nitpick on the website. However, the whole point of static analysis is that you're nitpicking because not nitpicking can sometimes have disproportionate consequences. So I agree that it would be nice if they cleaned up that particular section to be a bit more accurate.
- mre 3y agoThe idea is that tools for all programming languages are accepted. If there's a missing tool, you can contribute by opening a pull request.
- junke 3y agoThey could list https://en.wikipedia.org/wiki/ACL2 https://en.wikipedia.org/wiki/ACL2 but nobody cares I guess
- carapace 3y agoI see the whole thing as a huge advertisement for Forth, Lisp, and Prolog.
- npsimons 3y ago> i guess that means you can't write security bugs in lisp, right? I know this is sarcasm, but there's a reason Greenspun's tenth rule holds true.
- smcameron 3y agoclang scan-build is decent and easy to use. It's found a few bugs for me. Back when hacktoberfest gave away t-shirts, I would use it to find bugs in other people's projects and try to fix them. It can give some false positives sometimes though, but not too bad.
- NLips 3y agoHas anyone found a static analysis tool which understands C11 annex K (aka “safe C”) functions? I’ve found some tools like CLANG static analysis will raise errors for potentially incorrect calls to stdlib C functions, but doesn’t understand the replacements, which means some errors previously caught by analysis can only be caught at runtime.
- bierjunge 3y agoAnnex K is optional and the only compiler I'm aware of implementing it is MSVC (and only Microsoft wanted that in the standard), so the support for it will be nonexistent in "normal" tooling. If you need it, check if MS has something.
- lelanthran 3y ago> Annex K is optional and the only compiler I'm aware of implementing it is MSVC (and only Microsoft wanted that in the standard), And to rub salt into the wound, the Annex K functions supplied with MSVC are non-conforming to the standards Annex K functions, which were also pushed hard by Microsoft, which make them kinda doubly pointless: you use them and make code that is neither portable to another compiler nor conforming to the standard :-/
- account42 3y agoAlmost sounds like yet another EEE tactic.
- mumblemumble 3y agoI'd just as happily attribute this one to Microsoft's systemic inability to stick to a single plan for five minutes in a row.
- pjmlp 3y agoTo be fair, most the stuff ISO adopts is rarely taken as suggested, that isn't the first, nor will be the last.
- ChrisRR 3y agoThis is an interesting list, given that PC-lint moved to a subscription model. As more tools move to a subscription based model, I don't want to tie myself down to 5 subscriptions just to maintain some ancient code.
- bluGill 3y agoWhat is the difference between a subscription and a yearly fee for the new version? When you look at it that way subscriptions make more sense - why wait until the end of the year for a new version (that might have some feature you really need sooner). Of course if you are content to sit back several versions (without support) the above doesn't make sense. As you say, if the code itself you are looking at is ancient and only minimally maintain it isn't worth keeping subscriptions and it would be better to have a permanent license to tools that you won't change.
- ChrisRR 3y agoI don't want a yearly fee for a new version, I want to use the same version that I paid for. Especially when you're developing software for safety applications, it becomes important to maintain/archive the same tools/compilers/libraries so that rebuilding old code in a few years still produces the exact same binary. So at that point it's not as important to have the newest and shiniest tools, but more important to pay for a tool once and know that it works reliably.
- kaetemi 3y agoThe difference between subscription and buying is when a product is discontinued.
- bluGill 3y agoCompanies regularly have a clause in their subscription that if the service is discontinued they get a perpetual right to the last version. Sadly this doesn't seem common for consumer software.
- account42 3y ago
- synergy20 3y agofor C I just use -fanalyzer from gcc for c++ I use clang-tiday and cppcheck, neither can handle all c++20 though with noisy false positives, but better than not using them
- Alifatisk 3y agoIs the flag "-Xanalyzer" for macos?
- account42 3y agoWait what a list of tools needs sponsors now? What?
- shortrounddev2 3y agoShould add microsoft's SAL. Comes built in to MSVC
- colesantiago 3y agoThis looks off to me. This site is making money from open source free tools, (where the site claims it's open source) asking for contributors to contribute for free and not giving back to them or the authors of the tools. We have no clue about where the money is going as this site isn't transparent even though they claim they are 'open source' Is the sponsor money also going to these open source projects as well and what is the proportion?
- kmarc 3y agoA similarly useful list is vim's famous ALE plug-in's list of supported linters: * https://github.com/dense-analysis/ale/blob/master/supported-tools.md https://github.com/dense-analysis/ale/blob/master/supported-... While less comprehensive¹, this is my go-to list when I start working with a new language. Just brew/yum/apt installing the tool makes it work in the editor² ¹this list mostly has foss,static analyzers, however anyone can contribute (mine was the gawk linting) ²alright, there are some tools that might need some setup