9 ms·
Facebook Open Sources flint, a C++ linter written in D
- agwa 13y agoGitHub repo: https://github.com/facebook/flint https://github.com/facebook/flint
- staunch 13y agoOnce you get used to tools like `go fmt` or Perl::Tidy it feels barbaric not to be able to cleanly format all your code instantly. Read only linters are nice but something that can reformat code is much nicer.
- sliverstorm 13y agoHuh? You know that a linter is not a formatter, right?
- DanHulton 13y agoPretty sure he does. His comment only makes sense in that context.
- sliverstorm 13y agoI guess I just don't see that. perltidy and lint(s) are completely nonoverlapping, to my knowledge. Something that can reformat your code doesn't even do the same job as a linter.
- mbell 13y agoI'm not really following you. It's very common for linters to flag stylistic and formatting issues.
- sliverstorm 13y agoAh well, perhaps it's just a difference in experience. I've never used a linter (there are countless linters) that fixed indentation, arranged if statements, and inserted newlines to wrap long lines. Nor has a formatter ever helped me find the sort of issues I use a linter to find.
- maw 13y agoThere is overlap. I once lost a fair amount of time because some dimwit predecessor thought putting anything other than a space after commas was acceptable. (It sounds ridiculous but it's true. I can explain if you care.) So: a formatter will, for example, make sure your lines are indented to some standard amount of indentation, a linter will bitch if they are not, and both the formatter and the linter should DTRT when they identify certain indefensible coding practices.
- he_the_great 13y agoI wouldn't want to reach for lint to handle formatting, that is what indent is for. That is to say, lint should not complain that the if should contain a space before the (.
- maw 13y agoI agree in that case, but when it comes to commas, using spaces in a certain way is objectively better.
- Scaevolus 13y agoClang-format aims to provide this for C++. http://clang.llvm.org/docs/ClangFormat.html http://clang.llvm.org/docs/ClangFormat.html http://www.irill.org/videos/euro-llvm-2013/jasper-hires.webm http://www.irill.org/videos/euro-llvm-2013/jasper-hires.webm
- jbergstroem 13y agoWhile reading the article, I also thought why they didn't at least mention it (and why they abandoned the idea). It's powerful, fast and will most likely be the base for a lot of {h,l}inting in editors/IDE's moving forward. edit: Somehow missed that they mentioned Clang not being mature enough when starting this project.
- letzjuc 13y agowell there is clang-format and clang-tidy and they are really easy to set up in CMake (I use "make format").
- eliasmacpherson 13y agohttp://astyle.sourceforge.net/ http://astyle.sourceforge.net/ linters are more important that pretty formatting to my mind. c++ has had both linters and reformatters for a long time. I suppose the distinction is that they are not built in.
- puppetmaster3 13y agoI'm going to the D conference!!! :-)
- doe88 13y agoYears ago I remember learning template metaprogramming in C++ from an amazing book [1] from the same author of this tool. Even if looking back, I now think template metaprogramming when over-used may be a little bit over the top, I nevertheless have no doubt judging by his author that this new tool must have some very good qualities. [1] http://en.wikipedia.org/wiki/Modern_C%2B%2B_Design http://en.wikipedia.org/wiki/Modern_C%2B%2B_Design
- eco 13y agoIf you enjoyed that you should pick up The D Programming Language by Andrei. It's a good read. Metaprogramming with D is a dream compared to C++ so some of the techniques in Modern C++ Design aren't even necessary. http://www.amazon.com/D-Programming-Language-Andrei-Alexandrescu/dp/0321635361/ref=sr_1_1?s=books&ie=UTF8&qid=1393281290&sr=1-1 http://www.amazon.com/D-Programming-Language-Andrei-Alexandr...
- dman 13y agoI am surprised how short the tokenizer is. The other thing is D looks surprisingly approachable coming from C++.
- jlarocco 13y agoI'm a little skeptical that it doesn't use a standard parser like clang. My gut instinct, from experience with crappy c++ "parsers", would be that it works great 95% of the time, but fails the 5% of the time when it'd be most useful. On the other hand, given the author, and the fact he explicitly mentions wanting to support C++11, I'm not sure my skepticism is warranted. I wonder if anybody's tried it on a big C++ code base outside of Facebook?
- apetresc 13y agoIt doesn't seem that flint parses at all – it only tokenizes, and then you write "rules" on "streams of tokens." It kinda sounds like a set of fancy regexes, but on tokens instead of characters.
- evincarofautumn 13y agoThat’s exactly right. And tokenising C++ is infinitely easier than parsing it, though there are still the odd context-sensitive edge cases like “> >” versus “>>”.
- p0nce 13y ago> I wonder if anybody's tried it on a big C++ code base outside of Facebook? I used a similar tool (cppcheck => http://cppcheck.sourceforge.net/ http://cppcheck.sourceforge.net/) which also does not try to build a correct AST and does no proper semantics. The results are surprisingly useful, and most importantly it's very easy to add user-defined checks (edit: of course you also get some related false positives but it's very flexible).
- jbergstroem 13y agoTo me, this is more a proof of concept implementation in D than a tool I'd consider having part of my workflow. The dependency chain and "exotic" language choice makes just having it available in your ecosystem a higher jump than what utilities like linters should require.
- lmm 13y agoOn the contrary, a small standalone tool like a linter is a perfect place to start experimenting with a new language before introducing it to your core projects. I'd expect there'll be packages available for it soon if it's really too much effort to build.
- andralex 13y agoClearly that's something to worry about. What won our team over was the (sometimes spectacular) speed gains compared to the C++ linter.
- vl 13y agoCould you comment on why it is faster?
- eru 13y agoWhy? I'd rather put some crazy requirements in my build tools, than in the delivered product. Eg hypothetical: who cares if it only builds on, say Linux? We have VMs for that. As long as the output program runs on the target platform, say Windows, we are golden.
- shin_lao 13y ago"Even now, clang cannot compile some of our C++ codebase." Andrei, I'm a bit surprised by this statement. Could you give an example? Otherwise great work.
- Gownta 13y agoHere's a gem that clang produces when compiling folly: "clang-3.4: error: unable to execute command: Segmentation fault"
- shin_lao 13y agoSo we're not the only ones. We had to go back to Clang 3.3 because Clang 3.4 had issues with our meta-programming craziness. But still, I submit a clang-based tool is a better bet in the long run.
- inglorion 13y agoHere are some examples that we've run into in the past: Some of our code used GCC nested functions, which Clang did not support. We were using __sync_val_compare_and_swap with __int128 values, which wasn't supported by the Clang version we were using. I believe the current version of Clang supports this. Clang is stricter about some things than GCC, e.g. Clang doesn't like it when you declare a class as struct in one place and in a class in another place. I think there is also a difference where Clang deduces that your enum is unsigned if none of its declared values are negative, and warns you if you compare it < 0. Since we compile with -Werror, this will cause compilation to fail. None of these are unsurmountable, but they do mean we can't just drop in Clang and expect everything to work.
- agame 13y agoI believe clang also doesn't currently support the "ifunc" attribute, which we also use in a few places.
- rquirk 13y agoNot facebook examples, but things I've seen are as follows. All these were fixable to be correct for both GCC and Clang (it's worth it for the clang static analysis). GCC can call the base destructor explicitly in destructors (an error in clang). i.e. this->~BaseClass(); This is normally a coding error anyway, since the call is added by the compiler. You sometimes have to add this-> in templates to disambiguate which function is called. GCC has, or had, a better (different?) two-phase look up for templates, so you can #include a file after its template is used in another header. Clang is stricter and requires you #include "A.h" and #include "foo.h" before using A<foo>, even if the overall compilation unit does eventually declare the A template and the foo class. GCC supports variable length arrays, so you can do `MyClass x[someValue + 1];`, but with Clang you need to use a vector of MyClass, like vector<MyClass> x(someValue + 1). This sort of thing compiles in GCC, but not clang: template <typename T> class List : public list<T> {}; const List<string> tmpList; It is missing a default constructor, GCC finds one from somewhere :) GCC allows friends of derived classes to call static base class protected functions. i.e. Base::protectedFunc() can be called if the current class is a friend only of BaseSubclass. In clang you need to call BaseSubclass::protectedFunc(). Clang requires more explicit template instantiation to avoid link errors. Not sure exactly when, just "sometimes" and you'll know it when you see it :-) Then as mentioned elsewhere clang shouts a lot about mismatched forward declaration (struct vs class).
- haberman 13y agoIt amazes me how far out of their way many people will go to avoid using parsing tools. He avoided using a lexer generator because "I'd figured using a lexer generator was more trouble than it was worth". To him it was more convenient to write a lexer manually (using macros) in C++, then later completely rewrite it in a different language (D) as a one-off trie matcher code generator. I am amazed that this could be thought of as less work. How can writing two lexers from scratch manually be less trouble than writing a bunch of regexes in a Flex or Ragel file? Especially since the result was likely slower than using Flex or Ragel would have been? To me the interesting question is: how much of this allergy to external tools is: 1. the trouble of learning/maintaining something new/different (inherent tool overhead) 2. design of the tool isn't as user-friendly as it could be (incidental tool overhead) 3. irrational belief that the tool will be more work/trouble than it actually is (non-optimal decision making) If part of the answer is (2), then improved tools will lead to greater adoption. And hopefully more convenient tools will lead to them being better known and more widely adopted, which should lessen (1) also. Everyone uses regexes; using a lexer generator should (in my opinion) be as easy as using a one-off regex. I think the key is to make the lexer generator an embeddable library like regex libraries are.
- p0nce 13y agoSee my comment here https://news.ycombinator.com/item?id=7293796 https://news.ycombinator.com/item?id=7293796, to make anyone able to add a company-specific check quickly is a valuable asset (to ensure decisions like "We don't want to use idiom X anymore"). Moreover, fake parsers like cppcheck will lint your code even if the code does not even build, so they are really easy to setup.
- eco 13y ago> Everyone uses regexes; using a lexer generator should (in my opinion) be as easy as using a one-off regex. I think the key is to make the lexer generator an embeddable library like regex libraries are. There actually is a really powerful parser generator for D by Philippe Sigaud: https://github.com/PhilippeSigaud/Pegged https://github.com/PhilippeSigaud/Pegged It's much more pleasant to use than something like Boost Spirit. Somewhat relatedly, D's standard library has a compile time regex engine that compiles regular expressions down at compile-time resulting in some of the fastest regular expressions in the world.
- foobarian 13y agoI chuckled at this passage: "flint is written in the D language, making it the first D codebase open-sourced by Facebook. In fact, our initial version of flint was written in C++; the D rewrite started as an experiment. From the measurements and anecdotes we gathered, the translation turned out to be a win on all fronts: the D version of the linter turned out smaller, dramatically faster to build, significantly faster to run, and easier to contribute to." Seems like the argument is stronger to rewrite the code being linted, than to use the linter itself ;-)
- eru 13y agoD is actually a pretty nice language. (The caveats are around the libraries, especially the standard libraries to choose from.) I can see D being a better C++, in that it better solves that problems that C++ is supposed to be good for.
- MaxBarraclough 13y ago> The caveats are around the libraries, especially the standard libraries to choose from. The old Tango/Phobos situation? With the advent of D2, that's no longer a thing. Tango is now all but dead.
- eru 13y agoThanks for the update! I haven't toyed around with D in a while---but it was a mostly pleasant experience the last time I did.
- he_the_great 13y agoThis is likely what they are looking at. Start with a program everyone uses, but isn't critical production system, evaluate the language and get people familiar with the language. Follow it up with some more rewrites and eventually give up on the old C++.
- frou_dh 13y agoThis looks really good. Having checks like this integrated and automated is the big win. Like data backup, if it's not automated then it doesn't really count.
- dedosk 13y ago> Marking namespace-level data as static inside a header is > almost always a bad idea. Can anybody explain this to me with some example?
- adsche 13y agoNext sentences: Labeling the data as such potentially generates one instance of the static data inside each compilation unit, including that header. Fixing these issues has led to measurably smaller and faster executables. When you include that header in two different .cpp files, which compile to two different .o files, you have that data twice, local to the .o file (compilation unit). Now link them together and you have that data twice in the executable. You probably did not want that. (But -- as with some more of their examples -- maybe you did indeed want that?)