6 ms·
I use autotools for my C++ projects and I hate it (in my weak defense, I hate cmake more). config.cache helps (and ccache helps a lot, too; most of my ccache hi
by jonstewart 3y ago
I use autotools for my C++ projects and I hate it (in my weak defense, I hate cmake more). config.cache helps (and ccache helps a lot, too; most of my ccache hits are on stupid autoconf test programs; ya gotta use ccache if you use autoconf), but rerunning configure is still s l o w.
Worse, automake and autoconf have a cyclic dependency, so if you're a developer (as opposed to a user building from source on a system) you wind up running configure all the time. This is because, most probably, your Makefile.am lists every source file individually. If you add a new source file, you must rerun autoreconf to regenerate your Makefile — and that also generates configure anew. Then the Makefile sees that configure is newer than config.h (or some other output, I don't want to look up the gory details) and, blammo!, when you run make, first configure runs.
Of course with git, this happens not just when you add a new source file to your project... it happens whenever you switch branches, where there's variance in the source files in the project. configure configure configure configure.
As much as I hate it, autoconf isn't the worst part of autotools. Automake is... pointless? Is it really less effort to write Makefile.am than to write a straight Makefile? Wouldn't that be simpler?
And then there's libtool, which should be drug out into a back alley and executed. A 2000 LoC shell script, whose whole purpose in life is to figure out, over and over and over again, stateless in the same way as TFA complains about with autoconf, precisely which compiler flags should be used to build a shared library on a system amongst all the old compilers, even though today everyone just uses gcc and clang, which are flag-compatible, and determining which flags to use is not really that hard. Please, someone, anyone, please kill libtool and banish it from this earth.
I tried Meson — ran into a problem, read the docs, asked a question on the mailing list, never got an answer, never got past it. Bazel's obviously a terrible Google project and won't work if you're not sufficiently Googly. Cmake, a disaster. I liked tup, but it seems to have died (and it didn't solve the autoconf problem anyway).
Which new build system will deliver us from the 18th circle of hell? (probably cargo, as the Rust fanbois work to get C/C++ deprecated...)
- thechao 3y agoI'm be written lots of C code that runs on lots of architectures & platforms: PDPs, x86, ARM; Windows, BSD, Linux, VMS. My projects have spanned them gamut from tiny to gargantuan (millions of lines of code). I've never needed autoconf... what's it even for!?
- jonstewart 3y agoWell, as archaic as it is, and as awful as its execution model is, autoconf is at least somewhat useful: * determining availability of different syscalls (yay, printf) * determining compiler * verifying dependencies are installed and available (e.g., openssl) This last one is made much easier with pkg-config and the pkg-config macros in autoconf. There's an argument to be made that pkg-config is bad because it will fail the build if you're just missing the wrong .pc file somewhere, even if your system satisfies the needs of the program - true enough. But pkg-config _does_ make checking for dependencies in autoconf much easier, and it does something essential for correctness, which is gathering up the necessary compilation and linker flags of your dependencies to pass along to the compiler so that you can use them correctly and match ABIs. A good configure script will guide you through getting dependencies installed, which is much nicer for users than confronting them with compilation/linker errors.
- egberts1 3y agoUnfortunately, pkg-config couldn't handle C++ files nor the transition from lib32 to lib64; as Captain Kirk heard from Bone: "he's dead, Jim".
- db48x 3y agoIn decades past, if you were developing software for Unix you would inevitably run across compatibility problems. Even with the POSIX standard, every Unix was trying new things, inventing new things, or just implementing the same thing in slightly different ways. So your programs had to have a bunch of macros to switch between different variations. Some API would have two arguments on Unix A but three on Unix B, so you would have to turn that api call into a macro that would insert the missing third argument or whatever. But now your users needed to know which configuration of your software to use. You could try documenting all the little knobs and switches and make them choose the correct values, but someone hit on a clever idea. Just write a little program that tries calling the API with two arguments, and see if it compiles. If it does, you could automatically define THAT_API_HAS_TWO_ARGS and your code will do the right thing. If it fails to compile, then define something else. So you start writing a shell script that will run through all of these tests and dump out a header containing all of these configuration choices. Call the script `configure` and the header `config.h` and you are good. Except that actually writing a shell script that is really actually POSIX correct is pretty hard. And the code is pretty repetitive. Mistakes start to creep in, bashism break things when it’s actually run under `sh` and so on. And it turns out that all the little tools we rely on, like find and grep and sed and awk, are all different on different platforms too! Turns out that nice option you were passing to grep is just an extension, only present on your specific flavor of unix. So the next clever idea was to write a program that generates the configure script. The repetitive bits can mostly be solved by text substitution, so someone had the bright idea to use M4. Thus autoconf was born. Autoconf solved a lot of problems. It ensured that the configure script was actually POSIX correct. It did this even though the generated script was even more complex than ever, with extra logging and tracing and debug output that most people never look at, but which is actually super useful for debugging. The configure script you had written by hand didn’t have any of that! And autoconf came with baked–in knowledge about all the things you might need to test for. Any time someone discovered some new difference between Unices, new tests were added to autoconf for it. Autoconf users hardly had to do any more work than to look up the name of the macro that it would define for you. Of course, M4 causes actual brain damage in regular practitioners, and nobody cares to write software for any Unix except Linux these days. Even the BSDs are an afterthought. Technically OpenSolaris is still out there, with superior features like ZFS and Zones and whatnot, but that’s a lot of extra work. So autoconf is a dinosaur, solving problems that nobody really has any more. Rachel is not wrong that it’s crazy to rerun the same configure script over and over, but really only developers have to do that. Most people just run the thing once when they install your software, and never see it again. And most people don't even do that any more, because they download binaries from their package manager most of the time. It’s just the distro packagers that actually run your configure script. And she’s right that the answers configure gets should be cached system–wide. Autoconf does support that, but it’s harder to do correctly than she remembers. And by the time it was possible usage was already waning in practice.
- amluto 3y agolibtool does serve a purpose: it causes plain 'make' to produce shell script wrappers as outputs, which is maybe kinda sorta vaguely useful when developing and utterly and completely obnoxious if you want an actual ELF file. So you end up running 'make install' when you didn’t actually want to install or trying to remember how to find the actual ELF files. And you then wonder why 'make install' appears to be building things instead of just installing them, and you curse at libtool. In its slight defense, the magic incantation to convince gcc (or, yikes, pre-gcc cc or ld or…) to competently produce a DSO is both nontrivial and platform dependent. Last time I tried to get all the options right off the top of my head, sensible defaults were not a thing.
- jonstewart 3y agoOh right, how could I forget that awful aspect of using libtool? I hate my life; maybe 2024 will be the year I find a decent build tool and the motivation to move projects to it. Contra your slight defense, is it really so hard to produce a good DSO these days? -fPIC is half the battle and a couple other weird flags for macOS/darwin will do you. Cross-compiling for Windows, sure, a few more flags — but you look them up once on stackoverflow, you have three options, and you're done.
- drewg123 3y ago100% this. Libtool is awful. In the early 2000s, I ripped libtool out of a product that built on Linux, Solaris, FreeBSD, Aix, Tru64, Mac OSX and Windows.
- amluto 3y ago> Contra your slight defense, is it really so hard to produce a good DSO these days? -fPIC is half the battle I did say “slight” for a reason :)
- jcranmer 3y agoI always thought that purpose of libtool was to incorrectly guess that it was smarter than the developer in figuring how to link files together. When I've been trying non-default link magic stuff (something like lto), trying to figure out how to get libtool to actually pass the damn necessary linker flags to the linker was damn near impossible.
- jeremyjh 3y agoI don't know how you can think CMake is worse than autoconf. At least it has a declarative approach with good mechanisms for reusing common logic. Its much easier for IDEs/editors to work with it.
- jonstewart 3y agoI don't think it's worse, per se, but I hate it/myself _more_ when I contemplate switching from autotools to cmake -- it's like upgrading from CP/M to DOS, when what I'd like to have is a Mac. And as much as autotools blows, every time I try to build some project that uses cmake I inevitably go through some rigmarole making a build directory (not that out-of-source tree builds aren't the right thing, they are, but why make me create one?) and finding some obscure set of -D options to get the sucker to build on my system. configure && make still has cmake beat from a user's perspective.
- jeremyjh 3y agoYeah it’s not intended to be used directly for end user builds. Its ultimate product is a set of makefiles (or Visual Studio project files) and the developer should distribute those.
- dilyevsky 3y ago> Bazel's obviously a terrible Google project and won't work if you're not sufficiently Googly I don’t even know what that means, sounds like a bunch of fud to me. C++ and Java are first-class languages in Bazel. It works great for these, then Go, then you start venturing into the alpha land
- pianoben 3y agoWorks great if you follow Google's style of vendoring all of your dependencies. If you try to rely on third-party package managers like Maven or Ivy or whatever C++ uses, you're in for some serious pain. The more you venture into OSS tools like Spring, gRPC, containers, the more you have to incorporate shoddy and poorly-maintained third-party Bazel tools which inevitable stomp on each other in subtle ways. My team abandoned Bazel for Gradle, and it has been a tremendous quality-of-life improvement.
- dilyevsky 3y ago> Works great if you follow Google's style of vendoring all of your dependencies. If you try to rely on third-party package managers like Maven or Ivy or whatever C++ uses, you're in for some serious pain. It sounds like you've mainly dealt with Java because it's not a pain in C/C++ - if you wish you can link dynamically with system libs or vendor .so binaries. And at least for Maven I used rules_jvm_external which worked fine. I’ll grant that container rules were a mess for a long time but new oci rules are much improved. > My team abandoned Bazel for Gradle, and it has been a tremendous quality-of-life improvement. I hear as much if not more gripes about gradle of the same variety - "hurr durr it sucks bc i don't like it". Personally I'm thinking this is more appropriate critique for an art piece not a build system.
- pianoben 3y agoI've done some C/C++ with Bazel before. It's nice, except that you end up having to vendor all your dependencies if you want repeatable builds :). Had more success sharing projects and building a community with CMake, horrible as that tool is. The reason Gradle is a quality-of-life improvement is that we had boxed ourselves into a corner. rules_jvm_external, rules_docker, rules_proto, and a set of others all relied on internal details of other rules (accidentally, because pre-MODULE.bazel everything gets loaded into one giant global namespace and collisions happen). We basically got ourselves locked in, unable to move off of rules_docker, upgrade our JVM beyond 17, or even upgrade Spring, without breaking and having to upgrade everything else. Gradle, which has first-party support for dependency management, and which is the primary system supported by large OSS vendors (Spring, etc), doesn't have these pitfalls. Poorly-written Gradle plugins can sink you as surely as Bazel packages can, but there is less opportunity for you to discover them. I'm well aware that we were Doing Bazel Wrong, and I've certainly had my share of pain with Gradle, but never in my life have I seen so much time wasted on a build system. For plain old JVM stuff, Bazel didn't work well for us.
- _jackdk_ 3y ago> If you add a new source file, you must rerun autoreconf to regenerate your Makefile No you don't: automake generates rules for `Makefile.am -> Makefile.in` (`automake`) and `Makefile.in -> Makefile` (`config.status`). You should just be able to edit `Makefile.am` and run `make`. > Is it really less effort to write Makefile.am than to write a straight Makefile? Wouldn't that be simpler? I don't think so. Not if you want to get DESTDIR support right, and proper dependency tracking (dep info as a side-effect of compilation, so you get all the headers and don't slow down one-off builds), and support for all the standard Makefile targets, etc. As for your "which build tool to actually use?" question, I wrote a longer post about that in the context of bootstrappable software: http://jackkelly.name/blog/archives/2024/04/01/which_build_tool_for_a_bootstrappable_project/index.html http://jackkelly.name/blog/archives/2024/04/01/which_build_t... It really does seem like autotools remain one of the least bad options, but maybe muon (C99!meson) and samurai (C99!ninja) might be a workable option.
- foul 3y ago>I don't think so. Not if you want to get DESTDIR support right, and proper dependency tracking (dep info as a side-effect of compilation, so you get all the headers and don't slow down one-off builds), and support for all the standard Makefile targets, etc. With any make, yes. With modern GNU make you can embed sh code inside a makefile with the define hack, so no, ugly, but no.
- bee_rider 3y agoWe should just write Makefiles. If you want to be really nice, have it source a Makefile.inc file and keep all the user configuration stuff in there. If you want to be really really nice, consider accepting pull requests for .inc files that work with popular distros.
- Too 3y agoMakefiles are too brittle and allow all kinds of user errors. Targets with too many dependencies, too few dependencies, race conditions. The default doesn’t even handle transitive #include dependencies without you yourself plugging in complex .d-file generation using the compiler.
- dima55 3y agoAre they though? The "complex .d-file generation" is "gcc -MMD" and a "-include *.d" at the end of the Makefile. You specify the dependencies, and it's on you to get those right, as it is in every other build system. I've never seen a race condition. The problem with Make, is that it works just well-enough to allow people to sorta get it to work without requiring them to learn how to actually use the thing. Then they don't read the manual, and complain that Make doesn't, in fact, work.
- mid-kid 3y agoNice job, you just hit a footgun: try to remove any .h file (and the corresponding #include directive) and see your build just break. You need an empty `%.h: ;` directive for make to ignore missing .h files. Make is full of this stuff and as a distro packager having to deal with each's individual quirks is prone to drive one insane. Please consider using meson, at least.
- adrian_b 3y agoI do not believe that adding such rules for ignoring files is wise. I prefer to just run "make clean" after I delete, rename or move source files. With a properly written "Makefile" nothing else needs to be done and there will never be a broken build. With a properly written "Makefile" nothing has to be done when adding source files.
- ndiddy 3y agoIMO Cmake's "killer app" is that it will generate projects for Visual Studio and Xcode, which allows you to get access to the Windows/Mac specific tooling that those products offer. It's also a LOT faster on Windows than Autotools because of how NTFS performs poorly when dealing with writing a ton of small files. I agree that its syntax is ugly but once I got beyond that I found it fairly simple to work with. The most annoying part was figuring out which MSVC flags corresponded with the GCC flags I use. Out of curiosity, what does your C++ development workflow look like? Do you stick to using a text editor rather than an IDE?
- cout 3y agoHonestly I think cmake got a foothold early on because it had color.
- kergonath 3y agoIt also hides stuff so users look at simple, colourful lines with progress indicators instead of naked, overwhelming calls to the compiler. That said, as a user I prefer autoconf. The semantics of --with-foo or --enable-bar are simple and well documented. And ./configure --help shows all the possibilities. Faffing around with -DCMAKE_STUFF is horrible. Ccmake should be nice but in practice never works properly. As a developer, both autoconf and cmake are horrifying and a huge time sink to get them to behave.
- dima55 3y agoRather than the "killer app", that's the only thing that cmake does better than other systems, and the only reason for anybody to use cmake.
- professoretc 3y agoIt's kind of weird that these days, many (most?) projects use Cmake directly as their build tool. They don't generate a VC or Xcode project and then work from that; they generate a VC project and then immediately build it. The CMakeLists.txt effectively is the project definition. VS now has support for opening CMake projects directly, without creating a VS project file from them. So the thing that made CMake useful (and what also makes it complicated...), is now used less and less.