10 ms·
GNU Autotools: A Tutorial [pdf]
- jnxx 6y agoA nice introduction how autotools work, what are the basic concepts, and what one needs to know to configure an autotools build in a standard project. Especially compared to some other build systems, it is not only solid and well thought-out but also surprisingly well documented. Another really nice, more detailed introduction is this one: https://www.lrde.epita.fr/~adl/dl/autotools.pdf https://www.lrde.epita.fr/~adl/dl/autotools.pdf
- jnxx 6y agoOne thing that I am wondering: Why do git repos with totools build chain often include the generated files, like ./configure ? Wouldn't it just be better to provide the source files (autoconf.ac and Makefile.am) and instruct developers to run "autoreconf" to generate the rest? That would it also make much clearer which files need to be changed if e.g. source files are added.
- helaoban 6y agoYou'll find that many projects include a "bootstrap" script that does just that.
- geofft 6y agoIt depends on the extent to which the git repo is s product for end users vs. for developers. Traditionally, the product for end users was the source tarball... but also, traditionally, we didn't have git. :) The whole point of autotools is that the output (./configure) is highly portable and can be run on basically any mostly-POSIX-compliant /bin/sh. You don't need to install anything beyond sh and make just to perform the build - compare with a design like Bazel where you need a piece of software to do the build. If the generated files change rarely, checking in and committing the generated files means people who are just making code changes (or no changes at all) don't need to install autotools. Also, there are functionality changes in certain versions of autotools; you might want to make sure your users get a feature that was implemented in a recent version of autotools, but support users who are running on an older OS that doesn't have that version packaged. (If you expect the git repo to be primarily used by developers / contributors instead of end users who are just building it, then you should definitely gitgnore the generated files, but also make sure to include them in the release tarballs. "make dist" will do it. You should also guide end users to download that release tarball instead of an auto-generated tarball of your git sources - GitHub and GitLab both have support for file attachments in their release feature, these days.) In our modern world, this might be less important because installing software (even specific versions of software) is a lot easier. But then our modern world probably deserves something other than autotools, because a lot of what makes autotools hard to work with is the fact that it's intended to translate down to sh/make. (Options I'm aware of that look fairly reasonable include Meson/Ninja, CMake, and I guess Bazel.)
- froh 6y agoLinux distributions like suse or redhat actually bootstrap autotools during build, to.enforce consistency. at least they did 10y ago. you will sometimes run into subtle forward porting issues doing this. that's why not doing this and using the generated Makefile and ./configure is a good idea (tm) for most consumers of packages. in other words: if you're in for adventure and learning, go for autoreconf. https://wiki.debian.org/Autoreconf https://wiki.debian.org/Autoreconf
- stabbles 6y agoIn my experience they don't often have a configure file in the repo, but only in the release tarball (some repos on github have an action to setup a tarball with configure script). That way the user only needs a shell and make, nothing else.
- junon 6y agoI've seen quite a bit of both. GP isn't wrong.
- ksherlock 6y agoIt's extra work to install and run autoconf. With no benefit or necesity unless you're changing the autoconf script. Artificial dependencies are friction that keeps contributors away.
- jnxx 6y agoI see that point! On the other hand, it is generally strongly encouraged to keep generated files out of source control repos. Especially since one cannot merge them in a meaningful way.
- Thev00d00 6y agoOr use anything else. CMake or Meson have both learned the lessons of Autotools and so you dont need to subject yourself to them anymore.
- stonogo 6y agoMeson is tremendous pain to bootstrap in hardware development projects, because you have to get so much of Python working first. CMake used to be easier to cross-compile to bootstrap a project, but it's getting harder every year. Autotools just keeps working, even if you don't have a c++ runtime.
- ozmaverick72 6y agoI have been surprised at how good my CMake experience has been. CMake may have its warts but so far everything with a CMakeLists.txt that I have tried has worked out of the box
- cozzyd 6y agoYeah but they're not always installed on the random SBC you might be configuring, or an old incompatible version might be installed. And then I have to dig through the build system to figure out the right way to set prefix etc. cmake and meson should really have a way to output a configure script and Makefile...
- junon 6y ago> cmake and meson should really have a way to output a configure script and Makefile `cmake .. -GMakefile` not enough for you?
- dwheeler 6y agoIf you prefer a video, you might check out my 3-part autotools tutorial, starting here: https://www.youtube.com/watch?v=4q_inV9M_us https://www.youtube.com/watch?v=4q_inV9M_us The video shows an animation about how the autotools work (which I think helps understanding), and has a CLI demo of how to use it, all for the low price of free :-).
- smartmic 6y agoI also highly recommend John Calcote's book "Autotools" from no starch press: https://nostarch.com/autotools2e https://nostarch.com/autotools2e Probably the best technical book I bought in the last year. After the lecture, I have put aside all whispered-in prejudices and am convinced that it will still be worth betting on GNU autotools in the 2020s.
- chiumichael 6y agoWhat are some things that autotools can do that more modern tools can't?
- smartmic 6y agoFirst thing that comes to my mind is the best-in-class cross-UNIX compatibility. But shouldn't the question be the other way around: What can the modern tools do that autotools can't?
- dragontamer 6y agoThe obvious: CMake compiles into either Makefiles or Visual Studio solutions, allowing for cross Linux / Windows builds. Support for Ninja files also seems to speed up builds in my experience. (Ninja is a Makefile "assembly language", minimal features, assuming that things like CMake files compile into Ninja). Ninja is fully multithreaded and focused on speed instead of features, and it really does show. Since ./configure is single-threaded, a substantial amount of time needs to be spent whenever you make changes to your Autoconf / Automake stuff. CMake has a few nice things: a built in basic testing framework (just interpreting the exit codes of programs: 0 is success and anything else is test-failure). Said testing framework has been integrated into various other test frameworks (like Googletest: https://cmake.org/cmake/help/latest/module/GoogleTest.html https://cmake.org/cmake/help/latest/module/GoogleTest.html).
- galangalalgol 6y agoAnd good integration with conan. Dependency management is nice. Vcpkg is getting there too.
- jnwatson 6y agoThe burning question is: is Autotools worth it these days? In my early days of programming, I thought Autotools was the coolest thing in the whole world. It was essentially magic. These days, with the demand for running on non-Linux UNIX at an all time low, why put yourself through all this?
- aparsons 6y agoFor most programming (ie - web services), no. For anything that makes system calls within 1-2 layers of abstraction (graphics programming, shell utilities, etc) it is a godsend
- maxlybbert 6y agoI would be willing to spend an incredible effort to keep something POSIX instead of using autotools ( https://varnish-cache.org/docs/2.1/phk/autocrap.html https://varnish-cache.org/docs/2.1/phk/autocrap.html ). They may have been an acceptable solution to a problem before the turn of the century, but things have changed enough that I’m not convinced they’re still delivering much bang for the buck.
- wahern 6y agoAnd yet Varnish is still using not only autoconf, but also automake. I've never had the need for automake, but autoconf feature tests are still useful. Not because of POSIX compat headaches, but because of the steady addition of new interfaces and the reimplementation of old ones. musl libc hadn't even made its initial release when the above tirade was written, yet there's a good chance Varnish built against musl because of those autoconf feature tests. This comes from someone who has invested a significant amount of time trying to implement pretty much exactly what PHK is advocating: https://github.com/wahern/autoguess https://github.com/wahern/autoguess I don't reach out for autoconf reflexively, but I fully appreciate why people do make use of it.
- jnxx 6y agoApart from that the embedded universe needs to support a lot of CPU architectures, the demand for building on different hardware and slightly different library interfaces is unlikely to disappear. Just think in desktop apps using Qt4 / Qt5 or libraries which require CUDA support for different kinds of hardware.
- stabbles 6y agoToday I had to use autotools for the first time, where I previously only used CMake. I thought I should be scared, and m4 looks daunting, but what they at least get right is that it's trivial to generate both shared and static libraries, cross-compilation is a breeze, and in the end all you need is a shell to configure and make to install. CMake defaults to either shared or static libs, and sometimes you have to read the docs for a project to figure out what custom variables influence this; and cross-compilation is almost always guaranteed not to work because some find_x cmake script tries to compile and run a C source to figure out certain properties about the target (but it confuses that with the host system).
- haolez 6y agoI think autotool's biggest limitation is its UX: it's very difficult to learn what are the good practices, what's deprecated, what new resources are available to make your build scripts more robust. Discoverability is poor. Every autotools project that I come across is a mess of copy/paste from another autotools project, with unneeded (and sometimes wrong) checks and complexity.
- nightowl_games 6y agoSounds just like cmake
- jnxx 6y agoIn my experience, it is much better than cmake: Apart from having good and comprehensive introductory documentation, it has a nicely ordered reference and also, it is stable, while cmake is changing, sometimes in backward-incompatible ways, and you find a lot of information which is disconnected and fragmentary and one does not know to which versions it applies. The latter will also, by all experience I have, make it much much harder to maintain and fix cmake projects in the long run. Think in a "modern"C++ code base which is 15 years old and in which every developer tried to include its favorite language constructs.
- Pinus 6y ago
- hzhou321 6y agoAutotools is a pain for development. autoreconf is slow, configure is super slow, and the Makefile generated by automake is a monster that adds seconds just to do a no-op make and it is impossible to hack. So for development, I simply hack up my own scripts to generate the config headers, directly parse the Makefile.am (which is surprisingly easy to parse) and generate my own Makefile. This works well because development only need a specific environment and a simplified Makefile. But I value autotools and keep maintaining the configure.ac, because it really saves you a lot of trouble in supporting users.
- flukus 6y agoConfigure and autoreconf may not be the fastest, but they aren't run frequently enough to really bother me. I'd have to disagree with the generated makefile being slow though, I still find it by far the fastest build system. I've done the whole custom makefiles as well, but by the time you have tests, installation, etc it just becomes a burden to maintain them.
- jnxx 6y agoI'd agree with that. If a build happens on a build server, it does hardly matter if it takes an extra minute. What matters for developing and debugging stuff are fast incremental builds. And for this, one just needs to type "make", as it is not needed to run configure again unless one changes to a different system.
- hzhou321 6y ago> I've done the whole custom makefiles as well, but by the time you have tests, installation, etc it just becomes a burden to maintain them. It is generated from the same `Makefile.am`, so no extra burden to maintain. It is just a simplified version of the Makefile. With my project, it is ~1000 lines vs ~44000 lines, so it does make a huge difference in its latency.
- tyingq 6y agoPerl's build system (Metaconfig) is fairly similar to autoconf. I've always been impressed on how many disparate systems, even decidedly non-unixy ones, it handles. You can tell, though, it was a lot of work to get to that state.
- 3v1n0 6y agoAfter years fighting with slowness and unmaintainability of Autotools, using meson is just a breeze. You can get it all, quickly and cleanly.
- millstone 6y agoWhat does "get it all" mean? All of what?
- 3v1n0 6y agoI mean, all the features
- jnxx 6y agoThere are some speed comparisons, measurements, and discussion here: https://lwn.net/Articles/706404/ https://lwn.net/Articles/706404/ https://thiblahute.github.io/jpakkane.github.io/Simple-comparison.html https://thiblahute.github.io/jpakkane.github.io/Simple-compa... The thing is apparently, ninja is indeed faster, but this matters only for very large projects with tens of thousands of files. Speed of incremental builds matters, and in this respect, ninja is an advance, but is not a day-and-night difference either.
- 3v1n0 6y agoThe speed isn't much related to the compilation itself... Make against ninja matters just for build machines. What it matters for developers is the development time, and doing things in automake takes 3-4 times of Autotools. Not to mention that the configuration time you loose waiting for a single line change.
- 3v1n0 6y agoI meant "3-4 times of meson"
- everybodyknows 6y agoComments on value of autotools from the author of the most recent release, 2.70: https://news.ycombinator.com/item?id=25910496 https://news.ycombinator.com/item?id=25910496
- Pinus 6y agoMy main gripe with autotools these days is that there seems to be no standard mechanism for specifying non-standard locations for dependencies. (Or if there is, that no packages use it...) If I build something from source, it is nearly always because I want a newer version than what is available from the appropriate package repositories, which usually means that I will want newer versions of the dependencies too, or because I want a built-from-source version of one of those dependencies. If I'm really lucky, pkg-config will pick them up. Sometimes just setting LDFLAGS=-L/my/build CPPFLAGS=-I/my/build will work. Sometimes I have to say --with-gnomovision=/my/build, sometimes I will have to say --with-gnomovision-includes=/my/build/include, ...
- jnxx 6y agoIf you want to build and install custom packages which in turn depend on libraries which are newer than system packages, this sounds like a perfect use case for GNU Guix as a package manager (e.g. on top of Debian or Arch): http://guix.gnu.org/cookbook/en/guix-cookbook.html#Advanced-package-management http://guix.gnu.org/cookbook/en/guix-cookbook.html#Advanced-... With this, you can create isolated environments in which you can work on specific software, and use and go back to arbitrary versions, without cluttering the base system with local installations. One of the best features is that defining an additional package oneself is merely an extension of the package definitions available with Guix: https://guix.gnu.org/cookbook/en/html_node/Packaging-Tutorial.html https://guix.gnu.org/cookbook/en/html_node/Packaging-Tutoria... And, of course, it works well with autotools....
- jnxx 6y agoJust to add, Guix configuration does not only look declarative - it uses a mostly functional language in a side-effect style to define the builds, in the same way as NixOS, just with a language which is established, well-supported, minimal, and elegant. One of the advantages is that this functional package definitions define very clearly what is provided by a package. It also works well for other languages like Rust or Clojure.
- arthur2e5 6y agoI think this is something to be solved with pkg-config(1). Autotools already uses pkg-config to find dependencies and query for any extra flags needed, and with PKG_CONFIG_PATH you can have the program prioritize your own installation.