17 ms·

Future Plans for Autotools

160 points by edward 6y ago

worik 6y ago
Really have to agree with the doubters. I have used Autotools since the nineties and I have never been able to understand it or make use of it for a fresh project (well once, using some gui tool that completely abstracted away the pain, was incompatible with anything else, and who's name escapes me now) In its day it was a wonder: tar xf; cd ... ; ./configure --prefix=`pwd` ; make ; make test (if you are very lucky) ; make install But times have changed. Interfaces are much more standardised now, and from where I sit it is the "polyglot" problem that is interesting, not different architectures. It is time to pick a winner from all the multitude of tools out there and GNU should go with that....
- enriquto 6y ago
  > It is time to pick a winner from all the multitude of tools out there and GNU should go with that.... plain Makefiles without configuration that work on linux and macos out of the box. Those are musch easier to write than to setup autotools or write a cmakelists.
  - pjmlp 6y ago
    If that is your bubble.
turminal 6y ago
I wish someone added a Makefile backend to meson. That would make it a much more appealing as an alternative to the autotools mess.
- 7786655 6y ago
  Why? You need Python to run Meson anyway, might as well install Ninja too.
  - turminal 6y ago
    You don't actually need python for ninja. Someone wrote a compatible tool in C called samurai.
- vmchale 6y ago
  meson has the problem of being written in Python and thus even an end-user has to endure Python library distribution.
  - turminal 6y ago
    And autotools have the problem of requiring perl, m4, and some other stuff i think. And cmake requires C++. I take python over all of these anytime. That said, it's still far from ideal.
    - pjmlp 6y ago
      > cmake requires C++ One more reason for C++ projects to settle with CMake,
    - vmchale 6y ago
      M4 doesn't need to be installed on the software distribution's computer (that was my point). Perl I don't even know if it needs to be installed on the packager's computer.
racingmars 6y ago
I've developed negative feelings toward autotools over time, but I'm not sure how much I should be blaming autotools versus the projects that use autotools. One annoyance is that it often feels like "./configure ..." takes longer to run than the actual "make". But my main annoyance derives from that fact that most projects I build from source which use autotools don't build cleanly on anything but Linux x86/amd64. I spend a lot of time playing with non-Linux and/or non-x86 Unix systems, or using vendor compilers instead of gcc -- primarily Solaris (both x86 and SPARC), Linux on POWER, and AIX on POWER. Current versions of IBM's compilers, Sun (err...Oracle's) compilers, etc. Run the configure script, let it slowly probe all the little corners of whatever it's probing, and sometimes I'm lucky enough that it completes successfully and claims to understand the environment it's in. Then run make (or, as is often required, GNU make), and hit build errors very quickly. Which makes me think... why did I just sit through that long "configure" script run, and why did the developer bother messing with autotools, if this project isn't even going to bother targeting anything but Linux with the GNU toolchain to build it? If you're not concerned about portability, it seems a relatively simple Makefile would be an easier and faster approach. Why foist the [insert your favorite colorful descriptor here, depending on your personal feelings] autoconf system on builders of your project?
- jjoonathan 6y ago
  Yeah, the autotools model of portability didn't stand the test of time, but there's no shame in that. It's in good company. Isn't CMake the usual tool these days?
  - rectang 6y ago
    Autotools, during its lifetime, has provide tremendous value across innumerable scenarios and systems. What does it matter if it didn't stand the "test of time" — especially if it shone a light for its successors to follow? The Autotools model of probing for system capabilities at build time isn't dead; it will never die, it will only be used in greater and lesser amounts and to complement other means of discovering system capabilities. CMake seems to me like a reimplementation of the Autotoools model of build-time probing. The true competitors are binary distribution after building on precursor systems with full specification and no probing, and cross-compilation with full specification and no probing.
ufo 6y ago
From the point of view of an end user, I love compiling projects that use autotools. Download a tarball and then compile make install without needing to install a bunch of dependencies just for the build system. You can also trust that standard command line flags like --prefix and so on will be there, and do what you expect. However, from the point of view of a developer, autotools are way too much for my brain. The m4 macros are inscrutable and I never felt like I had any hope of actually undesrtanding how they work. It's one of those technologies that I my only hope of getting work done is by copy pasting snippets of code I got from other people. Anyway, does anyone know if there are alternative build systems that follow the same paradigm as autotools, but more pleasant to use as a developer?
- curtis3389 6y ago
  IIRC from my C++ days, there's no alternative to Autotools that keeps the same pattern of configure, make, and make install. The big alternatives I remember are: SCons, Maven, and CMake. I liked CMake the most; Autotools was nearly unusable for me. It seemed like I needed to learn almost as much about Autotools to get productive as I did for C++. If I'm wrong about anything, someone will come along to correct me.
  - tomjakubowski 6y ago
    If you squint, `cmake ..` looks like `./configure`. From there, you `make` and `make install`.
    - kelnos 6y ago
      Part of the issue is also the lack of standardized flags. I will remember "./configure --prefix=/opt/foo" until the day I die, but "cmake -DCMAKE_INSTALL_PREFIX=/opt/foo" is something I have to look up every time (like I did just now to type it here).
      - unnah 6y ago
        That could be solved by a "configure" script that translates its arguments to ones that cmake will understand. And it seems it's such a good idea that someone has already made one: https://github.com/Richard-W/cmake-configure-wrapper https://github.com/Richard-W/cmake-configure-wrapper
aaron-santos 6y ago
It's been nearly 10 years since I last touched an autotools project. Recently I was re-exposed to autotools when trying to setup Underworld2[1] for what I thought would be a fun simulation project. Underworld2 requires PETSc, and OpenMPI. Finding the right environment variables so that ./configure would work with CUDA and each would reference the other was nightmarish. Who knows if it was autotools-related or something else that broke. The point is that even after 13 years of experience programming I am in no way equipped to deal with that stack to even find out. [1] https://underworld2.readthedocs.io/en/latest/ https://underworld2.readthedocs.io/en/latest/
kstrauser 6y ago
I wonder how much cruft could be thrown away if the project tossed out support for, say: - Everything but Linux and a few of the more popular BSDs - 32 bit CPUs - CPUs with less than 1,000 installations - Ancient toolsets that don't support features everyone else takes for granted in the last 20 years For instance, it has to be a pain in the neck to sit on a code base with Solaris 2.6 support, or NetBSD on Alpha, or... Also, it seems like there'd have to be some legacy pain around things like "we can't use this flag on sed because Amix didn't have it". But I don't know enough about the infrastructure to know if pruning those out would make a bit of engineering difference. How much is support for ancient or little-used stuff slowing down development? Edit: Alternatively, I wonder what it would look like to have a build farm that precomputed all the values. "Oh, you're on macOS 11.1 on Intel? Here's the list of 32 envvars you need to set." If a million people are compiling the same file on a million identical computers, is it a great idea for all of them to have run the same probes? Edit 2: I'd never advocate for support for such systems to be scrubbed from the Internet. I'm just saying it's unreasonable to expect a maintainer today to support ancient systems. The people still using those systems have the right to fork the tools and maintain their own version, but that's not the same as making upstream do it for them.
- yjftsjthsd-h 6y ago
  If you're going to cut off the long tail of compatibility, what's the point of using autotools? I thought the whole point of autotools was to deal with the (painful) diversity of operating systems.
  - stefan_ 6y ago
    That diversity no longer exists. These machines only run in museums, if at all.
    - deleted 6y ago
      [deleted]
    - ISL 6y ago
      Plenty of industrial systems have been working fine, unchanged, for decades. An example off the top of my head -- a CNC mill at my previous job, purchased new perhaps ten years ago, runs DOS 6.22 with Windows CE on top. I suspect that the vendor is still selling the same software stack on the same hardware today. Before someone chimes in to say, "That's insane, why isn't it running an RTOS?": That mill had zero software faults in the years I worked with it. Edit: Poking around Trak's website, it looks like the software on today's machines is unchanged. Here are some example screenshots: https://www.southwesternindustries.com/software/page/prototrak-tmx-cnc-software-m11 https://www.southwesternindustries.com/software/page/prototr...
crb002 6y ago
Autotools is slow as balls - but then CMake configuration is slow too - a minute if I recall for llvm-project on my M1. Much of the modern Linux build issues are RPM vs DEB packages not playing nice. Rust Cargo has it's own issues with snowflake dependency hell instead of more generic libraries. GNU Make is a different story. Easy to get core saturation for large workloads. Stanford GG allows you to massively distribute it to thousands of workers. Still very relevant.
- vmchale 6y ago
  > Much of the modern Linux build issues are RPM vs DEB Which in turn are not the best formats for binary distributions.
diegocg 6y ago
Related: Rejuvenating Autoconf https://lwn.net/Articles/834682/ https://lwn.net/Articles/834682/
jonstewart 6y ago
Please just sunset it and recommend projects adopt better tools. And if not, please please please please deprecate libtool, which is the worst abomination. Making shared libraries with gcc and clang is not hard, and libtool should never be adopted for new projects (yet the GNU docs recommend it).
- intc 6y ago
  > deprecate libtool.. I don't quite understand this. What if one uses something else than clang or gcc on the target system?
  - jonstewart 6y ago
    Like an ancient compiler that’s also deprecated...?
  - JoshTriplett 6y ago
    libtool isn't primarily about portability across compilers; libtool is about portability across linkers and library formats, from the days when there were many systems with deficient library support. It's not unreasonable to assume a modern linker, and modern library support.
    - rvp-x 6y ago
      People still use macOS, which is the main reason I see "please add libtool support" these days.
      - JoshTriplett 6y ago
        It's possible to handle libraries on macOS without libtool.
- dig1 6y ago
  I had the same opinion until I had to ship the app with shared libraries on different BSD versions and a few older Solarises, besides multiple Linux distributions. Even then, I wasn't sure it would work without problems. libtool is ugly because it tries to solve an ugly problem. To support the above, my manual Makefile was uglier than libtool :) These days, if I'd have to do that again, I'll pick libtool without overthinking. People need to learn why some things are as such and should see outside of Ubuntu/Windows/macOS comfort.
ximeng 6y ago
Telling response on why these systems don’t evolve to be more user friendly from “Gavin Smith”: It could be made easier to get started with automake. For example, "AUTOMAKE_OPTIONS=foreign" should be the default; otherwise automake refuses to finish if required files aren't found in the project. There's a political problem here to persuade people of this. The documentation could also be improved. I remember somebody was complaining about this page: https://www.gnu.org/software/automake/manual/html_node/Program-and-Library-Variables.html https://www.gnu.org/software/automake/manual/html_node/Progr... and asking what "maude" meant - it turned out it was the name of the dog or something of the person who first wrote the program, who didn't want it to change. So think getting consensus is impossible for changes that are needed to make the program simpler, easier and better. There's a vacuum of responsibility where nobody wants to step on others' feet. Forking the project would go nowhere as nobody would use the fork.
- djur 6y ago
  The explanation on that page that "maude" is being used as a placeholder for your program name seems pretty clear to me.
  - rleigh 6y ago
    I found it confusing in 1999/2000 when I first autotooled a project and read it for the first time. If others find it confusing, maybe it needs clarifying or changing to make the example clearer.
mshockwave 6y ago
> GNU project status discourages new contributors because of the paperwork requirements Can somebody elaborate this point? Does that mean I need to go through paperwork even for contributing a bug fix?
- ebiester 6y ago
  If it hasn't changed recently, you have to send in paperwork assigning your contribution to GNU as to maintain a single copyright owner.
  - lmm 6y ago
    Plenty of projects require a CLA, which seems pretty similar paperwork-wise?
    - rleigh 6y ago
      Absolutely similar, but don't underestimate the friction this introduces, and the number of potential contributors you cut off by doing this. Getting corporate sign-off for such things can take months while it wends its way through legal departments and upper management. Many people won't even bother, it's such a hassle. Last one I did was for Apache. It took at least six months. Previously did several for GNU which were much quicker (in a different company).
- rleigh 6y ago
  Yes. You have to assign copyright to the FSF in order to contribute anything. This is one reason for its demise. I've done the copyright assignment for both Autoconf and Automake. It's one reason why there are external macro archives (e.g. Autoconf Archive), when much of this functionality could be in the core projects. Example: Why, in 2021, should you have to rely on external support to do something as trivial as enabling multithreading [ACX_PTHREAD]. It's not even portable to non-pthreads platforms. So much for portability. Contrast with CMake [find_package(Threads); use Threads::Threads]. Compared with CMake, which doesn't require copyright assignment, and anyone can contribute to. It has hundreds of contributors, tons of built-in functionality, and more third-party extensions than Autotools could ever dream of. Copyright assignment is a big part of that, but also so is upstream attitude to third-party contributions. Kitware do this very well. The Autotools maintainers not so much; partly down to their own historical bad choices which prevented change to the internals due to the potential for breaking compatibility.
etaioinshrdlu 6y ago
I don't often say this but they should probably throw it away and rewrite it from scratch with a smaller vision. There is no part of the current code that should actually be kept. There is no place for a pile of Bourne, m4, and perl in today's stack.
- geofft 6y ago
  It's difficult because the API is m4-templated shell (configure.ac, macros) and makefiles, and so you can't easily handle any existing autoconf project without using what is effectively an implementation of m4, shell, and make. Yes, projects themselves could move to something else, but there are many options for that (mentioned in the article). I'm not sure how much value an incompatible autotools-flavored tool would have to anyone. (If you're not templating shell, there are way better flavors of build tool, anyway.) This is, incidentally, a software design lesson: if your input space is too big and you basically encourage people to see straight through your abstractions, you have no abstractions. (I'm facing this at work: we have an intricate set of homegrown makefiles that evolved over some 15 years, and make is hard to teach and debug, but it's pretty difficult to switch to anything else because we allowed people to write custom makefile rules too and there are lots of them. We can automatically convert the easy cases, which are the majority, but then we'd have two systems, which is worse.)
- enriquto 6y ago
  > There is no place for a pile of Bourne, m4, and perl in today's stack What do you have against m4? It's a lovely little macro language.
  - db48x 6y ago
    Textual macro languages are evil. Use anything else if you have any choice in the matter.
    - db48x 6y ago
      In fact, I’m not sure that I’ve stated that strongly enough. I’ve recently written a non–trivial amount of M4 to generate Rust, and I think that M4 is worse than PHP (which I’ve also written a non–trivial amount of, sadly). https://github.com/db48x/flex/blob/retargeted-to-rust/src/rust-flex.skl https://github.com/db48x/flex/blob/retargeted-to-rust/src/ru... This kind of macro language is really the worst way to write any kind of software. It’s not maintainable, it’s not readable, there’s no way to collaborate with other developers on it. There’s no way to create abstractions, so you must keep all the details in mind at all times. You can’t put those details into a box and forget about them. It is Cthulu–esqe. It would be a chapter of The Book. Practicing M4, or any language in this family, has a permanent SAN cost. Just learning the language probably doesn’t have much cost, but even small sacrifices should not be made lightly.
hendry 6y ago
Love the SWOT writeup https://www.owlfolio.org/development/autoconf-swot/ https://www.owlfolio.org/development/autoconf-swot/ I honestly don't see the need for autotools for AT LEAST a decade now. Except for a steady wall of text to make it look like you were doing some cool hacking work.
smitty1e 6y ago
What the FSF needs is some marketing people who can come up with a better subscription model to support these requirements. I've been a member for years, but the business model (gasp) languishes for lack of a practical vision.
schoen 6y ago
Recent HN discussion (of an LWN article about this work, not of this GNU mailing list thread): https://news.ycombinator.com/item?id=24872032 https://news.ycombinator.com/item?id=24872032
souprock 6y ago
My memory of GNU autotools and friends, from when I shipped one release of a project with it: * the size of my project doubled * naturally, autotools would check for FORTRAN despite my project being in C * if users built the software as root, it would trash critical system files * instead of installing ELF library files in /lib, it installed non-functional shell scripts in /usr/lib * cross compiling broke The solution was plain old make. You can do a lot with make. It performs really well. I even did a "make install" that would depend on the final locations for everything. It was all configurable. It worked great.
- nrclark 6y ago
  I've used autotools on a ton of my projects, and I like it a lot. I can't speak to any of the problems you had. The one thing I'll say about Autotools is that the learning curve is pretty steep, and it took a fair amount of experimentation to get it right on my first Autoolized project. Maybe you never made it all the way over the hump?
gcblkjaidfj 6y ago
this could be an interesting project to agree on a portable subset of a modern language that can be either interpreted or compiled in the background so quickly that you wouldn't mind. Something like portable-rust. Could replace bash/m4/whatever else auto tools use (and it uses a lot of stuff bolted on) Would also drive contributors as just mentioning a new languages on your README floods your issue tracker these days :)
nh2 6y ago
I got some cynical thoughts reading this well-done summary: > There are few active developers and no continuing funders. Like most open-source projects. > There is no continuous integration Like Signal. > Bugs, feature requests, and submitted patches are not tracked systematically. Like PostgreSQL. > there is code shared among Autoconf, Automake, and/or Gnulib by copying files between source repositories Like libmtp / libgphoto2.
cpeterso 6y ago
How many of Autotools' configure-time checks could just be replaced with a giant static header file of platform #ifdefs?
chaz6 6y ago
I respect the years of work that have gone into autotools, and the vast ecosystem of projects it has made possible, but nowadays I use cmake whenever possible, mainly because it can generate ninja build files. It would be great to see autotools on https://github.com/ninja-build/ninja/wiki/List-of-generators-producing-ninja-build-files https://github.com/ninja-build/ninja/wiki/List-of-generators...
account42 6y ago
Why do we have so many different configure / build systems and none of them can run their config checks in parallel. Compiling Gentoo packages on a 32 core 64 threadripper is getting ridiculous with many packages, even some "larger" ones, taking longer to configure than to actually compile.
dig1 6y ago
I'm sad that M4 is getting a bad reputation; probably it is more due to lack of knowledge than the actual tool, because the language is really simple. As usual, many devs would copy/paste m4 macros without understanding what is going on. Also, heavy use by Sendmail or SELinux didn't add to reputation either (again, not due M4 itself). But I find M4 a true gem "no one heard of" - a portable macro processor you can use with basically anything involved with the plain text. Especially it is useful where you have tons of boilerplate code to type (Java, Ansible, Go, js/html/css to name a few).
josteink 6y ago
While I don’t hold much love for autoconf and autotools as full suites to work with, count me in as someone who appreciates GNU make and simple makefiles. I use it for lots of things, even NodeJS, and it helps me keep my builds more portable. I’d hate to see GNU make completely abandoned.
- nrclark 6y ago
  GNU Make is widely used, and actively maintained. New features are added regularly. It's the build tool targeted by Kconfig (used in the Linux kernel and elsewhere), and also it's probably true to call it CMake's most common backend. So, don't worry - GNU Make isn't going anywhere, regardless of what eventually happens to Autoconf.
mangecoeur 6y ago
I've always found it emblematic of the basic problems of this community that they choose to a communicate through mailing lists with an online interface that looks crap no matter how you try to view it. Like, if they had dumped the text into an html <body> tag it would have re-flowed gracefully, but somehow they broke that too, and have not managed to do anything about it in decades.
- globular-toast 6y ago
  You can't just "cast" it as html and let it reflow. Email has significant whitespace and writers use it a lot. Do you have examples of projects that have used a "better" method of communication that have achieved what GNU, Linux or any of the BSDs have achieved?
- Anthony-G 6y ago
  The OP, Zack Weinberg helpfully published his analysis on his own web page¹ and the formatting (different typeface for headings, flowing paragraph text, hypertext links, bullet points) made the analysis much more readable. I think email is fine as the communication medium for software projects – particularly for projects that have been around for as long as Autotools†. For me, the biggest downside is that not all mailing lists use GNU mailman or other listserv software that provide archiving of list posts. Having a searchable archive of discussions is invaluable. † though I was surprised to see a number of contributors top-posting; that practice was universally discouraged on Internet mailing lists until the number of new members using MS Outlook overwhelmed members who followed the established “netiquette” 1. https://www.owlfolio.org/development/autoconf-swot/ https://www.owlfolio.org/development/autoconf-swot/
  - mangecoeur 6y ago
    So we agree that the online presentation is not good - I have no issue with email particularly, i have a problem with the email archives. More specifically, the fact that no one cares about the usability of the archive or online presentation is proof that they don't care about the impression they make external people. They can hardly complain about the lack of new contributors if they make no effort to be accessible to them.
rleigh 6y ago
Can't disagree with most of the points made here, except for a handful. Firstly, that the "Autotools are not obsolete". Sorry, but they are. They have been for well over a decade. They don't serve today's needs effectively, and that is the reason for the decline in contributors and end users. They moved on. And that includes me too (I did the initial support for multiple language standards in Autoconf, starting with C99). Support for new language standards and features in Autoconf is woeful. Support for integration with various libraries, tools and frameworks is woeful. Other systems support them out of the box. As for whether support for them should continue. That depends upon what individual projects need in terms of portability requirements. Clearly a lot of projects have multiple decades of investment into the Autotools. There is a sunk cost here, and replacement is costly. Despite that, I've converted all of my own projects, plus many other projects I work with, over to CMake. In every case it was a net improvement, and well worth the migration hassle. Many projects overestimate the difficulty. In doing these migrations, I've seen many examples of buggy use of the Autotools. It only provides "theoretical portability". Unless you actively test it, it's most likely broken. In direct comparisons during project conversions, CMake has worked on various minor platforms (Cygwin, MinGW, Solaris) where the Autotools support was broken for years and no one noticed. That is to say, direct empirical evidence showed worse Unix portability for the Autotools than CMake. If the Autotools want to retain relevance, the main hurdle they need to cross is portability to modern platforms. They can't stick to Unix-only portability problems of the '90s and remain relevant. So many projects have needs beyond that, and that is the main reason for the declining use. Fix that, and people might have a reason to stay. Unfortunately, if you follow the mailing list and such, you'll see that most of the long-time developers and users have zero interest in portability outside the niche they have inhabited for the past 25 years. They don't understand how the rest of the world works or what their needs are. There is no comprehension about using tools other than shell and make. The developers who did care about this stuff moved on. I became a CMake contributor instead. Newer build systems moved on to ninja and other tools, support various IDEs, and integrate with all sorts of other systems and tools. The Autotools don't do any of that, and likely never will. Yet for most developers, these are non-negotiable. Want to use CLion, Visual Studio or Eclipse?
vkwallace 6y ago
Very nice writeup, I hope autotools will continue the good work. It is still the best build system. - If you ship a trimmed down version of ./configure, it will run quite fast. Many people copy&paste configure.ac from other projects and do not delete unnecessary tests. - It is easy for users of other systems to add portability patches. With ./configure you can do anything, not so with meson or cmake. - ./configure's command line syntax is still the most elegant compared to the enterprise syntax of cmake and meson. --prefix=/home/foo/bar vs. -DCMAKE_INSTALL_PREFIX=/home/foo/bar. Specifying CFLAGS in cmake is painful. - Unlike cmake or meson, autotools do not dictate a rigid source tree structure. - You can easily build both shared and static libraries, which the "modern" build systems do not support. - Cross compiling on Linux with autotools/gcc is heaven compared to cmake/clang. You just need the host triplet and you are done. - The build output is exactly what I want to see. cmake and meson/ninja hide the output by default. If you enable it using yet another arcane option, the output is more chaotic and unreadable than in most autotools projects. In general I find autotools projects like gcc easier to compile. The flags are more intuitive, the output is organized and readable. Compare that to cmake projects like llvm, which always seem to expand the option set to more than 100 unintuitive -DENABLE_ARCANE_FOOBAR_QUUX="a;b;...;z" flags. Autotools is very much appreciated. Thank you for another release, I am looking forward to future ones!
Sniffnoy 6y ago
The letter mentions that "building an Autotools-based project directly from its VCS checkout is often significantly harder than building it from a tarball release, and may involve tracking down and installing any number of unusual tools." Why would that be? They both just seem like a source distribution to me... why would there be a difference? Can anyone explain this?
- db48x 6y ago
  The tarball distribution includes the output of the Automake and Autoconf tools (the Makefile.in and configure.sh files, respectively), while building from a VCS checkout does not. Thus, building from a checkout requires extra steps and software that you may not already have ready to hand.
- deleted 6y ago
  [deleted]
- jsmith45 6y ago
  A source tallball already has the Makefile.in and configure script pre-generated, along with a few other autotools generated files. These expect `make`, the compiler, and needed libraries are installed, and some other generic unix tools, but do not expect users to have any GNU specific software installed. (Like the autotools). The source repositories typically do not have these already generated (since checking in generated files is frequently frowned upon), and building them often require getting the right version of autoconf/automake/etc installed, as a different version may not be fully compatible.
  - jjgreen 6y ago
    I used to not include autoconf generated artefacts in my repos, but with .gitattributes making them sort-of invisible and the reduction in setup aggravation for other devs, I've completely changed my mind.

160 points by edward 6y ago

Really have to agree with the doubters. I have used Autotools since the nineties and I have never been able to understand it or make use of it for a fresh project (well once, using some gui tool that completely abstracted away the pain, was incompatible with anything else, and who's name escapes me now) In its day it was a wonder: tar xf; cd ... ; ./configure --prefix=`pwd` ; make ; make test (if you are very lucky) ; make install But times have changed. Interfaces are much more standardised now, and from where I sit it is the "polyglot" problem that is interesting, not different architectures. It is time to pick a winner from all the multitude of tools out there and GNU should go with that....

enriquto 6y ago
> It is time to pick a winner from all the multitude of tools out there and GNU should go with that.... plain Makefiles without configuration that work on linux and macos out of the box. Those are musch easier to write than to setup autotools or write a cmakelists.
- pjmlp 6y ago
  If that is your bubble.

turminal 6y ago

I wish someone added a Makefile backend to meson. That would make it a much more appealing as an alternative to the autotools mess.

7786655 6y ago
Why? You need Python to run Meson anyway, might as well install Ninja too.
- turminal 6y ago
  You don't actually need python for ninja. Someone wrote a compatible tool in C called samurai.
vmchale 6y ago
meson has the problem of being written in Python and thus even an end-user has to endure Python library distribution.
- turminal 6y ago
  And autotools have the problem of requiring perl, m4, and some other stuff i think. And cmake requires C++. I take python over all of these anytime. That said, it's still far from ideal.
  - pjmlp 6y ago
    > cmake requires C++ One more reason for C++ projects to settle with CMake,
  - vmchale 6y ago
    M4 doesn't need to be installed on the software distribution's computer (that was my point). Perl I don't even know if it needs to be installed on the packager's computer.

racingmars 6y ago

I've developed negative feelings toward autotools over time, but I'm not sure how much I should be blaming autotools versus the projects that use autotools. One annoyance is that it often feels like "./configure ..." takes longer to run than the actual "make". But my main annoyance derives from that fact that most projects I build from source which use autotools don't build cleanly on anything but Linux x86/amd64. I spend a lot of time playing with non-Linux and/or non-x86 Unix systems, or using vendor compilers instead of gcc -- primarily Solaris (both x86 and SPARC), Linux on POWER, and AIX on POWER. Current versions of IBM's compilers, Sun (err...Oracle's) compilers, etc. Run the configure script, let it slowly probe all the little corners of whatever it's probing, and sometimes I'm lucky enough that it completes successfully and claims to understand the environment it's in. Then run make (or, as is often required, GNU make), and hit build errors very quickly. Which makes me think... why did I just sit through that long "configure" script run, and why did the developer bother messing with autotools, if this project isn't even going to bother targeting anything but Linux with the GNU toolchain to build it? If you're not concerned about portability, it seems a relatively simple Makefile would be an easier and faster approach. Why foist the [insert your favorite colorful descriptor here, depending on your personal feelings] autoconf system on builders of your project?

jjoonathan 6y ago
Yeah, the autotools model of portability didn't stand the test of time, but there's no shame in that. It's in good company. Isn't CMake the usual tool these days?
- rectang 6y ago
  Autotools, during its lifetime, has provide tremendous value across innumerable scenarios and systems. What does it matter if it didn't stand the "test of time" — especially if it shone a light for its successors to follow? The Autotools model of probing for system capabilities at build time isn't dead; it will never die, it will only be used in greater and lesser amounts and to complement other means of discovering system capabilities. CMake seems to me like a reimplementation of the Autotoools model of build-time probing. The true competitors are binary distribution after building on precursor systems with full specification and no probing, and cross-compilation with full specification and no probing.

ufo 6y ago

From the point of view of an end user, I love compiling projects that use autotools. Download a tarball and then compile make install without needing to install a bunch of dependencies just for the build system. You can also trust that standard command line flags like --prefix and so on will be there, and do what you expect. However, from the point of view of a developer, autotools are way too much for my brain. The m4 macros are inscrutable and I never felt like I had any hope of actually undesrtanding how they work. It's one of those technologies that I my only hope of getting work done is by copy pasting snippets of code I got from other people. Anyway, does anyone know if there are alternative build systems that follow the same paradigm as autotools, but more pleasant to use as a developer?

curtis3389 6y ago
IIRC from my C++ days, there's no alternative to Autotools that keeps the same pattern of configure, make, and make install. The big alternatives I remember are: SCons, Maven, and CMake. I liked CMake the most; Autotools was nearly unusable for me. It seemed like I needed to learn almost as much about Autotools to get productive as I did for C++. If I'm wrong about anything, someone will come along to correct me.
- tomjakubowski 6y ago
  If you squint, `cmake ..` looks like `./configure`. From there, you `make` and `make install`.
  - kelnos 6y ago
    Part of the issue is also the lack of standardized flags. I will remember "./configure --prefix=/opt/foo" until the day I die, but "cmake -DCMAKE_INSTALL_PREFIX=/opt/foo" is something I have to look up every time (like I did just now to type it here).
    - unnah 6y ago
      That could be solved by a "configure" script that translates its arguments to ones that cmake will understand. And it seems it's such a good idea that someone has already made one: https://github.com/Richard-W/cmake-configure-wrapper https://github.com/Richard-W/cmake-configure-wrapper

aaron-santos 6y ago

It's been nearly 10 years since I last touched an autotools project. Recently I was re-exposed to autotools when trying to setup Underworld2[1] for what I thought would be a fun simulation project. Underworld2 requires PETSc, and OpenMPI. Finding the right environment variables so that ./configure would work with CUDA and each would reference the other was nightmarish. Who knows if it was autotools-related or something else that broke. The point is that even after 13 years of experience programming I am in no way equipped to deal with that stack to even find out. [1] https://underworld2.readthedocs.io/en/latest/ https://underworld2.readthedocs.io/en/latest/

kstrauser 6y ago

I wonder how much cruft could be thrown away if the project tossed out support for, say: - Everything but Linux and a few of the more popular BSDs - 32 bit CPUs - CPUs with less than 1,000 installations - Ancient toolsets that don't support features everyone else takes for granted in the last 20 years For instance, it has to be a pain in the neck to sit on a code base with Solaris 2.6 support, or NetBSD on Alpha, or... Also, it seems like there'd have to be some legacy pain around things like "we can't use this flag on sed because Amix didn't have it". But I don't know enough about the infrastructure to know if pruning those out would make a bit of engineering difference. How much is support for ancient or little-used stuff slowing down development? Edit: Alternatively, I wonder what it would look like to have a build farm that precomputed all the values. "Oh, you're on macOS 11.1 on Intel? Here's the list of 32 envvars you need to set." If a million people are compiling the same file on a million identical computers, is it a great idea for all of them to have run the same probes? Edit 2: I'd never advocate for support for such systems to be scrubbed from the Internet. I'm just saying it's unreasonable to expect a maintainer today to support ancient systems. The people still using those systems have the right to fork the tools and maintain their own version, but that's not the same as making upstream do it for them.

yjftsjthsd-h 6y ago
If you're going to cut off the long tail of compatibility, what's the point of using autotools? I thought the whole point of autotools was to deal with the (painful) diversity of operating systems.
- stefan_ 6y ago
  That diversity no longer exists. These machines only run in museums, if at all.
  - deleted 6y ago
    [deleted]
  - ISL 6y ago
    Plenty of industrial systems have been working fine, unchanged, for decades. An example off the top of my head -- a CNC mill at my previous job, purchased new perhaps ten years ago, runs DOS 6.22 with Windows CE on top. I suspect that the vendor is still selling the same software stack on the same hardware today. Before someone chimes in to say, "That's insane, why isn't it running an RTOS?": That mill had zero software faults in the years I worked with it. Edit: Poking around Trak's website, it looks like the software on today's machines is unchanged. Here are some example screenshots: https://www.southwesternindustries.com/software/page/prototrak-tmx-cnc-software-m11 https://www.southwesternindustries.com/software/page/prototr...

crb002 6y ago

Autotools is slow as balls - but then CMake configuration is slow too - a minute if I recall for llvm-project on my M1. Much of the modern Linux build issues are RPM vs DEB packages not playing nice. Rust Cargo has it's own issues with snowflake dependency hell instead of more generic libraries. GNU Make is a different story. Easy to get core saturation for large workloads. Stanford GG allows you to massively distribute it to thousands of workers. Still very relevant.

vmchale 6y ago
> Much of the modern Linux build issues are RPM vs DEB Which in turn are not the best formats for binary distributions.

diegocg 6y ago

Related: Rejuvenating Autoconf https://lwn.net/Articles/834682/ https://lwn.net/Articles/834682/

jonstewart 6y ago

Please just sunset it and recommend projects adopt better tools. And if not, please please please please deprecate libtool, which is the worst abomination. Making shared libraries with gcc and clang is not hard, and libtool should never be adopted for new projects (yet the GNU docs recommend it).

intc 6y ago
> deprecate libtool.. I don't quite understand this. What if one uses something else than clang or gcc on the target system?
- jonstewart 6y ago
  Like an ancient compiler that’s also deprecated...?
- JoshTriplett 6y ago
  libtool isn't primarily about portability across compilers; libtool is about portability across linkers and library formats, from the days when there were many systems with deficient library support. It's not unreasonable to assume a modern linker, and modern library support.
  - rvp-x 6y ago
    People still use macOS, which is the main reason I see "please add libtool support" these days.
    - JoshTriplett 6y ago
      It's possible to handle libraries on macOS without libtool.
dig1 6y ago
I had the same opinion until I had to ship the app with shared libraries on different BSD versions and a few older Solarises, besides multiple Linux distributions. Even then, I wasn't sure it would work without problems. libtool is ugly because it tries to solve an ugly problem. To support the above, my manual Makefile was uglier than libtool :) These days, if I'd have to do that again, I'll pick libtool without overthinking. People need to learn why some things are as such and should see outside of Ubuntu/Windows/macOS comfort.

ximeng 6y ago

Telling response on why these systems don’t evolve to be more user friendly from “Gavin Smith”: It could be made easier to get started with automake. For example, "AUTOMAKE_OPTIONS=foreign" should be the default; otherwise automake refuses to finish if required files aren't found in the project. There's a political problem here to persuade people of this. The documentation could also be improved. I remember somebody was complaining about this page: https://www.gnu.org/software/automake/manual/html_node/Program-and-Library-Variables.html https://www.gnu.org/software/automake/manual/html_node/Progr... and asking what "maude" meant - it turned out it was the name of the dog or something of the person who first wrote the program, who didn't want it to change. So think getting consensus is impossible for changes that are needed to make the program simpler, easier and better. There's a vacuum of responsibility where nobody wants to step on others' feet. Forking the project would go nowhere as nobody would use the fork.

djur 6y ago
The explanation on that page that "maude" is being used as a placeholder for your program name seems pretty clear to me.
- rleigh 6y ago
  I found it confusing in 1999/2000 when I first autotooled a project and read it for the first time. If others find it confusing, maybe it needs clarifying or changing to make the example clearer.

mshockwave 6y ago

> GNU project status discourages new contributors because of the paperwork requirements Can somebody elaborate this point? Does that mean I need to go through paperwork even for contributing a bug fix?

ebiester 6y ago
If it hasn't changed recently, you have to send in paperwork assigning your contribution to GNU as to maintain a single copyright owner.
- lmm 6y ago
  Plenty of projects require a CLA, which seems pretty similar paperwork-wise?
  - rleigh 6y ago
    Absolutely similar, but don't underestimate the friction this introduces, and the number of potential contributors you cut off by doing this. Getting corporate sign-off for such things can take months while it wends its way through legal departments and upper management. Many people won't even bother, it's such a hassle. Last one I did was for Apache. It took at least six months. Previously did several for GNU which were much quicker (in a different company).
rleigh 6y ago
Yes. You have to assign copyright to the FSF in order to contribute anything. This is one reason for its demise. I've done the copyright assignment for both Autoconf and Automake. It's one reason why there are external macro archives (e.g. Autoconf Archive), when much of this functionality could be in the core projects. Example: Why, in 2021, should you have to rely on external support to do something as trivial as enabling multithreading [ACX_PTHREAD]. It's not even portable to non-pthreads platforms. So much for portability. Contrast with CMake [find_package(Threads); use Threads::Threads]. Compared with CMake, which doesn't require copyright assignment, and anyone can contribute to. It has hundreds of contributors, tons of built-in functionality, and more third-party extensions than Autotools could ever dream of. Copyright assignment is a big part of that, but also so is upstream attitude to third-party contributions. Kitware do this very well. The Autotools maintainers not so much; partly down to their own historical bad choices which prevented change to the internals due to the potential for breaking compatibility.

etaioinshrdlu 6y ago

I don't often say this but they should probably throw it away and rewrite it from scratch with a smaller vision. There is no part of the current code that should actually be kept. There is no place for a pile of Bourne, m4, and perl in today's stack.

geofft 6y ago
It's difficult because the API is m4-templated shell (configure.ac, macros) and makefiles, and so you can't easily handle any existing autoconf project without using what is effectively an implementation of m4, shell, and make. Yes, projects themselves could move to something else, but there are many options for that (mentioned in the article). I'm not sure how much value an incompatible autotools-flavored tool would have to anyone. (If you're not templating shell, there are way better flavors of build tool, anyway.) This is, incidentally, a software design lesson: if your input space is too big and you basically encourage people to see straight through your abstractions, you have no abstractions. (I'm facing this at work: we have an intricate set of homegrown makefiles that evolved over some 15 years, and make is hard to teach and debug, but it's pretty difficult to switch to anything else because we allowed people to write custom makefile rules too and there are lots of them. We can automatically convert the easy cases, which are the majority, but then we'd have two systems, which is worse.)
enriquto 6y ago
> There is no place for a pile of Bourne, m4, and perl in today's stack What do you have against m4? It's a lovely little macro language.
- db48x 6y ago
  Textual macro languages are evil. Use anything else if you have any choice in the matter.
  - db48x 6y ago
    In fact, I’m not sure that I’ve stated that strongly enough. I’ve recently written a non–trivial amount of M4 to generate Rust, and I think that M4 is worse than PHP (which I’ve also written a non–trivial amount of, sadly). https://github.com/db48x/flex/blob/retargeted-to-rust/src/rust-flex.skl https://github.com/db48x/flex/blob/retargeted-to-rust/src/ru... This kind of macro language is really the worst way to write any kind of software. It’s not maintainable, it’s not readable, there’s no way to collaborate with other developers on it. There’s no way to create abstractions, so you must keep all the details in mind at all times. You can’t put those details into a box and forget about them. It is Cthulu–esqe. It would be a chapter of The Book. Practicing M4, or any language in this family, has a permanent SAN cost. Just learning the language probably doesn’t have much cost, but even small sacrifices should not be made lightly.

hendry 6y ago

Love the SWOT writeup https://www.owlfolio.org/development/autoconf-swot/ https://www.owlfolio.org/development/autoconf-swot/ I honestly don't see the need for autotools for AT LEAST a decade now. Except for a steady wall of text to make it look like you were doing some cool hacking work.

smitty1e 6y ago

What the FSF needs is some marketing people who can come up with a better subscription model to support these requirements. I've been a member for years, but the business model (gasp) languishes for lack of a practical vision.

schoen 6y ago

Recent HN discussion (of an LWN article about this work, not of this GNU mailing list thread): https://news.ycombinator.com/item?id=24872032 https://news.ycombinator.com/item?id=24872032

souprock 6y ago

My memory of GNU autotools and friends, from when I shipped one release of a project with it: * the size of my project doubled * naturally, autotools would check for FORTRAN despite my project being in C * if users built the software as root, it would trash critical system files * instead of installing ELF library files in /lib, it installed non-functional shell scripts in /usr/lib * cross compiling broke The solution was plain old make. You can do a lot with make. It performs really well. I even did a "make install" that would depend on the final locations for everything. It was all configurable. It worked great.

nrclark 6y ago
I've used autotools on a ton of my projects, and I like it a lot. I can't speak to any of the problems you had. The one thing I'll say about Autotools is that the learning curve is pretty steep, and it took a fair amount of experimentation to get it right on my first Autoolized project. Maybe you never made it all the way over the hump?

gcblkjaidfj 6y ago

this could be an interesting project to agree on a portable subset of a modern language that can be either interpreted or compiled in the background so quickly that you wouldn't mind. Something like portable-rust. Could replace bash/m4/whatever else auto tools use (and it uses a lot of stuff bolted on) Would also drive contributors as just mentioning a new languages on your README floods your issue tracker these days :)

nh2 6y ago

I got some cynical thoughts reading this well-done summary: > There are few active developers and no continuing funders. Like most open-source projects. > There is no continuous integration Like Signal. > Bugs, feature requests, and submitted patches are not tracked systematically. Like PostgreSQL. > there is code shared among Autoconf, Automake, and/or Gnulib by copying files between source repositories Like libmtp / libgphoto2.

cpeterso 6y ago

How many of Autotools' configure-time checks could just be replaced with a giant static header file of platform #ifdefs?

chaz6 6y ago

I respect the years of work that have gone into autotools, and the vast ecosystem of projects it has made possible, but nowadays I use cmake whenever possible, mainly because it can generate ninja build files. It would be great to see autotools on https://github.com/ninja-build/ninja/wiki/List-of-generators-producing-ninja-build-files https://github.com/ninja-build/ninja/wiki/List-of-generators...

account42 6y ago

Why do we have so many different configure / build systems and none of them can run their config checks in parallel. Compiling Gentoo packages on a 32 core 64 threadripper is getting ridiculous with many packages, even some "larger" ones, taking longer to configure than to actually compile.

dig1 6y ago

I'm sad that M4 is getting a bad reputation; probably it is more due to lack of knowledge than the actual tool, because the language is really simple. As usual, many devs would copy/paste m4 macros without understanding what is going on. Also, heavy use by Sendmail or SELinux didn't add to reputation either (again, not due M4 itself). But I find M4 a true gem "no one heard of" - a portable macro processor you can use with basically anything involved with the plain text. Especially it is useful where you have tons of boilerplate code to type (Java, Ansible, Go, js/html/css to name a few).

josteink 6y ago

While I don’t hold much love for autoconf and autotools as full suites to work with, count me in as someone who appreciates GNU make and simple makefiles. I use it for lots of things, even NodeJS, and it helps me keep my builds more portable. I’d hate to see GNU make completely abandoned.

nrclark 6y ago
GNU Make is widely used, and actively maintained. New features are added regularly. It's the build tool targeted by Kconfig (used in the Linux kernel and elsewhere), and also it's probably true to call it CMake's most common backend. So, don't worry - GNU Make isn't going anywhere, regardless of what eventually happens to Autoconf.

mangecoeur 6y ago

I've always found it emblematic of the basic problems of this community that they choose to a communicate through mailing lists with an online interface that looks crap no matter how you try to view it. Like, if they had dumped the text into an html <body> tag it would have re-flowed gracefully, but somehow they broke that too, and have not managed to do anything about it in decades.

globular-toast 6y ago
You can't just "cast" it as html and let it reflow. Email has significant whitespace and writers use it a lot. Do you have examples of projects that have used a "better" method of communication that have achieved what GNU, Linux or any of the BSDs have achieved?
Anthony-G 6y ago
The OP, Zack Weinberg helpfully published his analysis on his own web page¹ and the formatting (different typeface for headings, flowing paragraph text, hypertext links, bullet points) made the analysis much more readable. I think email is fine as the communication medium for software projects – particularly for projects that have been around for as long as Autotools†. For me, the biggest downside is that not all mailing lists use GNU mailman or other listserv software that provide archiving of list posts. Having a searchable archive of discussions is invaluable. † though I was surprised to see a number of contributors top-posting; that practice was universally discouraged on Internet mailing lists until the number of new members using MS Outlook overwhelmed members who followed the established “netiquette” 1. https://www.owlfolio.org/development/autoconf-swot/ https://www.owlfolio.org/development/autoconf-swot/
- mangecoeur 6y ago
  So we agree that the online presentation is not good - I have no issue with email particularly, i have a problem with the email archives. More specifically, the fact that no one cares about the usability of the archive or online presentation is proof that they don't care about the impression they make external people. They can hardly complain about the lack of new contributors if they make no effort to be accessible to them.

rleigh 6y ago

Can't disagree with most of the points made here, except for a handful. Firstly, that the "Autotools are not obsolete". Sorry, but they are. They have been for well over a decade. They don't serve today's needs effectively, and that is the reason for the decline in contributors and end users. They moved on. And that includes me too (I did the initial support for multiple language standards in Autoconf, starting with C99). Support for new language standards and features in Autoconf is woeful. Support for integration with various libraries, tools and frameworks is woeful. Other systems support them out of the box. As for whether support for them should continue. That depends upon what individual projects need in terms of portability requirements. Clearly a lot of projects have multiple decades of investment into the Autotools. There is a sunk cost here, and replacement is costly. Despite that, I've converted all of my own projects, plus many other projects I work with, over to CMake. In every case it was a net improvement, and well worth the migration hassle. Many projects overestimate the difficulty. In doing these migrations, I've seen many examples of buggy use of the Autotools. It only provides "theoretical portability". Unless you actively test it, it's most likely broken. In direct comparisons during project conversions, CMake has worked on various minor platforms (Cygwin, MinGW, Solaris) where the Autotools support was broken for years and no one noticed. That is to say, direct empirical evidence showed worse Unix portability for the Autotools than CMake. If the Autotools want to retain relevance, the main hurdle they need to cross is portability to modern platforms. They can't stick to Unix-only portability problems of the '90s and remain relevant. So many projects have needs beyond that, and that is the main reason for the declining use. Fix that, and people might have a reason to stay. Unfortunately, if you follow the mailing list and such, you'll see that most of the long-time developers and users have zero interest in portability outside the niche they have inhabited for the past 25 years. They don't understand how the rest of the world works or what their needs are. There is no comprehension about using tools other than shell and make. The developers who did care about this stuff moved on. I became a CMake contributor instead. Newer build systems moved on to ninja and other tools, support various IDEs, and integrate with all sorts of other systems and tools. The Autotools don't do any of that, and likely never will. Yet for most developers, these are non-negotiable. Want to use CLion, Visual Studio or Eclipse?

vkwallace 6y ago

Very nice writeup, I hope autotools will continue the good work. It is still the best build system. - If you ship a trimmed down version of ./configure, it will run quite fast. Many people copy&paste configure.ac from other projects and do not delete unnecessary tests. - It is easy for users of other systems to add portability patches. With ./configure you can do anything, not so with meson or cmake. - ./configure's command line syntax is still the most elegant compared to the enterprise syntax of cmake and meson. --prefix=/home/foo/bar vs. -DCMAKE_INSTALL_PREFIX=/home/foo/bar. Specifying CFLAGS in cmake is painful. - Unlike cmake or meson, autotools do not dictate a rigid source tree structure. - You can easily build both shared and static libraries, which the "modern" build systems do not support. - Cross compiling on Linux with autotools/gcc is heaven compared to cmake/clang. You just need the host triplet and you are done. - The build output is exactly what I want to see. cmake and meson/ninja hide the output by default. If you enable it using yet another arcane option, the output is more chaotic and unreadable than in most autotools projects. In general I find autotools projects like gcc easier to compile. The flags are more intuitive, the output is organized and readable. Compare that to cmake projects like llvm, which always seem to expand the option set to more than 100 unintuitive -DENABLE_ARCANE_FOOBAR_QUUX="a;b;...;z" flags. Autotools is very much appreciated. Thank you for another release, I am looking forward to future ones!

Sniffnoy 6y ago

The letter mentions that "building an Autotools-based project directly from its VCS checkout is often significantly harder than building it from a tarball release, and may involve tracking down and installing any number of unusual tools." Why would that be? They both just seem like a source distribution to me... why would there be a difference? Can anyone explain this?

db48x 6y ago
The tarball distribution includes the output of the Automake and Autoconf tools (the Makefile.in and configure.sh files, respectively), while building from a VCS checkout does not. Thus, building from a checkout requires extra steps and software that you may not already have ready to hand.
deleted 6y ago
[deleted]
jsmith45 6y ago
A source tallball already has the Makefile.in and configure script pre-generated, along with a few other autotools generated files. These expect `make`, the compiler, and needed libraries are installed, and some other generic unix tools, but do not expect users to have any GNU specific software installed. (Like the autotools). The source repositories typically do not have these already generated (since checking in generated files is frequently frowned upon), and building them often require getting the right version of autoconf/automake/etc installed, as a different version may not be fully compatible.
- jjgreen 6y ago
  I used to not include autoconf generated artefacts in my repos, but with .gitattributes making them sort-of invisible and the reduction in setup aggravation for other devs, I've completely changed my mind.