9 ms·

I've developed negative feelings toward autotools over time, but I'm not sure how much I should be blaming autotools versus the projects that use autotools. On

I've developed negative feelings toward autotools over time, but I'm not sure how much I should be blaming autotools versus the projects that use autotools. One annoyance is that it often feels like "./configure ..." takes longer to run than the actual "make". But my main annoyance derives from that fact that most projects I build from source which use autotools don't build cleanly on anything but Linux x86/amd64. I spend a lot of time playing with non-Linux and/or non-x86 Unix systems, or using vendor compilers instead of gcc -- primarily Solaris (both x86 and SPARC), Linux on POWER, and AIX on POWER. Current versions of IBM's compilers, Sun (err...Oracle's) compilers, etc. Run the configure script, let it slowly probe all the little corners of whatever it's probing, and sometimes I'm lucky enough that it completes successfully and claims to understand the environment it's in. Then run make (or, as is often required, GNU make), and hit build errors very quickly. Which makes me think... why did I just sit through that long "configure" script run, and why did the developer bother messing with autotools, if this project isn't even going to bother targeting anything but Linux with the GNU toolchain to build it? If you're not concerned about portability, it seems a relatively simple Makefile would be an easier and faster approach. Why foist the [insert your favorite colorful descriptor here, depending on your personal feelings] autoconf system on builders of your project?

jjoonathan 6y ago
Yeah, the autotools model of portability didn't stand the test of time, but there's no shame in that. It's in good company. Isn't CMake the usual tool these days?
- rectang 6y ago
  Autotools, during its lifetime, has provide tremendous value across innumerable scenarios and systems. What does it matter if it didn't stand the "test of time" — especially if it shone a light for its successors to follow? The Autotools model of probing for system capabilities at build time isn't dead; it will never die, it will only be used in greater and lesser amounts and to complement other means of discovering system capabilities. CMake seems to me like a reimplementation of the Autotoools model of build-time probing. The true competitors are binary distribution after building on precursor systems with full specification and no probing, and cross-compilation with full specification and no probing.
  - jfb 6y ago
    Saying that something is obsolete isn't a value judgement -- it's an observation of the differences between the state of the world when it was necessary and now. This describes autotools to a 't'. It's not that there are that many fewer build targets these days, but rather that there are many many many fewer installations of most of those build targets these days.
- 1996 6y ago
  > Yeah, the autotools model of portability didn't stand the test of time, but there's no shame in that. xmkmf joined the chatroom
- rurban 6y ago
  Cmake is only useful in very limited scenarios. I do maintain lots of projects with both, and cmake is always the second citizen, for msvc users only. Serious users use autotools (with mingw), but the occasional windows noob cannot be helped. So he gets his limited cmake experience. Cmake usually only does a tenth of the autotools probes, edge cases will never be detected and worked around. Such as broken compiler versions, as with gcc or icc frequently. Cmake is a visual basic like hack, but Visual Basic at least had a proper design. Just lacked the library infrastructure. Everybody uses autotools, show me a cmake-based distro. All this autotools bashing is hilarious. Probing is costly yes, but you only do it once, and you rarely do it by yourself. Distro packagers do that for you.
  - jcranmer 6y ago
    > Everybody uses autotools, show me a cmake-based distro. The LLVM project exclusively uses cmake.
  - rleigh 6y ago
    This is a complete mischaracterisation of CMake usage. Today, it's better at portability than most Autotools builds I've used. In projects which maintain both systems side-by-side, I've had Autotools fail on Solaris, MinGW and Cygwin while with CMake it worked without trouble. And that's for projects which hadn't specifically tested and fixed defects for portability issues on these platforms. Empirically, CMake had better platform coverage. As for the probing of features, take an actual look at what CMake provides. Over an order of magnitude more feature tests. The problem with the Autoconf feature testing is that the vast majority of the tests are 15 years obsolete at this point. The CMake tests are mostly for contemporary platforms. That's where the value lies.
hzhou321 6y ago
> ... but I'm not sure how much I should be blaming autotools versus the projects that use autotools. Definitely the latter. Few projects treat `configure.ac` and its macros as serious as programming should be. They produce spaghetti code, duplicate code, patched code, slow code, unmaintainable code and they leave it there for decades only add code to make it worse, without ever feel the embarrassment to fix it. Of course blame the coder before blame the language.
- yjftsjthsd-h 6y ago
  I dunno, if a majority of projects using X come out bad, eventually it is a fault in X, not just bad users.
  - kelnos 6y ago
    Not really. As an example, the configure script will often output a "config.h" header that will have a bunch of preprocessor macros that the developer can use to make decisions about what platform features are supported. So you might have HAVE_FOO and HAVE_BAR in config.h. But if the developer doesn't do something like: #ifdef HAVE_FOO int ret = foo(); #elif HAVE_BAR int ret = bar(); #else int ret = do_some_fallback(); #endif ... then that's on the developer for just working on a system that has foo() and doesn't consider other systems that only have bar(), or have neither. Of course, that assumes they even put the tests for foo() and bar() in configure.ac in the first place! I would still use autotools for a C project even if I was only targeting linux/x86_64, just so I could write a simple declarative Makefile.am file that lists sources and defines target outputs, and have autoconf/automake figure out everything else, and put it behind a standard interface (that is, one that respects prefix, destdir, etc.). Also being able to easily write tests for dependencies and get the appropriate cflags and ldflags makes life easier.
    - pjmlp 6y ago
      Yes really, if everyone does it wrong, it means something is lacking to put those peoples in the right track from the start.
flukus 6y ago
> Which makes me think... why did I just sit through that long "configure" script run, and why did the developer bother messing with autotools, if this project isn't even going to bother targeting anything but Linux with the GNU toolchain to build it? There's more than just compiling, it adds a bunch of common stuff "out of the box" (with a line or two of configuration). Test runners, pkg-config, installation, cleaning, out of tree builds, etc. Nothing magic that you couldn't do yourself, but IME by the time you do it your makefile is much less simple, it's taken a non-trivial amount of time and possibly has a bunch of subtle errors. Even on my own little projects that will never run on another computer let alone another OS I find autotools worthwhile. If I had one major complaint it's that it still puts build artifacts in the root directory.
throwaway894345 6y ago
Lucky you. I rarely get autotools projects to compile cleanly even under idyllic circumstances.
zajio1am 6y ago
> Which makes me think... why did I just sit through that long "configure" script run, and why did the developer bother messing with autotools, if this project isn't even going to bother targeting anything but Linux with the GNU toolchain to build it? Even if target is just Linux with GNU toolchain, there are still plenty of issues that needs to be managed by configure script - tests for mandatory and optional libraries / header files, some functions are in different libraries on different systems (e.g. tinfo vs ncurses, or librt), different versions of kernel headers, different versions of GCC (older ones may not support some options).
- enriquto 6y ago
  > tests for mandatory and optional libraries I never understood the concept of "optional" library. Either your program requires a library or it doesn't. If it requires the library, the easiest thing to do is to assume that it is already installed and just link to it. In case it is not installed the compiler will fail with a very clear error message. Say, your program is an editor of png files and it requires libpng. Thus in your code you will have #include <libpng.h> and in your makefile you will link your executable with -lpng. And that is all! This works in linux, bsd and macOS. What the fucking else do you need?
  - throwaway9d0291 6y ago
    Sure, it can be simple. To give you a less clear example, think about writing a library that sends telemetry to some cloud service. The telemetry could be readings from a sensor attached to a microcontroller with 512K of RAM or it could be readings from a server sitting in a datacenter with 256GB of RAM. You need some helper library to handle the protocol and there's a really nice full-featured library that comes with a bunch of handy debugging tools but it's too memory-hungry for that tiny microcontroller. Another option is a minimalist library that uses very little memory but also has less flexibility in say TLS. If you pick just one, one of your platforms ends up suffering needlessly. Adding the options allows the user to decide what's best for them.
    - 6y ago
m463 6y ago
Which linux x86/amd64? Do you mean the linux that puts libraries in /usr/lib/x86_64-linux-gnu or the one that puts them in /usr/lib64 ?
- enriquto 6y ago
  who cares? wherever they are, the linker finds them when it sees the option -lpng
  - m463 6y ago
    I would assume autoconf is just checking for existence of packages, but yes I know other tools like cmake will actually try compiling/linking something to see if it's available.
lmm 6y ago
As someone who ran a fair bit of FreeBSD and Interix/SFU/SUA, autotools was much more reliable than any other build system (though CMake came close). 99% of autotools projects would build fine. Approximately 0% of "simple Makefile"s would build at all.
- m463 6y ago
  I really like CMake... except at some point you realize you changed one set of problems for another. the "modern cmake" movement is a step in the right direction It is nice that windows and linux folks can both work on a project (which I see as windows developers being able to contribute to linux software) The reality is that it's kind of messy when you actually try to develop something with it. (but not try-to-figure-out-m4 messy)
- AshamedCaptain 6y ago
  Likewise, except at some point I worked on cross compiling and embedded systems. When I had to build something using CMake, I would tremble. Autotools may be pure hell for the author side, but it is quite useful in the user side.
shp0ngle 6y ago
the `./configure` often tests edge-cases from the 90s and early 2000s, that are not relevant anymore, but everyone is afraid to touch any of that. ...because almost nobody can really write any code in M4sh (M4 with bash spliced together) and other odd languages. Really, the e-mail from the actual link is surprisingly sober and calls out the weaknesses of autotools well.
zvr 6y ago
> [...] I'm not sure how much I should be blaming autotools versus the projects that use autotools. I'd say that a large portion of the bad name that autotools currently have is due to their users. On one hand developers, who as you say do not actually use all the deductions of configure scripts,. But also users (the ones who blindly type ./configure) are to blame a little. If you complain that you wait while configure checks to find whether your machine is 32 or 64 bits, etc., please read on how to specify "site defaults" once and then all your configure runs take a fraction of the time.
db48x 6y ago
> One annoyance is that it often feels like "./configure ..." takes longer to run than the actual "make". Agreed. The configure script doesn’t take advantage of concurrency, but make can. But the number one cause of slow configure is that people forget to run configure -C. The -C option makes it cache the outcome of all of the tests, so that it doesn’t have to do them all again. This is a huge time–saver!
- deleted 6y ago
  [deleted]
- xzzze 6y ago
  Since ./configure just probes the compiler it could be paralellized. Make has concurrency (-j4) why doesn't ./configure have the same feature
  - zvr 6y ago
    Because typically make produces a number of independent targets (for example, object files of different source files) while configure only produces a single output file containing all variable assignments.
  - db48x 6y ago
    It’s also a shell script. It’s a shell script written purely in POSIX–compatible sh, that is generated by the worst M4 script ever written. The information which will be printed to stdout, to the log files, to the cache files, to the status file, and so on all needs to be buffered until the end of any asynchronous task, then written atomically so that it can be understood and will work consistently. That’s synchronizing writes to six or seven different file descriptors, just so that everything comes out in a comprehensible order. I bet you're already thinking about how to implement it though. And it does quite a lot more than just probe the compiler. And finally, it doesn’t support concurrent execution because nobody who was tried to write it has returned. Usually it starts with confident estimates and preparations, followed by departure and sporadic reports. Gradually the reports dwindle in frequency and comprehensibility, until eventually nothing more is heard.
devdiary 6y ago
Same for me but there's no better alternative to autotools that I can rely upon

jjoonathan 6y ago

Yeah, the autotools model of portability didn't stand the test of time, but there's no shame in that. It's in good company. Isn't CMake the usual tool these days?

rectang 6y ago
Autotools, during its lifetime, has provide tremendous value across innumerable scenarios and systems. What does it matter if it didn't stand the "test of time" — especially if it shone a light for its successors to follow? The Autotools model of probing for system capabilities at build time isn't dead; it will never die, it will only be used in greater and lesser amounts and to complement other means of discovering system capabilities. CMake seems to me like a reimplementation of the Autotoools model of build-time probing. The true competitors are binary distribution after building on precursor systems with full specification and no probing, and cross-compilation with full specification and no probing.
- jfb 6y ago
  Saying that something is obsolete isn't a value judgement -- it's an observation of the differences between the state of the world when it was necessary and now. This describes autotools to a 't'. It's not that there are that many fewer build targets these days, but rather that there are many many many fewer installations of most of those build targets these days.
1996 6y ago
> Yeah, the autotools model of portability didn't stand the test of time, but there's no shame in that. xmkmf joined the chatroom
rurban 6y ago
Cmake is only useful in very limited scenarios. I do maintain lots of projects with both, and cmake is always the second citizen, for msvc users only. Serious users use autotools (with mingw), but the occasional windows noob cannot be helped. So he gets his limited cmake experience. Cmake usually only does a tenth of the autotools probes, edge cases will never be detected and worked around. Such as broken compiler versions, as with gcc or icc frequently. Cmake is a visual basic like hack, but Visual Basic at least had a proper design. Just lacked the library infrastructure. Everybody uses autotools, show me a cmake-based distro. All this autotools bashing is hilarious. Probing is costly yes, but you only do it once, and you rarely do it by yourself. Distro packagers do that for you.
- jcranmer 6y ago
  > Everybody uses autotools, show me a cmake-based distro. The LLVM project exclusively uses cmake.
- rleigh 6y ago
  This is a complete mischaracterisation of CMake usage. Today, it's better at portability than most Autotools builds I've used. In projects which maintain both systems side-by-side, I've had Autotools fail on Solaris, MinGW and Cygwin while with CMake it worked without trouble. And that's for projects which hadn't specifically tested and fixed defects for portability issues on these platforms. Empirically, CMake had better platform coverage. As for the probing of features, take an actual look at what CMake provides. Over an order of magnitude more feature tests. The problem with the Autoconf feature testing is that the vast majority of the tests are 15 years obsolete at this point. The CMake tests are mostly for contemporary platforms. That's where the value lies.

hzhou321 6y ago

> ... but I'm not sure how much I should be blaming autotools versus the projects that use autotools. Definitely the latter. Few projects treat `configure.ac` and its macros as serious as programming should be. They produce spaghetti code, duplicate code, patched code, slow code, unmaintainable code and they leave it there for decades only add code to make it worse, without ever feel the embarrassment to fix it. Of course blame the coder before blame the language.

yjftsjthsd-h 6y ago
I dunno, if a majority of projects using X come out bad, eventually it is a fault in X, not just bad users.
- kelnos 6y ago
  Not really. As an example, the configure script will often output a "config.h" header that will have a bunch of preprocessor macros that the developer can use to make decisions about what platform features are supported. So you might have HAVE_FOO and HAVE_BAR in config.h. But if the developer doesn't do something like: #ifdef HAVE_FOO int ret = foo(); #elif HAVE_BAR int ret = bar(); #else int ret = do_some_fallback(); #endif ... then that's on the developer for just working on a system that has foo() and doesn't consider other systems that only have bar(), or have neither. Of course, that assumes they even put the tests for foo() and bar() in configure.ac in the first place! I would still use autotools for a C project even if I was only targeting linux/x86_64, just so I could write a simple declarative Makefile.am file that lists sources and defines target outputs, and have autoconf/automake figure out everything else, and put it behind a standard interface (that is, one that respects prefix, destdir, etc.). Also being able to easily write tests for dependencies and get the appropriate cflags and ldflags makes life easier.
  - pjmlp 6y ago
    Yes really, if everyone does it wrong, it means something is lacking to put those peoples in the right track from the start.

flukus 6y ago

> Which makes me think... why did I just sit through that long "configure" script run, and why did the developer bother messing with autotools, if this project isn't even going to bother targeting anything but Linux with the GNU toolchain to build it? There's more than just compiling, it adds a bunch of common stuff "out of the box" (with a line or two of configuration). Test runners, pkg-config, installation, cleaning, out of tree builds, etc. Nothing magic that you couldn't do yourself, but IME by the time you do it your makefile is much less simple, it's taken a non-trivial amount of time and possibly has a bunch of subtle errors. Even on my own little projects that will never run on another computer let alone another OS I find autotools worthwhile. If I had one major complaint it's that it still puts build artifacts in the root directory.

throwaway894345 6y ago

Lucky you. I rarely get autotools projects to compile cleanly even under idyllic circumstances.

zajio1am 6y ago

> Which makes me think... why did I just sit through that long "configure" script run, and why did the developer bother messing with autotools, if this project isn't even going to bother targeting anything but Linux with the GNU toolchain to build it? Even if target is just Linux with GNU toolchain, there are still plenty of issues that needs to be managed by configure script - tests for mandatory and optional libraries / header files, some functions are in different libraries on different systems (e.g. tinfo vs ncurses, or librt), different versions of kernel headers, different versions of GCC (older ones may not support some options).

enriquto 6y ago
> tests for mandatory and optional libraries I never understood the concept of "optional" library. Either your program requires a library or it doesn't. If it requires the library, the easiest thing to do is to assume that it is already installed and just link to it. In case it is not installed the compiler will fail with a very clear error message. Say, your program is an editor of png files and it requires libpng. Thus in your code you will have #include <libpng.h> and in your makefile you will link your executable with -lpng. And that is all! This works in linux, bsd and macOS. What the fucking else do you need?
- throwaway9d0291 6y ago
  Sure, it can be simple. To give you a less clear example, think about writing a library that sends telemetry to some cloud service. The telemetry could be readings from a sensor attached to a microcontroller with 512K of RAM or it could be readings from a server sitting in a datacenter with 256GB of RAM. You need some helper library to handle the protocol and there's a really nice full-featured library that comes with a bunch of handy debugging tools but it's too memory-hungry for that tiny microcontroller. Another option is a minimalist library that uses very little memory but also has less flexibility in say TLS. If you pick just one, one of your platforms ends up suffering needlessly. Adding the options allows the user to decide what's best for them.
  - 6y ago

m463 6y ago

Which linux x86/amd64? Do you mean the linux that puts libraries in /usr/lib/x86_64-linux-gnu or the one that puts them in /usr/lib64 ?

enriquto 6y ago
who cares? wherever they are, the linker finds them when it sees the option -lpng
- m463 6y ago
  I would assume autoconf is just checking for existence of packages, but yes I know other tools like cmake will actually try compiling/linking something to see if it's available.

lmm 6y ago

As someone who ran a fair bit of FreeBSD and Interix/SFU/SUA, autotools was much more reliable than any other build system (though CMake came close). 99% of autotools projects would build fine. Approximately 0% of "simple Makefile"s would build at all.

m463 6y ago
I really like CMake... except at some point you realize you changed one set of problems for another. the "modern cmake" movement is a step in the right direction It is nice that windows and linux folks can both work on a project (which I see as windows developers being able to contribute to linux software) The reality is that it's kind of messy when you actually try to develop something with it. (but not try-to-figure-out-m4 messy)
AshamedCaptain 6y ago
Likewise, except at some point I worked on cross compiling and embedded systems. When I had to build something using CMake, I would tremble. Autotools may be pure hell for the author side, but it is quite useful in the user side.

shp0ngle 6y ago

the `./configure` often tests edge-cases from the 90s and early 2000s, that are not relevant anymore, but everyone is afraid to touch any of that. ...because almost nobody can really write any code in M4sh (M4 with bash spliced together) and other odd languages. Really, the e-mail from the actual link is surprisingly sober and calls out the weaknesses of autotools well.

zvr 6y ago

> [...] I'm not sure how much I should be blaming autotools versus the projects that use autotools. I'd say that a large portion of the bad name that autotools currently have is due to their users. On one hand developers, who as you say do not actually use all the deductions of configure scripts,. But also users (the ones who blindly type ./configure) are to blame a little. If you complain that you wait while configure checks to find whether your machine is 32 or 64 bits, etc., please read on how to specify "site defaults" once and then all your configure runs take a fraction of the time.

db48x 6y ago

> One annoyance is that it often feels like "./configure ..." takes longer to run than the actual "make". Agreed. The configure script doesn’t take advantage of concurrency, but make can. But the number one cause of slow configure is that people forget to run configure -C. The -C option makes it cache the outcome of all of the tests, so that it doesn’t have to do them all again. This is a huge time–saver!

deleted 6y ago
[deleted]
xzzze 6y ago
Since ./configure just probes the compiler it could be paralellized. Make has concurrency (-j4) why doesn't ./configure have the same feature
- zvr 6y ago
  Because typically make produces a number of independent targets (for example, object files of different source files) while configure only produces a single output file containing all variable assignments.
- db48x 6y ago
  It’s also a shell script. It’s a shell script written purely in POSIX–compatible sh, that is generated by the worst M4 script ever written. The information which will be printed to stdout, to the log files, to the cache files, to the status file, and so on all needs to be buffered until the end of any asynchronous task, then written atomically so that it can be understood and will work consistently. That’s synchronizing writes to six or seven different file descriptors, just so that everything comes out in a comprehensible order. I bet you're already thinking about how to implement it though. And it does quite a lot more than just probe the compiler. And finally, it doesn’t support concurrent execution because nobody who was tried to write it has returned. Usually it starts with confident estimates and preparations, followed by departure and sporadic reports. Gradually the reports dwindle in frequency and comprehensibility, until eventually nothing more is heard.

devdiary 6y ago

Same for me but there's no better alternative to autotools that I can rely upon