7 ms·
Autodafe: Tools for freeing your project from the clammy grip of autotools
- db48x 3y agoHeh, wish I'd thought of that.
- zoky 3y agoIt’s what you oughtn’t to do, but you do anyway!
- szundi 3y agoFeature creep will do this one too
- buserror 3y agoI've been building distros for years, and the most wasteful part of it is running zillions of auto. process that waste most of the build time of each projects looking for mostly the same things. I always thought it was insane. Perhaps it was a good idea 30 years ago when we were actually building for dozens of 'unstable' variants of UNIX with a dozen compilers, but these days? And yes, MOST projects can be compiled just fine with a 2 pages Makefile, more often than not with -j for parallel build, and as a bonus, won't keep around a dozen turd files. Oh also, it is supposed to help 'portability' but MOST of my time is wasted trying to fix autotools configs when it invariably break in some new interesting and arcane ways.
- zokier 3y agoI doubt there are many new greenfield autotools using projects popping up around. So the question is not really if one should use autotools or not, but if projects should spend effort migrating away from working autotools setup which inevitably is also a breaking change for downstream consumers. And framing it like that it is suddenly much more difficult question, especially as that migration often might be quite non-trivial
- wewxjfq 3y agoDealing with a zillion of handwritten Makefiles sounds more hellish than dealing with autotools. These Makefiles will be more fickle, too. I'm not seeing a convenience win.
- vidarh 3y agoYou dealing with a zillion handwritten configuration files anyway, just hidden behind autotools in ways that most developers are even more clueless about than we are about makefiles.
- josephg 3y agoAre we talking about the same autotools? I'm getting flashbacks to running into some weird bash error in my configure file on line 3563. Where the configure script in question was generated by autoconf, automake, aclocal and all that jazz. Or autoconf mysteriously failing after automake works, or some similar nightmare. When autotools go wrong, trying to fix it feels like trying to breathe while you're held underwater. I'm sure part of the problem is I don't understand autotools as well as I could. But when "build expertise in autotools" is on the table, stabbing my eyes out with a fork starts to seem like an appealing option. Suffice it to say, I prefer to work with handwritten makefiles.
- fl7305 3y agoI'm not sure what the best solution is, but I agree on the pain dealing with autotools when they fail. Shotgun debugging where you start poking at things randomly can work for some stuff, but never works here. After working on many different software libraries and frameworks, my firm belief is that you really need to understand the lower layers to use and troubleshoot them effectively. Best case scenario there are excellent error messages and you can easily review logfiles to understand the root cause of the problem. But that is rarely the case. Instead you have to have a detailed mental model of the library/framework you're using, and you must be able to quickly picture what it will do internally for the inputs that you give it. Once you get to that point, many bugs don't appear in the first place because you immediately see that the input/usage doesn't make sense. And the bugs that do happen become much easier to figure out from the output and cryptic error messages you get. All this to say that it is really unappealing to work on things like bugs in someone else's autotool scripts. I just want to compile the program to run it. I don't want to spend months of my life to understand the inner workings of autotools.
- pjmlp 3y agoNowadays we have dozens of 'unstable' variants of Linux distributions with a two compilers, and dozens of additional scripting and managed languages. And a couple of BSD derived ones. As per Distrowatch statement from 2023. "There are over 600 Linux distros and about 500 in active development"
- cqqxo4zV46cp 3y agoAnd how many are based on Debian / Ubuntu? lol.
- pjmlp 3y agoDuring the UNIX wars heyday, except for special ones like Appolo (which used a Pascal dialect) or Tru64/QNX, they were also either based on AT&T UNIX, or the BSD spinoff. A few common bases hardly matter, when every distro is a special snowflake with its own set of incompatible changes.
- bregma 3y agoAnd dozens of non-Linux systems. AIX still lives, I still see HP-UX running, QNX is hidden away quietly running many things you take for granted every day, and there are a number of embedded executives make the world run. There a lot more to the world that writing scripts to steal people's bandwidth by pushing ads to web browsers.
- finaard 3y ago> Perhaps it was a good idea 30 years ago when we were actually building for dozens of 'unstable' variants of UNIX with a dozen compilers It wasn't. Back then what was typically getting in your way for porting a piece of software was autotools - and fixing it was typically significantly more complicated than adjusting a well written Makefile would've been. My hate for that autocrap mainly comes from that period.
- bregma 3y ago25 years ago autotools, and their predecessor the Cygnus tools, were a breath of fresh air. Porting stuff was a nightmare (imake, mkmk, many hand-rolled Makefiles that only supported the author's own system) and autotools made it easy, especially if you were running a non-homogeneous collection of Unix and Unix-like systems, including both libc5 and libc6 variants of Linux.
- finaard 3y agoI was living through that era with a zoo of Unix variants to build for (including, but not limited to, AIX, HP-UX, Solaris, IRIS, Tru64 and various Linux flavours) - and I always was excited about stuff that did just have hand rolled Makefiles, as that was something that was way easier to fix as the average software using autotools.
- kryptiskt 3y agoAutotools was always horrible, but it had a purpose back in the 90s, since then it's entirely vestigial and does nothing useful. It standardizes flags to config scripts and make targets, yes, but you can follow that standard without actually using autotools. It enables cross-compilation, yes, but the far biggest roadblock to successful cross-compilation is autotools, without that it's pretty easy. Meanwhile, these days, actually trying to build on a new platform is harder if the software is using autotools than if it's using plain makefiles. Because for all the noise about feature checking, nobody including the projects using autotools, are actually using the defines from autotools, they check the OS or architecture.
- dataflow 3y ago> using the defines from autotools Is there a list of them somewhere?
- bonzini 3y agoYou can decide what to test for, for example AC_CHECK_FUNCS([mlockall]) AC_CHECK_HEADERS([cpuid.h]) will define respectively HAVE_MLOCKALL and HAVE_CPUID_H.
- bregma 3y agoI currently make my living porting software to non-Linux and non-x86_64 cross-built systems, and I can vouch with certainty from lived experience that your assertions are entirely untrue.
- throw0101d 3y ago> […] looking for mostly the same things. In case folks don't know, caching exists: > A cache file is a shell script that caches the results of configure tests run on one system so they can be shared between configure scripts and configure runs. It is not useful on other systems. If its contents are invalid for some reason, the user may delete or edit it, or override documented cache variables on the configure command line. > By default, configure uses no cache file, to avoid problems caused by accidental use of stale cache files. > To enable caching, configure accepts --config-cache (or -C) to cache results in the file config.cache. Alternatively, --cache-file=file specifies that file be the cache file. The cache file is created if it does not exist already. When configure calls configure scripts in subdirectories, it uses the --cache-file argument so that they share the same cache. See Configuring Other Packages in Subdirectories, for information on configuring subdirectories with the AC_CONFIG_SUBDIRS macro. * https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.72/html_node/Cache-Files.html https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/a...
- fanf2 3y agoBut it doesn’t work reliably. If you change the configuration options, you have to clear the cache. The cache contents are not reliably shareable between multiple projects or different versions of autotools.
- rurban 3y agoHow naive! > every C99/POSIX system supports the entry points they are guarding The most popular compiler, msvc, still cannot support C99/POSIX. And some of my Makefile only projects still probes for features stored in a config.h Good luck with destroying our infrastructure.
- Too 3y agoFor a long time this was true but recently MS turned around and is now supposedly supporting even C17. https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-support-arriving-in-msvc/ https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-...
- camel-cdr 3y agoMeanwhile you still need to add a special compilation flag to get a c89 and above compatible preprocessor: https://learn.microsoft.com/en-us/cpp/build/reference/zc-preprocessor?view=msvc-170 https://learn.microsoft.com/en-us/cpp/build/reference/zc-pre...
- Kwpolska 3y agoDefault flags of compilers tend to prefer backwards compatibility over correctness and modernity. Passing flags to make the compiler behave correctly or report all errors is nothing new.
- camel-cdr 3y agoYes, but it shouldn't take +35 years to change the default to the correct behavior.
- jraph 3y agoDo one run ./configure on Windows to compile with msvc? (genuine question, not familiar with this)
- 3y ago
- CJefferson 3y agoI approve of the concept, but I don't think this isn't the answer. For a start, it says if you want to install into non-standard directories, or enable/disable options, you need to "edit your makefile". That isn't going to work for distributions, and I don't want to go to telling people to edit the makefile to change options on my program. Also, in my experience most autotools scripts do something, maybe dealing with inconsistent linker flags on mac vs linux, or enabling/disabling some debug option, or saying where to find some library (and optionally allowing building without said library). Coming up with some very simple format for configure scripts which are easy to read, and which do some simple search+replace substitution on a Makefile seems very sensible, and would get rid of awful autotools, is a great goal however.
- francescovv 3y ago> you need to "edit your makefile". That isn't going to work for distributions Is it not? [st] requires exactly that. And it works for distros, from what I can tell - debian/ubuntu, arch, almost everybody seem to ship it just fine. [st] https://st.suckless.org/ https://st.suckless.org/
- CJefferson 3y agoOK, maybe "not work well for distributions"? Looking at Debian, they are distributing a series of patches to the Makefile. Getting away from autotools by having every distro patch your makefile might still be better than autotools, but I'd hope we might be able to do even better?
- francescovv 3y ago> Debian (...) are distributing a series of patches to the Makefile. Where do you see that, sorry? I'm looking at the "Download Source Package" section here: https://packages.debian.org/sid/stterm https://packages.debian.org/sid/stterm ...and the only patch on there is debian/patches/0001-fix-buffer-overflow-when-handling-long-composed-inpu.patch, which doesn't touch Makefie.
- nanolith 3y agoI agree that autotools is often overkill, but I don't think the answer for most project is to force users to have to manually edit Makefiles. Feature detection is a good thing, but tools like CMake or Meson do a better job at this, in my opinion. I have done minimal GNU Make builds for large software projects before. In these cases, the main Makefile will depend on a file that is created after a one-time dependency check is done just to ensure that the build environment is sane. This file will exist until `make mrproper` is run, like any other build config file. I'm not personally a fan of monorepos, so instead, project dependencies that aren't already installed on the build machine/vm/container or that aren't a supported version will be automatically downloaded, configured, and installed in the build directory as part of the build process. Even with this sort of complexity, I adamantly believe that the average developer / user should just be able to type `make` and get either a useful error message or a working build. Likewise, I believe that the build scripts should be human readable, well, assuming that one has read through the GNU Make manual at least. These days, I have more or less standardized on one-shot CMake scripts for my personal projects. They will only check that the current build configuration has any dependencies already installed (e.g. via `PKG_CONFIG_PATH`), which allows for more intelligent scripting by downstream maintainers. The scripts themselves don't have any significant surprises and largely avoid patterns that can't be searched for using a search engine. This makes it easier for me to maintain cross-platform builds, at least across *nix. I haven't really tested Win10 / Win11 builds recently, but should.
- cryo 3y agoAfter a long period of rejecting CMake for "simple" build.sh and Makefile projects, nowadays CMake is my goto choice. Top features: - supports all kinds of compilers, even Watcom for DOS and Windows <all versions> projects. - cross platform compiliation inlc. MSVC - my #1 feature wide IDE support + usage from command line.
- viraptor 3y ago> but tools like CMake or Meson do a better job at this, in my opinion. Not much experience with Meson, but as a package maintainer, I really hate cmake. It's got the most of my "why isn't it calling it to pkg-config properly" debugging hours. Maybe it's easier to write, but finding what a specific option does through the forest of macros and pages of "this can do 10 different things" documentation and how it interacts with the rest of the system is not my favourite thing...
- deleted 3y ago[deleted]
- bhaak 3y agoStandards are good. Replacing standard autoconf with handedited Makefiles is going back to dark practices from the 20th century when this tool actually wants to "throw away the 20th century".
- jgrahamc 3y agoAutotools treats Makefiles like they are object code. We should promote Makefiles to being source code in a project and learn how to write them. The need for autotools died long, long ago when there were so many different platforms people were trying to support.
- iforgotpassword 3y agoBut but but "Checking whether compiler supports int... yes"
- astrobe_ 3y agoIt's probably still the case a little; I've seen 21st century projects shipping a dozen or so of makefiles because in Unix/Linux land there's so many little variations, like BSD vs Linux. I seems to me that Autotools is like Systemd : a relief for those who do that kind of task every day, and a bloated mess for others when you have to hack/fix it.
- jgrahamc 3y agoYeah, sure, but you can just 'import' a platform-specific Makefile and make the rest generic.
- iainmerrick 3y agoVery clever name, although I hope the tool isn’t as extreme as its namesake! See https://en.wikipedia.org/wiki/Auto-da-fé https://en.wikipedia.org/wiki/Auto-da-fé
- croisillon 3y agoNot sure if the name is so clever. For me autodafe is akin to pogrom or, at a bigger level, holocaust. Not sure i want to use tools called like that.
- iainmerrick 3y agoGood point. I knew the phrase at a surface level, but as you say, the real thing was seriously nasty.
- PhilipRoman 3y agoAnecdotal evidence but I work on a large number of C repositories and autotools is by far the most common cause of build related issues. Hand written makefiles are typically the easiest to debug. Note that this doesn't include makefile-based systems like Buildroot or Linux kernel makefiles. I don't consider them good examples due to the stateful config file management. My golden standard for building a project is the "empty makefile" only containing definitions for CFLAGS and LDFLAGS.
- teo_zero 3y agoThat's possible if you have one source for each executable. For slightly more complex projects don't you need at least an additional line? Something like App: main.o whatever.o ...
- PhilipRoman 3y agoOf course, reality isn't always simple. But the core idea is that Make has a sane model with reasonable defaults and most faults of makefiles are introduced by those who add unnecessary complexity and disrespect the functional nature of Make.
- deleted 3y ago[deleted]