4 ms·

I'm not sure I understand. What I do in my project is very common practice: generated files (like the configure script) are not part of the repository, but the

I'm not sure I understand. What I do in my project is very common practice: generated files (like the configure script) are not part of the repository, but they are put in released tarballs

sspiff 1y ago
This is true and standard for (really) old projects, and dealing with this scripts and their problems used to be the bane of my existence 10 years ago. But I can't say I've encountered any such projects in the last 5 or so years. Either they use a modern programming language (which typically has an included build system, like rust's cargo or simply go build) of they use simple Makefiles. For C/C++ codebases, it seems like CMake has become the dominant build system. All of these are typically better than what GNU autoconf offers, with modern modern features and equally or better flexibility to deal with differences between operating systems, distributions, and/or optional or alternative libraries. I don't really see why anyone would pick autoconf for a modern project.
- varjag 1y ago
  CMake is really more of a C++ crowd thing, it never won the mindshare with C. > I don't really see why anyone would pick autoconf for a modern project. If you build for your system only and never ever plan to cross compile by all means go with static makefile.
  - fc417fc802 1y ago
    A few of my personal projects cross compile via static makefile. Is there something wrong with that?
    - deleted 1y ago
      [deleted]
    - kazinator 1y ago
      If you're not writing something which compiles its own tools which are then used for the rest of the build, or is not a compiled programming language, all you have to do is respect the CC, CFLAGS, LDFLAGS and LDLIBS variables coming from the distro. Then you will use the correct cross toolchain and libs. If you need to compile programs that run on the build machine, you should have a ./configure script which allows a host CC and target CC to be specified, and use them accordingly. Even if you deviate a bit from what others are doing, if it is clearly documented and working, the downstream package maintainer can handle it.
  - kazinator 1y ago
    A good way to make sure your project won't cross compile is to use Autoconf. Rampant use of Autoconf is the main reason distros gave up on cross compiling and started using QEMU. Developers who use Autoconf and who don't know what cross-compiling is will not end up with a cleanly cross-compiling project that downstream packagers don't have to patch into submission. Most of my disdain for Autoconf was formed when I worked at a company where I developed a embedded Linux distro from scratch. I cross-compiled everything. Most of the crap I had to fight with was Autoconf projects. I was having to do things like export various ac_cv_... internal variables that nobody should know about, and patching configure scripts themselves. Fast forward a few years and I see a QEMU everywhere for "cross" builds. The rest of my disdain comes from having worked with the internals of various GNU programs. To bootstrap their build systems from a repository checkout (not a release tarball) you have to follow their specific instructions. Of course you must have the Autotools installed. But there are multiple versions, and they generate different code. For each program you have to have the right version that it wants. If you have to do a git bisect, older commits may need an older version of the Autotools. Bootstrap from the configure system from scratch, the result of which is the privilege to now run configure from scratch. It's simply insane. You learn things like to touch certain files in a certain order to prevent a reconfigure that has about a 50% chance of working. Let's not even going to libtool. The main idea behind Autoconf is political. Autoconf based programs are deliberately intended to hinder those who are able to build a program on a non-GNU system and then want to make contributions while just staying on that system, not getting a whole GNU environment. What I want is something different. I want a user to be able to use any platform where the program works to obtain a checkout if exactly what is in git, and be able to make a patch to the configuration stuff, test it and send upstream without installing anything that is not required for just building the program for use.
    - pabs3 1y ago
      Autoconf and automake has the best support for cross-compiling there is, everything else is a poor imitation. At least from the perspective of the folks doing Debian's cross-build stuff. With Debian's multi-arch policy, cross-toolchain packages and dpkg-dev/debhelper support for driving common cross-compiling options, plus fixing a ton of edge cases, IIRC more than 50% of Debian packages are now cross-compilable without qemu. Often they are bit-for-bit identical to the native compilation too. https://wiki.debian.org/CrossCompiling https://wiki.debian.org/CrossCompiling https://crossqa.debian.net/ https://crossqa.debian.net/
- JonChesterfield 1y ago
  Cmake is by a wide margin the worst build tool I've used. That covers at least autoconf, gmake, nmake, scons, waf, tup, visual studio, the boost thing, bash scripts and lua scripts. Even the hand edited xml insanity of visual studio caused negligible grief compared to cmake.
  - kazinator 1y ago
    I strongly concur. Cmake is incompetently designed and implemented. The authors had no idea how to make a build language, but didn't let it stop them.
- shawn_w 1y ago
  Having used both autoconf and cmake, I have a strong preference for autoconf (plus hand written makefiles; never been able to get into automake). It's just easier to use for me, especially when it comes to writing tests for supported functions and adding optional features you want to enable or disable via configure script options.
  - andreamonaco 1y ago
    In my opinion, automake is the weakest part of the autotools chain. Look at this section of the manual for example https://www.gnu.org/software/automake/manual/automake.html#General-Operation https://www.gnu.org/software/automake/manual/automake.html#G...: it says that automake doesn't recognize many Gnu Make extension, and can get confused even by weird whitespace...
    - deleted 1y ago
      [deleted]
    - deleted 1y ago
      [deleted]
kazinator 1y ago
It's a bad practice commonly found in GNU projects, which results in an overcomplicated, inconvenient and unstable build system that will discourage future collaboration. Many of these projects are very old, two decades or more; they are living with ancient decisions made in a world of dozens of incompatible Unix forks. One thing to do instead is to just write a ./configure script which detects what you need. In other words, be compatible at the invocation level. Make sure this is checked into the repo Anyone checking out any commit runs that, and that's it. Someone who makes a tarball using git, out of a tagged release commit, should have a "release tarball". A recent HN submission shows a ./configure system made from scratch using makefiles, which parallelizes the tests. That could be a good starting point for a C on Linux project today.
- amszmidt 1y ago
  > A recent HN submission shows a ./configure system made from scratch using makefiles, which parallelizes the tests. That could be a good starting point for a C on Linux project today. Not everything is C, or GNU/Linux. The example also misses much of the basic functionality that makes GNU autotools amazing. The major benefit of GNU autotools is that it works well, specially for new platforms and cross compilation. If all you care about is your own system, a simple Makefile will do just fine. And with GNU autotools you can also pick to just use GNU autoconf .. or just GNU automake. Having generated files in the release tarball is a good practise, why should users have to install a bunch of extra tools just to get PDF of the manual or other non-system specific files? It is not just build scripts all over the place, installing TeX Live just to get a PDF manual of something is super annoying. Writing your own ./configure that works remotely as something users would expect is non-trivial, and complicated -- we did that 30 years ago before GNU autoconf. There is a reason why we stopped doing that ... I'd go so far to think that GNU autotools is the most sensible build system out there...
  - kazinator 1y ago
    > Not everything is C, AutoTools are squarely oriented toward C, though. If you're not using C or C++, you're probably not using AutoTools. (I think I might have seen someone's purely Python or shell script project using Autoconf, but that was just ridiculously unnecessary.) > Having generated files in the release tarball is a good practise Without a doubt, it is a good idea to ship certain generated files in a source code distribution. For instance, if we ship a generated y.tab.c file, the user doesn't have to have a Yacc program (let alone the exact version of the one we would like them to use). What's not good practice is that anything is different in the release tarball compared to the git commit it was made from. "Release tarball" itself a configuration management antipattern. We are a good two decades past tarballs now. A tarball should only be a convenience for people who don't need a git clone. Every generated thing that a downstream user might need should be in version control, and updated whenever its prerequisites change. This is a special exception to the general rule that only primary objects should be in version control. Secondary objects for which downstream users don't have generation tools should also be in version control.

I'm not sure I understand. What I do in my project is very common practice: generated files (like the configure script) are not part of the repository, but they are put in released tarballs

sspiff 1y ago

This is true and standard for (really) old projects, and dealing with this scripts and their problems used to be the bane of my existence 10 years ago. But I can't say I've encountered any such projects in the last 5 or so years. Either they use a modern programming language (which typically has an included build system, like rust's cargo or simply go build) of they use simple Makefiles. For C/C++ codebases, it seems like CMake has become the dominant build system. All of these are typically better than what GNU autoconf offers, with modern modern features and equally or better flexibility to deal with differences between operating systems, distributions, and/or optional or alternative libraries. I don't really see why anyone would pick autoconf for a modern project.

varjag 1y ago
CMake is really more of a C++ crowd thing, it never won the mindshare with C. > I don't really see why anyone would pick autoconf for a modern project. If you build for your system only and never ever plan to cross compile by all means go with static makefile.
- fc417fc802 1y ago
  A few of my personal projects cross compile via static makefile. Is there something wrong with that?
  - deleted 1y ago
    [deleted]
  - kazinator 1y ago
    If you're not writing something which compiles its own tools which are then used for the rest of the build, or is not a compiled programming language, all you have to do is respect the CC, CFLAGS, LDFLAGS and LDLIBS variables coming from the distro. Then you will use the correct cross toolchain and libs. If you need to compile programs that run on the build machine, you should have a ./configure script which allows a host CC and target CC to be specified, and use them accordingly. Even if you deviate a bit from what others are doing, if it is clearly documented and working, the downstream package maintainer can handle it.
- kazinator 1y ago
  A good way to make sure your project won't cross compile is to use Autoconf. Rampant use of Autoconf is the main reason distros gave up on cross compiling and started using QEMU. Developers who use Autoconf and who don't know what cross-compiling is will not end up with a cleanly cross-compiling project that downstream packagers don't have to patch into submission. Most of my disdain for Autoconf was formed when I worked at a company where I developed a embedded Linux distro from scratch. I cross-compiled everything. Most of the crap I had to fight with was Autoconf projects. I was having to do things like export various ac_cv_... internal variables that nobody should know about, and patching configure scripts themselves. Fast forward a few years and I see a QEMU everywhere for "cross" builds. The rest of my disdain comes from having worked with the internals of various GNU programs. To bootstrap their build systems from a repository checkout (not a release tarball) you have to follow their specific instructions. Of course you must have the Autotools installed. But there are multiple versions, and they generate different code. For each program you have to have the right version that it wants. If you have to do a git bisect, older commits may need an older version of the Autotools. Bootstrap from the configure system from scratch, the result of which is the privilege to now run configure from scratch. It's simply insane. You learn things like to touch certain files in a certain order to prevent a reconfigure that has about a 50% chance of working. Let's not even going to libtool. The main idea behind Autoconf is political. Autoconf based programs are deliberately intended to hinder those who are able to build a program on a non-GNU system and then want to make contributions while just staying on that system, not getting a whole GNU environment. What I want is something different. I want a user to be able to use any platform where the program works to obtain a checkout if exactly what is in git, and be able to make a patch to the configuration stuff, test it and send upstream without installing anything that is not required for just building the program for use.
  - pabs3 1y ago
    Autoconf and automake has the best support for cross-compiling there is, everything else is a poor imitation. At least from the perspective of the folks doing Debian's cross-build stuff. With Debian's multi-arch policy, cross-toolchain packages and dpkg-dev/debhelper support for driving common cross-compiling options, plus fixing a ton of edge cases, IIRC more than 50% of Debian packages are now cross-compilable without qemu. Often they are bit-for-bit identical to the native compilation too. https://wiki.debian.org/CrossCompiling https://wiki.debian.org/CrossCompiling https://crossqa.debian.net/ https://crossqa.debian.net/
JonChesterfield 1y ago
Cmake is by a wide margin the worst build tool I've used. That covers at least autoconf, gmake, nmake, scons, waf, tup, visual studio, the boost thing, bash scripts and lua scripts. Even the hand edited xml insanity of visual studio caused negligible grief compared to cmake.
- kazinator 1y ago
  I strongly concur. Cmake is incompetently designed and implemented. The authors had no idea how to make a build language, but didn't let it stop them.
shawn_w 1y ago
Having used both autoconf and cmake, I have a strong preference for autoconf (plus hand written makefiles; never been able to get into automake). It's just easier to use for me, especially when it comes to writing tests for supported functions and adding optional features you want to enable or disable via configure script options.
- andreamonaco 1y ago
  In my opinion, automake is the weakest part of the autotools chain. Look at this section of the manual for example https://www.gnu.org/software/automake/manual/automake.html#General-Operation https://www.gnu.org/software/automake/manual/automake.html#G...: it says that automake doesn't recognize many Gnu Make extension, and can get confused even by weird whitespace...
  - deleted 1y ago
    [deleted]
  - deleted 1y ago
    [deleted]

kazinator 1y ago

It's a bad practice commonly found in GNU projects, which results in an overcomplicated, inconvenient and unstable build system that will discourage future collaboration. Many of these projects are very old, two decades or more; they are living with ancient decisions made in a world of dozens of incompatible Unix forks. One thing to do instead is to just write a ./configure script which detects what you need. In other words, be compatible at the invocation level. Make sure this is checked into the repo Anyone checking out any commit runs that, and that's it. Someone who makes a tarball using git, out of a tagged release commit, should have a "release tarball". A recent HN submission shows a ./configure system made from scratch using makefiles, which parallelizes the tests. That could be a good starting point for a C on Linux project today.

amszmidt 1y ago
> A recent HN submission shows a ./configure system made from scratch using makefiles, which parallelizes the tests. That could be a good starting point for a C on Linux project today. Not everything is C, or GNU/Linux. The example also misses much of the basic functionality that makes GNU autotools amazing. The major benefit of GNU autotools is that it works well, specially for new platforms and cross compilation. If all you care about is your own system, a simple Makefile will do just fine. And with GNU autotools you can also pick to just use GNU autoconf .. or just GNU automake. Having generated files in the release tarball is a good practise, why should users have to install a bunch of extra tools just to get PDF of the manual or other non-system specific files? It is not just build scripts all over the place, installing TeX Live just to get a PDF manual of something is super annoying. Writing your own ./configure that works remotely as something users would expect is non-trivial, and complicated -- we did that 30 years ago before GNU autoconf. There is a reason why we stopped doing that ... I'd go so far to think that GNU autotools is the most sensible build system out there...
- kazinator 1y ago
  > Not everything is C, AutoTools are squarely oriented toward C, though. If you're not using C or C++, you're probably not using AutoTools. (I think I might have seen someone's purely Python or shell script project using Autoconf, but that was just ridiculously unnecessary.) > Having generated files in the release tarball is a good practise Without a doubt, it is a good idea to ship certain generated files in a source code distribution. For instance, if we ship a generated y.tab.c file, the user doesn't have to have a Yacc program (let alone the exact version of the one we would like them to use). What's not good practice is that anything is different in the release tarball compared to the git commit it was made from. "Release tarball" itself a configuration management antipattern. We are a good two decades past tarballs now. A tarball should only be a convenience for people who don't need a git clone. Every generated thing that a downstream user might need should be in version control, and updated whenever its prerequisites change. This is a special exception to the general rule that only primary objects should be in version control. Secondary objects for which downstream users don't have generation tools should also be in version control.