3 ms·

We don't need a new build system, we need a new mentality. The problem with build systems is they don't cover the entire scope required to actually build softw

by sparkie 13y ago

We don't need a new build system, we need a new mentality. The problem with build systems is they don't cover the entire scope required to actually build software reliably. Ideally, you want to take just your code as input and produce a target, but the reality is that you're taking your code, plus an environment with potentially infinite number of configuration options, and you're asking the build system to produce a target. This is effectively saying "Here's some code and some shit, please do your wizardry and make me something that looks like this." Is it any wonder that every proposed solution to this difficult (impossible?) problem gets it wrong? The correct solution is to declare the target you want, then build up the dependencies you need as input to meet that target and declare them explicitly, then perform the build in that isolated environment to ensure that no undeclared configuration options can alter the result of the build. This is what Nix (http://nixos.org/nix/ http://nixos.org/nix/) does, and by derivation, Guix (https://www.gnu.org/software/guix/ https://www.gnu.org/software/guix/), and Debian's ReproducibleBuilds is also attempting. The build process becomes effectively a pure function, free of unwanted side-effects. Nix doesn't actually perform builds in itself, except for trivial ./configure && make. It piggy-backs on existing build systems via bash scripts if needed, but it completely manages the environment in which such script will run, so it has local-side effects which are controlled, similar to how you might use the ST monad in place of IO in Haskell perhaps. I find Guix a bit more interesting because it can take on the whole problem - including the build system. While it can piggy-back on existing build systems in the same way Nix can, you can also write your own in Guile (or some other language and invoke from Guile). It doesn't really make sense to depend on the old cruft under this new mentality, because things like pkg-config are obsoleted by known, explicitly declared dependencies. The build process can be simplified (and probably sped-up).

_delirium 13y ago
How portable is nix? I haven't looked into it much, but from a (possibly mistaken) first impression it looks more like a replacement for yum or apt than for autotools. Despite being crufty, what I find valuable about autotools is that, given one source directory, I can successfully compile and link the package on many things: various distributions of Linux, the BSDs, Solaris, HP-UX, AIX, OS X, nearly whatever else you care to dream up. Does Nix actually do that (or target it, if not yet)?
- sparkie 13y ago
  Nix is indeed a replacement for package managers, but the point still stands. Package management and software builds are not completely disjoint - the latter is depends quite heavily on the former. Nix itself should work on BSD (I think it's tested on FreeBSD), but Guix is linux-only so far, as it is still in early development. I'm pretty sure Guix intends to be fully portable, even to Hurd in the long term. The ideas behind Nix is platform independent though - there shouldn't be any reason it can't be ported to other platforms. As for building individual software on other platforms - they require different targets. (Or in fact, just an alternative configuration of the same piece of software on the same platform requires a different target). This is intentional though - targets have an identity (a SHA1 of their entire inputs), so any new platform will require a new package. The package configuration format is quite useful in that regard though, as you can derive new package definitions from existing ones, so it should be possible to make partial "generic" configurations, then derive the platform specific targets from them. There's quite a bit of configuration for each platform though, as the dependencies will all have different identities, all the way down to the compiler and the kernel. Another potential option would be to create a template system to automatically generate the Nix configuration for multiple platforms, which would be somewhat similar to what autotools et al are doing today. The difference being, once one person has performed a build and tested it for a platform, anyone else should be able to reproduce it exactly, or to explicitly specify where he wants the target to differ. In effect, Nix removes the "hidden knowledge" that goes into building software by requiring that you specify even the commands you used to perform the build. Of course, Nix isn't a panacea - it introduces a new set of problems, including some social ones. It might take more effort to write portable targets for each platform - but for that trade off you get reliability and simplicity for the users of the software.

by sparkie 13y ago

_delirium 13y ago

How portable is nix? I haven't looked into it much, but from a (possibly mistaken) first impression it looks more like a replacement for yum or apt than for autotools. Despite being crufty, what I find valuable about autotools is that, given one source directory, I can successfully compile and link the package on many things: various distributions of Linux, the BSDs, Solaris, HP-UX, AIX, OS X, nearly whatever else you care to dream up. Does Nix actually do that (or target it, if not yet)?

sparkie 13y ago
Nix is indeed a replacement for package managers, but the point still stands. Package management and software builds are not completely disjoint - the latter is depends quite heavily on the former. Nix itself should work on BSD (I think it's tested on FreeBSD), but Guix is linux-only so far, as it is still in early development. I'm pretty sure Guix intends to be fully portable, even to Hurd in the long term. The ideas behind Nix is platform independent though - there shouldn't be any reason it can't be ported to other platforms. As for building individual software on other platforms - they require different targets. (Or in fact, just an alternative configuration of the same piece of software on the same platform requires a different target). This is intentional though - targets have an identity (a SHA1 of their entire inputs), so any new platform will require a new package. The package configuration format is quite useful in that regard though, as you can derive new package definitions from existing ones, so it should be possible to make partial "generic" configurations, then derive the platform specific targets from them. There's quite a bit of configuration for each platform though, as the dependencies will all have different identities, all the way down to the compiler and the kernel. Another potential option would be to create a template system to automatically generate the Nix configuration for multiple platforms, which would be somewhat similar to what autotools et al are doing today. The difference being, once one person has performed a build and tested it for a platform, anyone else should be able to reproduce it exactly, or to explicitly specify where he wants the target to differ. In effect, Nix removes the "hidden knowledge" that goes into building software by requiring that you specify even the commands you used to perform the build. Of course, Nix isn't a panacea - it introduces a new set of problems, including some social ones. It might take more effort to write portable targets for each platform - but for that trade off you get reliability and simplicity for the users of the software.