28 ms·

My main problem with guix, and nix, and debs (the 3 i have tried is they are great for software wrapped using automake/autoconf, but I can never figure out how

My main problem with guix, and nix, and debs (the 3 i have tried is they are great for software wrapped using automake/autoconf, but I can never figure out how to package things using a custom build system.

cstrahan 8y ago
It's certainly doable. I contributed Envoy to Nixpkgs (built with bazel) and developed the current incarnation of the RubyGems/Bundler build infrastructure.
chriswarbo 8y ago
Guix and Nix just run scripts. I've less experience with Guix, but in Nix you can just run a bash script which stores the build products in a path given as `$out`. For example, the following is a perfectly valid Nix package, if we replaced the nonsense names and commands: with import <nixpkgs> {}; runCommand "package-name-goes-here" { buildInputs = [ build tools go here ]; envVar1 = "hello"; envVar2 = "world"; } '' bash command 1 bash command 2 mv my-amazing-output "$out" '' For example, I've never written any Go, but I was able to package some Go programs by copy-pasting their install instructions into the above template and putting the relevant build tools into `buildInputs`: https://github.com/Warbo/warbo-packages/blob/master/packages/goat.nix https://github.com/Warbo/warbo-packages/blob/master/packages... In contrast, the automake/autoconf support is actually a very complicated tower of helper scripts, override hooks, etc. which is built up on top of the system's built in support for running bash scripts (actually it runs whatever we call `builder` with whatever we call `args`; `runCommand` sets the `builder` to be bash and the `args` to be a file containing our script).
- rekado 8y ago
  While Guix eventually runs a builder script, the script is 1) a Guile script and 2) not directly provided by the user but the result of compilation. Guix does not stitch shell scripts together. Builds in Guix are split into build phases, which are Guile procedures. Build system abstractions are collections of these procedures. A packager can inject additional build phases or delete existing build phases from these build systems on a package by package basis. The `trivial-build-system` is used for packages where none of the existing build systems are applicable, not even with build phases removed or added. This works but is not used very often as it is often more convenient to start with something like the `gnu-build-system` and to replace phases by providing new procedures; this is done to benefit from build phases that the `gnu-build-system` provides, such as those that unpack the sources, patch shebangs, or validate the binaries.
  - chriswarbo 8y ago
    Thanks for the Guix info, as I say I've not really used it. The use of "phases", etc. is what I was getting at w.r.t. Nix's complicated "autoconf/automake support" (AKA `stdenv.mkDerivation`). As you say, Guix's use of Scheme rather than stiching shell scripts together should be more powerful (in particular, concatenating strings of bash results in a rather opaque builder; whilst a more declarative s-expression approach would make traversing and modifying the builder more feasible)
tathougies 8y ago
In your derivation, do builder = ./builder.sh and then write a shell script named builder.sh that builds what you want. This is documented at https://nixos.org/nix/manual/#chap-writing-nix-expressions https://nixos.org/nix/manual/#chap-writing-nix-expressions
rekado 8y ago
Guix provides build system abstractions like `gnu-build-system` for traditional build systems, but you can easily override or extend the build phases. You can run scripts with `(invoke "my-script.sh")`. The bigger problem, in my opinion, is that packages that use custom build systems often have strange ideas about what is desirable in a build system. Sometimes they prefer to download things during the build (that's not permitted in Guix) or they will make assumptions about the layout of directories or the discovery of dependencies. Patching away all of these invalid assumptions can be annoying, but it is possible as is demonstrated in many package definitions in the `(gnu packages bioinformatics)` module.

cstrahan 8y ago

It's certainly doable. I contributed Envoy to Nixpkgs (built with bazel) and developed the current incarnation of the RubyGems/Bundler build infrastructure.

chriswarbo 8y ago

Guix and Nix just run scripts. I've less experience with Guix, but in Nix you can just run a bash script which stores the build products in a path given as `$out`. For example, the following is a perfectly valid Nix package, if we replaced the nonsense names and commands: with import <nixpkgs> {}; runCommand "package-name-goes-here" { buildInputs = [ build tools go here ]; envVar1 = "hello"; envVar2 = "world"; } '' bash command 1 bash command 2 mv my-amazing-output "$out" '' For example, I've never written any Go, but I was able to package some Go programs by copy-pasting their install instructions into the above template and putting the relevant build tools into `buildInputs`: https://github.com/Warbo/warbo-packages/blob/master/packages/goat.nix https://github.com/Warbo/warbo-packages/blob/master/packages... In contrast, the automake/autoconf support is actually a very complicated tower of helper scripts, override hooks, etc. which is built up on top of the system's built in support for running bash scripts (actually it runs whatever we call `builder` with whatever we call `args`; `runCommand` sets the `builder` to be bash and the `args` to be a file containing our script).

rekado 8y ago
While Guix eventually runs a builder script, the script is 1) a Guile script and 2) not directly provided by the user but the result of compilation. Guix does not stitch shell scripts together. Builds in Guix are split into build phases, which are Guile procedures. Build system abstractions are collections of these procedures. A packager can inject additional build phases or delete existing build phases from these build systems on a package by package basis. The `trivial-build-system` is used for packages where none of the existing build systems are applicable, not even with build phases removed or added. This works but is not used very often as it is often more convenient to start with something like the `gnu-build-system` and to replace phases by providing new procedures; this is done to benefit from build phases that the `gnu-build-system` provides, such as those that unpack the sources, patch shebangs, or validate the binaries.
- chriswarbo 8y ago
  Thanks for the Guix info, as I say I've not really used it. The use of "phases", etc. is what I was getting at w.r.t. Nix's complicated "autoconf/automake support" (AKA `stdenv.mkDerivation`). As you say, Guix's use of Scheme rather than stiching shell scripts together should be more powerful (in particular, concatenating strings of bash results in a rather opaque builder; whilst a more declarative s-expression approach would make traversing and modifying the builder more feasible)

tathougies 8y ago

In your derivation, do builder = ./builder.sh and then write a shell script named builder.sh that builds what you want. This is documented at https://nixos.org/nix/manual/#chap-writing-nix-expressions https://nixos.org/nix/manual/#chap-writing-nix-expressions

rekado 8y ago

Guix provides build system abstractions like `gnu-build-system` for traditional build systems, but you can easily override or extend the build phases. You can run scripts with `(invoke "my-script.sh")`. The bigger problem, in my opinion, is that packages that use custom build systems often have strange ideas about what is desirable in a build system. Sometimes they prefer to download things during the build (that's not permitted in Guix) or they will make assumptions about the layout of directories or the discovery of dependencies. Patching away all of these invalid assumptions can be annoying, but it is possible as is demonstrated in many package definitions in the `(gnu packages bioinformatics)` module.