4 ms·
> I am really not convinced of the need of any incremental build system (like Bazel) over Nix [...] Without smaller compilation units than the default mkDeriva
by q3k 5y ago
> I am really not convinced of the need of any incremental build system (like Bazel) over Nix [...]
Without smaller compilation units than the default mkDerivation-per-component-style in Nix you end up having _extremely_ long build/test cycles on every change. You could alleviate this by deferring to whatever build system is called in mkDerivation by providing a short-circuiting shell.nix for every subcomponent (eg. a shell.nix to drop you into somewhere where you can `cargo build`). But IMO that's not nearly as nice of an experience as a `nix-build -A some-component` or `bazel build //some/component`, especially when changes span multiple components.
> You can also simulate incremental build in Nix-only by using multiple derivations over a single source tree and combine them together.
Yes, you can put in the effort to replicate Bazel-style small-compilation-unit language rules in nix. But then your evaluation time will skyrocket, with nix spending dozen of seconds re-evaluating these tiny derivations before even getting to the build - this already happens when evaluating complex derivations like NixOS configuration builds. This also kills your build/test cycle. Plus, you end up polluting your /nix/store with thousands of tiny out-of-date artifacts that then need to be garbage collected (which also becomes slow when the graph grows).
Bazel's build/evaluation model (workspace rules -> targets -> build graph -> action graph -> artifact cache) and the fact that it runs as a server makes it much easier to have snappy rebuilds out of the box without any special consideration for derivation granularity or crutches like shell.nix-per-component. I'm not saying Bazel is perfect, and it's quite difficult to get running in an organization if you don't know it well, but once it's set up correctly it's pretty much the best development workflow I've ever used.
Nix is very good for packaging software that you barely change, eg. for packaging software, as it makes it very easy to hermetically build software that comes with its own build system. But from my experience it's not a great experience for building software you develop, as build times rapidly rise to the point of frustration.
> [...] excepted if you inherited of a giant mono-repo mess.
I'm not sure how to interpret this - both Nix and Bazel shine in monorepo setups (and are IMO a prerequisite to even consider using a monorepo). Even nixpkgs is its own monorepo, and thanks to that is probably the best contributing-to-a-Linux-distribution experience that exists, by a long shot. Are you instead using Nix with flakes accross multiple repositories? Does that alleviate evaluation times?
- adev_ 5y ago> ou could alleviate this by deferring to whatever build system is called in mkDerivation by providing a short-circuiting shell.nix for every subcomponent (eg. a shell.nix to drop you into somewhere where you can `cargo build`). But IMO that's not nearly as nice of an experience as a `nix-build -A some-component` And you describe exactly what we do :) That's not that bad. The multi-components scenario can also be scripted through some specific nix-shell targets that you configure for a local source usage. It is not perfect but it does the job. I do consider that if multi-components modifications happen on regular base, it is anyway a sign of a problem: they should probably not be separated components in the first place. > But then your evaluation time will skyrocket, with nix spending dozen of seconds re-evaluating these tiny derivations before even getting to the build - this already happens when evaluating complex derivations like NixOS configuration builds We do that by using overlays for our package set ( > 250 packages) over the bigger package set (the entire nixpkgs) which is provided through Nix channels. That allows to leverage the Nix internal cache and keep evaluation under few seconds. Also we do use the nix-daemon. > Plus, you end up polluting your /nix/store with thousands of tiny out-of-date artifacts that then need to be garbage collected (which also becomes slow when the graph grows). That is true, but space is cheap :) > [...] I'm not saying Bazel is perfect, and it's quite difficult to get running in an organization if you don't know it well, but once it's set up correctly it's pretty much the best development workflow I've ever used. I understand the point of view but I have the opposite experience. Bazel is one of the worst build system I had to experience. The fact he tries to act both as a package manager and as a build system (CMake, make, configure) makes it conflict with almost everything else in the Open Source world which is not made from day 1 for Bazel: This, for me is unacceptable. I will also pass under silence the long list of quirks it has and the instability of the previous versions. > Even nixpkgs is its own monorepo, and thanks to that is probably the best contributing-to-a-Linux-distribution experience that exists, by a long shot. Nix requires only its configuration (Nix's expression) to be in monorepo. At the opposite, Bazel design is centered around the source code being in monorepo. That is a major difference. One centralize versioning, the other one centralize everything.
- jvolkman 5y ago> Nix requires only its configuration (Nix's expression) to be in monorepo. At the opposite, Bazel design is centered around the source code being in monorepo. That is a major difference. Modern Bazel rule sets are more pragmatic than that. Python rules can pull packages using `pip install`, java rules can pull from Maven, rust rules from crates.io, etc.