6 ms·

I've been waiting for this announcement for a few weeks. Since the home page is quite lean on an actual explanation, let me give a rough summary of interesting

I've been waiting for this announcement for a few weeks. Since the home page is quite lean on an actual explanation, let me give a rough summary of interesting points as i understand it. I apologize in advance if I have anything wrong. 1. It is a from-scratch source build system written in rust that borrows a huge amount from nix. This means repeatable software builds with isolated dependencies. 2. It attempts to move some configuration management primitives into the build pipeline so that the deployment config has enough flexibility for real-world use. 3. It intends to replace single host process supervision with a distributed model based on gossip. This project is most definitely ambitious, and I'm not sure the whole goal is realizable but some aspects are particularly interesting. 1. The build process seems quite useful for building containers, especially if it gets some traction. The main drawback of nix is that no one knows how to use it. If this gets the backing of the hordes of chef-aware system administrators, we may have a good solution for isolated repeatable builds, especially if they focus on deterministic reproducible builds[1] as well. 2. Distributed systems are notoriously hard to configure in a general way. It seems like every company has their own magic combination of config management, monitoring, and scripting for keeping systems like zookeeper running. Its pretty clear that we don't have the right primitives for sharing this work. I don't think a container management system has the right primiteves for this either, because I don't think you can solve for all distributed systems in a generic way. You really need custom logic for the particular software you are trying to distribute. I'm not convinced that a gossip-based process-supervisor is the best approach for this, but it could give us a place to start collaborating on these primitives. [1] https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds

davexunit 10y ago
>1. It is a from-scratch source build system written in rust that borrows a huge amount from nix. This means repeatable software builds with isolated dependencies. Except the builds aren't at all repeatable in a meaningful way, judging by the example code on the home page. The 'do_build' function runs 'npm install'. This means that 1) the build container has network access (i.e. it's nondeterministic by definition) and 2) the declared dependencies do not fully describe the dependency graph. So, I really don't see how much it could possibly borrow from Nix if it threw out the most important part of it.
- vishvananda 10y ago
  They used npm install in their example code? Well that was a poor choice. Having dealt with npm and node apps in my own build system, I have not found a good solution for deterministically building node apps. Does nix have a solution for this? Thankfully the prepackaged software with reasonable source build systems don't do craziness like this. See the redis package for example: https://app.habitat.sh/#/pkgs/adam/redis/3.0.7/20160613205253 https://app.habitat.sh/#/pkgs/adam/redis/3.0.7/2016061320525...
  - trishume 10y ago
    Nix has a generator that generates a Nix file based on an npm package including SHAs and locked down versions. Unfortunately because typical JS packages use a bajillion dependencies each recursively these Nix files can be tens of thousands of lines, but they work. Nix also has similar generators for rubygems, python, haskell and some other languages. Nix's cargo integration is even cooler since cargo is already quite deterministic, you just put in a SHA of the final folder cargo creates with all the dependencies.
  - cwp 10y ago
    The nix solution is npm2nix, which runs npm against a package.json file and generates a nix expression for all the npm modules you need. This mostly works, but occasionally you need to override the generated derivations to add implicit dependencies that npm doesn't capture, or otherwise tweak packages that rely on quirks of npm.
    - jjuhl 10y ago
      So basically it "does not work"...
      - jonaf 10y ago
        Well, let's be clear, if it doesn't work, you know at compile time, not at runtime. I don't care how broken things are at compile time if I'm guaranteed that everything will work at runtime. That's kind of the whole point of nix. It's not impossible to write code with errors, but you should not be able to build as long as the errors exist.
mbrock 10y ago
Thanks for trying to explain. So many times when I visit these websites advertising developer-oriented tools I just click around trying to find an actual explanation of what the thing does and how it works, only to close the tab a couple of minutes later with no clear idea.

davexunit 10y ago

>1. It is a from-scratch source build system written in rust that borrows a huge amount from nix. This means repeatable software builds with isolated dependencies. Except the builds aren't at all repeatable in a meaningful way, judging by the example code on the home page. The 'do_build' function runs 'npm install'. This means that 1) the build container has network access (i.e. it's nondeterministic by definition) and 2) the declared dependencies do not fully describe the dependency graph. So, I really don't see how much it could possibly borrow from Nix if it threw out the most important part of it.

vishvananda 10y ago
They used npm install in their example code? Well that was a poor choice. Having dealt with npm and node apps in my own build system, I have not found a good solution for deterministically building node apps. Does nix have a solution for this? Thankfully the prepackaged software with reasonable source build systems don't do craziness like this. See the redis package for example: https://app.habitat.sh/#/pkgs/adam/redis/3.0.7/20160613205253 https://app.habitat.sh/#/pkgs/adam/redis/3.0.7/2016061320525...
- trishume 10y ago
  Nix has a generator that generates a Nix file based on an npm package including SHAs and locked down versions. Unfortunately because typical JS packages use a bajillion dependencies each recursively these Nix files can be tens of thousands of lines, but they work. Nix also has similar generators for rubygems, python, haskell and some other languages. Nix's cargo integration is even cooler since cargo is already quite deterministic, you just put in a SHA of the final folder cargo creates with all the dependencies.
- cwp 10y ago
  The nix solution is npm2nix, which runs npm against a package.json file and generates a nix expression for all the npm modules you need. This mostly works, but occasionally you need to override the generated derivations to add implicit dependencies that npm doesn't capture, or otherwise tweak packages that rely on quirks of npm.
  - jjuhl 10y ago
    So basically it "does not work"...
    - jonaf 10y ago
      Well, let's be clear, if it doesn't work, you know at compile time, not at runtime. I don't care how broken things are at compile time if I'm guaranteed that everything will work at runtime. That's kind of the whole point of nix. It's not impossible to write code with errors, but you should not be able to build as long as the errors exist.

mbrock 10y ago

Thanks for trying to explain. So many times when I visit these websites advertising developer-oriented tools I just click around trying to find an actual explanation of what the thing does and how it works, only to close the tab a couple of minutes later with no clear idea.