4 ms·

Isn't a lot of this wisdom encapsulated in the (old, uncool) configuration management stuff like chef and puppet? The main difference being that you still need

Isn't a lot of this wisdom encapsulated in the (old, uncool) configuration management stuff like chef and puppet? The main difference being that you still need to rebuild your systems regularly to keep yourself honest (no lazy one-off changes, everything goes into the CM codebase). I mean, I get that NixOS can pin versions of everything, and we've all been bitten by a new server build on identical CM code failing because of an upstream version change. But it's an eternal problem: pinning all the versions means that you a now micromanaging a zillion different versions of shit that you don't want to care about.

PterodactylsRUs 5y ago
The problem with Chef and Puppet is that you tell it what to _do_ to a machine, but what you care about is what the machine _is_ or _has_. If you install a package on a machine using Chef, and then delete the line to install the package, it won't go ahead and uninstall the package. Even if you want to keep all your machines up-to-date with your configuration, and tell Chef to do an uninstall, uninstalls don't necessarily restore the state of the machine to what it was before the install. Data frequently gets left behind.
- snicker7 5y ago
  It’s worse than that. “Installing a package” is not meaningful without a reference to its version. Nix/Guix are supposed to give you byte-indentical systems.
  - raffraffraff 5y ago
    CM systems don't just install any old package. You can absolutely specify a version.
    - bfrog 5y ago
      Only what has been packaged, nix let’s you roll your own package if you like, letting you overlay ontop of existing expressions to produce your own. Want to rebuild a specific OpenSSL with a specific tool chain and compile time flags? Easily expressed in nix if you don’t mind the time to build it all
      - raffraffraff 5y ago
        We had an internal RPM repo where we pushed internally built packages. Not part of CM, but part of our solution.
- raffraffraff 5y ago
  I'm more familiar with puppet, but I'd venture to say that it really depends in how well you write your code. In my previous job where I used it heavily, every module had an enable parameter. If it was true, the module would follow one logic branch (install package, write config, perform action) but if it was false then it would follow another (remove package, delete config, perform action). This was a basic standard, and a new module would be rejected if it wasn't possible to toggle it on/off and configure it from Hiera (hierarchical yaml data). It wasn't perfect, but once the complex stuff was dockerized the puppet codebase shrank a lot. Most of the differences between our VMs were defined in Hiera (edit a yaml, toggle something on, override some default).
  - watusername 5y ago
    > In my previous job where I used it heavily, every module had an enable parameter. If it was true, the module would follow one logic branch (install package, write config, perform action) but if it was false then it would follow another (remove package, delete config, perform action). This is one area where NixOS shines: The ability to remove things from your configuration with no mental overhead. Even if you take great care to do all the cleanup work in your runbook, something is bound to get missed and you still haven't accounted for the state of the system before the runbook was run the first time. For NixOS it's easy: "Not in config" = "It's not there". All system configurations are themselves normal Nix expressions that get evaluated into derivations which are then built into immutable paths under /nix/store. Most files in /etc are simply symlinks to files in the Nix store, and the activation script takes care of creating and deleting those symlinks when a new configuration is activated.
  - paulfurtado 5y ago
    You can be fairly good about it, but there's always state with the potential to be left behind or become inconsistent: transitive dependencies of packages sticking around, packages with pre/post scripts, services writing out data files to unexpected locations on disk you forget to cleanup, a failure somewhere between a resource and a "notify" firing causing your service to not restart after a config file update and the next run not rerunning the notify, etc.
watusername 5y ago
> But it's an eternal problem: pinning all the versions means that you a now micromanaging a zillion different versions of shit that you don't want to care about. This is one common question people ask when I introduce them to Nix and its ability to pin packages. They are often surprised to hear that you only need to pin one thing, which is the version of the Nixpkgs package collection. Nixpkgs [1] is the central repo that contains all the Nix expressions for the "official" packages. It contains everything Nix needs to know to build a package. When you rebuild your system configuration, Nix will resolve all the packages using your local checkout of Nixpkgs into store derivations (.drv). A .drv file uniquely identifies a specific version of a package. When any input to the expression change (source URLs, dependencies, etc.), it will result in a different derivation that will be installed under different paths in /nix/store. In other words, different inputs -> different outputs. This is where binary distribution comes into play. Nix builds packages in sandboxes with tight isolation (no access to network or outside fs). Because the resulting store paths depend on the inputs being the same, for the same store path you will pretty much [2] get the exact same thing no matter where it's built. So Nix will lookup the binary cache for a matching object with the same store path. If it fails to find a match (e.g., because you modified the expression), Nix will simply build it. Binary cache can thus be seen as a transparent optimization that will get you binaries that are the same and will behave the same, as if you built it locally. To summarize, when you use NixOS, you already have the source (Nixpkgs) to be able pin everything. In fact, you cannot rebuild the system without it. If the expressions don't change, the results won't. The only thing you need to to is to actually pin it to a specific commit and keep track of it in your VCS. [1] https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs [2] https://r13y.com https://r13y.com
papishow 5y ago
What do you mean with old and uncool and wisdom encapsulated? NixOS predates puppet by 2 and chef by 7 years.
- deleted 5y ago
  [deleted]
l0b0 5y ago
I've configured my own and work systems for a few years using Puppet and Ansible, and I have to say Nix is miles ahead of them both. - Actually reproducible builds, as opposed to "only if all the external factors involved, like time and third party packages, are identical between builds". One thing this enables is building your system packages in your automated pipeline and just downloading the resulting cache. - Much more compact than either. For an example, the last state of my own Puppet configuration was [1] - five directories with hundreds of files. The current state is 13 files total, only one of which contains the Nix code actually needed to set up dozens of packages and services for a full desktop OS. That file clocks in at 346 lines, and comes in at a fraction of the size and complexity of the less featureful Puppet. - Nix shell[2], where you can build a set of packages for use within a single project without having to worry about it borking other projects (like adding things to $PATH). - Very few shell commands are ever needed. - Built-in features I haven't seen in either of the other systems: allowing unfree packages either globally or individually, setting up UEFI, LUKS and LVM with a handful of lines, and stopping the compilation immediately if there's a problem rather than continuing by default. The only caveat is the Nix language itself. The whole thing might be more complicated than the Puppet language, but you can see for yourself that the current configuration[3] isn't exactly rocket science. [1] https://gitlab.com/victor-engmark/root/-/tree/daa391e1d957f72d950ec8f2538556a251679d72 https://gitlab.com/victor-engmark/root/-/tree/daa391e1d957f7... [2] https://gitlab.com/victor-engmark/root/-/blob/master/shell.nix https://gitlab.com/victor-engmark/root/-/blob/master/shell.n... [3] https://gitlab.com/victor-engmark/root/-/blob/7f52eccc8b4c4c27d10a52b82f16617e6df49f88/configuration.nix https://gitlab.com/victor-engmark/root/-/blob/7f52eccc8b4c4c...
- raffraffraff 5y ago
  Can you ever see it used by a large organisation to manage fleets of machines? Can you continually apply a NixOS codebase across thousands of machines, securely? Is the language capable of handling roles, environments etc? Does it support an external database where you can manage variables (bonus if the lookups are environment-aware, so a 'dev' machine in Europe doesn't get 'test' config from the US). If so then it could genuinely replace any existing CM system. But displacing the existing tech, rewriting the whole CM codebase and getting the team over the learning curve is a monumental. I get that Puppet isn't 100% reproducible, but my point (in another part of this thread) is that time marches on and there are only so many things you need to (or want to) 'pin'. In the past, our practice was to leave software versions all 'latest' except for known issues or requirements. Otherwise you're in a constant fight with the security folks over old software versions. If we hit an issue on a puppet run in our test environment, we put the brakes on the puppet rollout until it's. Once it gets through a few days in test, we promote it to the next environment, and eventually do a staged release through the production environments.

PterodactylsRUs 5y ago

The problem with Chef and Puppet is that you tell it what to _do_ to a machine, but what you care about is what the machine _is_ or _has_. If you install a package on a machine using Chef, and then delete the line to install the package, it won't go ahead and uninstall the package. Even if you want to keep all your machines up-to-date with your configuration, and tell Chef to do an uninstall, uninstalls don't necessarily restore the state of the machine to what it was before the install. Data frequently gets left behind.

snicker7 5y ago
It’s worse than that. “Installing a package” is not meaningful without a reference to its version. Nix/Guix are supposed to give you byte-indentical systems.
- raffraffraff 5y ago
  CM systems don't just install any old package. You can absolutely specify a version.
  - bfrog 5y ago
    Only what has been packaged, nix let’s you roll your own package if you like, letting you overlay ontop of existing expressions to produce your own. Want to rebuild a specific OpenSSL with a specific tool chain and compile time flags? Easily expressed in nix if you don’t mind the time to build it all
    - raffraffraff 5y ago
      We had an internal RPM repo where we pushed internally built packages. Not part of CM, but part of our solution.
raffraffraff 5y ago
I'm more familiar with puppet, but I'd venture to say that it really depends in how well you write your code. In my previous job where I used it heavily, every module had an enable parameter. If it was true, the module would follow one logic branch (install package, write config, perform action) but if it was false then it would follow another (remove package, delete config, perform action). This was a basic standard, and a new module would be rejected if it wasn't possible to toggle it on/off and configure it from Hiera (hierarchical yaml data). It wasn't perfect, but once the complex stuff was dockerized the puppet codebase shrank a lot. Most of the differences between our VMs were defined in Hiera (edit a yaml, toggle something on, override some default).
- watusername 5y ago
  > In my previous job where I used it heavily, every module had an enable parameter. If it was true, the module would follow one logic branch (install package, write config, perform action) but if it was false then it would follow another (remove package, delete config, perform action). This is one area where NixOS shines: The ability to remove things from your configuration with no mental overhead. Even if you take great care to do all the cleanup work in your runbook, something is bound to get missed and you still haven't accounted for the state of the system before the runbook was run the first time. For NixOS it's easy: "Not in config" = "It's not there". All system configurations are themselves normal Nix expressions that get evaluated into derivations which are then built into immutable paths under /nix/store. Most files in /etc are simply symlinks to files in the Nix store, and the activation script takes care of creating and deleting those symlinks when a new configuration is activated.
- paulfurtado 5y ago
  You can be fairly good about it, but there's always state with the potential to be left behind or become inconsistent: transitive dependencies of packages sticking around, packages with pre/post scripts, services writing out data files to unexpected locations on disk you forget to cleanup, a failure somewhere between a resource and a "notify" firing causing your service to not restart after a config file update and the next run not rerunning the notify, etc.

watusername 5y ago

> But it's an eternal problem: pinning all the versions means that you a now micromanaging a zillion different versions of shit that you don't want to care about. This is one common question people ask when I introduce them to Nix and its ability to pin packages. They are often surprised to hear that you only need to pin one thing, which is the version of the Nixpkgs package collection. Nixpkgs [1] is the central repo that contains all the Nix expressions for the "official" packages. It contains everything Nix needs to know to build a package. When you rebuild your system configuration, Nix will resolve all the packages using your local checkout of Nixpkgs into store derivations (.drv). A .drv file uniquely identifies a specific version of a package. When any input to the expression change (source URLs, dependencies, etc.), it will result in a different derivation that will be installed under different paths in /nix/store. In other words, different inputs -> different outputs. This is where binary distribution comes into play. Nix builds packages in sandboxes with tight isolation (no access to network or outside fs). Because the resulting store paths depend on the inputs being the same, for the same store path you will pretty much [2] get the exact same thing no matter where it's built. So Nix will lookup the binary cache for a matching object with the same store path. If it fails to find a match (e.g., because you modified the expression), Nix will simply build it. Binary cache can thus be seen as a transparent optimization that will get you binaries that are the same and will behave the same, as if you built it locally. To summarize, when you use NixOS, you already have the source (Nixpkgs) to be able pin everything. In fact, you cannot rebuild the system without it. If the expressions don't change, the results won't. The only thing you need to to is to actually pin it to a specific commit and keep track of it in your VCS. [1] https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs [2] https://r13y.com https://r13y.com

papishow 5y ago

What do you mean with old and uncool and wisdom encapsulated? NixOS predates puppet by 2 and chef by 7 years.

deleted 5y ago
[deleted]

l0b0 5y ago

I've configured my own and work systems for a few years using Puppet and Ansible, and I have to say Nix is miles ahead of them both. - Actually reproducible builds, as opposed to "only if all the external factors involved, like time and third party packages, are identical between builds". One thing this enables is building your system packages in your automated pipeline and just downloading the resulting cache. - Much more compact than either. For an example, the last state of my own Puppet configuration was [1] - five directories with hundreds of files. The current state is 13 files total, only one of which contains the Nix code actually needed to set up dozens of packages and services for a full desktop OS. That file clocks in at 346 lines, and comes in at a fraction of the size and complexity of the less featureful Puppet. - Nix shell[2], where you can build a set of packages for use within a single project without having to worry about it borking other projects (like adding things to $PATH). - Very few shell commands are ever needed. - Built-in features I haven't seen in either of the other systems: allowing unfree packages either globally or individually, setting up UEFI, LUKS and LVM with a handful of lines, and stopping the compilation immediately if there's a problem rather than continuing by default. The only caveat is the Nix language itself. The whole thing might be more complicated than the Puppet language, but you can see for yourself that the current configuration[3] isn't exactly rocket science. [1] https://gitlab.com/victor-engmark/root/-/tree/daa391e1d957f72d950ec8f2538556a251679d72 https://gitlab.com/victor-engmark/root/-/tree/daa391e1d957f7... [2] https://gitlab.com/victor-engmark/root/-/blob/master/shell.nix https://gitlab.com/victor-engmark/root/-/blob/master/shell.n... [3] https://gitlab.com/victor-engmark/root/-/blob/7f52eccc8b4c4c27d10a52b82f16617e6df49f88/configuration.nix https://gitlab.com/victor-engmark/root/-/blob/7f52eccc8b4c4c...

raffraffraff 5y ago
Can you ever see it used by a large organisation to manage fleets of machines? Can you continually apply a NixOS codebase across thousands of machines, securely? Is the language capable of handling roles, environments etc? Does it support an external database where you can manage variables (bonus if the lookups are environment-aware, so a 'dev' machine in Europe doesn't get 'test' config from the US). If so then it could genuinely replace any existing CM system. But displacing the existing tech, rewriting the whole CM codebase and getting the team over the learning curve is a monumental. I get that Puppet isn't 100% reproducible, but my point (in another part of this thread) is that time marches on and there are only so many things you need to (or want to) 'pin'. In the past, our practice was to leave software versions all 'latest' except for known issues or requirements. Otherwise you're in a constant fight with the security folks over old software versions. If we hit an issue on a puppet run in our test environment, we put the brakes on the puppet rollout until it's. Once it gets through a few days in test, we promote it to the next environment, and eventually do a staged release through the production environments.