5 ms·

Anyone using NixOS as a server in production? We're on FreeBSD now but I think Linux in our stack is inevitable (many servers, like Aerospike, are Linux only).

Anyone using NixOS as a server in production? We're on FreeBSD now but I think Linux in our stack is inevitable (many servers, like Aerospike, are Linux only). There's always Ubuntu, but this looks lovely. I've had my eye on it for a personal Linux workstation but haven't heard anything about it as a server.

teh 10y ago
We run a fleet. Happy to answer questions here or shoot us a mail at team@wearewizards.io if it's sensitive.
- tikhonj 10y ago
  Do you have a blog post about it or something? I don't have specific questions, I'd just love to see how Nix looks in production.
  - teh 10y ago
    No high level overview, just a fairly specific (and sadly slightly out of date) one here: https://blog.wearewizards.io/how-to-use-nixops-in-a-team https://blog.wearewizards.io/how-to-use-nixops-in-a-team It's still Linux but a lot of common problems go away. E.g. if you remove a user from your config then it's removed on the server (as opposed to Ansible). Or: Because you have a dependency DAG all services that need restarting after a change are known (no more Chef notifies :restart). Or: No more apt-get dist-upgrade. You can have a single package depend on an entirely new version of your OS and keep the old OS in place (except for the kernel version of course).
- viraptor 10y ago
  How's the binary security on NixOS these days? The official information I find are... worrying. Specifically: a) security updates are the same as all the other updates and may take days to get to you, do runtime replacements manually yourself (https://nixos.org/wiki/Security_Updates https://nixos.org/wiki/Security_Updates) and b) next to no hardening during compilation (https://nixos.org/wiki/Hardened_NixOS https://nixos.org/wiki/Hardened_NixOS)
  - frio 10y ago
    Good news! Hardened NixOS isn't far: https://github.com/NixOS/nixpkgs/pull/12895 https://github.com/NixOS/nixpkgs/pull/12895 :). I've been watching/waiting for that for a while myself, and the progress on that ticket has been encouraging enough that I've started building out NixOS stuff in anticipation of it landing by 16.09. Now I'm somewhat slavering for it because I've come to like NixOS a lot more than I thought I would...
  - teh 10y ago
    Both good points. In practice a) hasn't been an issue but for b) we had to apply a fair amount of patches ourselves. Luckily b is almost fixed, and if combined with e.g. grsecurity I think nix has a good story.
drvdevd 10y ago
I've used it as a server before and it's quite nice, it just requires getting used to the nix language a bit and possibly dealing with customization work when something you need isn't exposed already in /etc/nixos/configuration.nix. For example, imagine you needed some weird iptables functionality - you might end up writing some shell-ish rules using 'networking.firewall.extraCommands' [https://github.com/NixOS/nixos/blob/master/modules/services/networking/firewall.nix https://github.com/NixOS/nixos/blob/master/modules/services/...]. Of course other admins on your team might have to be on board with this learning curve too.
- drvdevd 10y ago
  I should add that there are the obvious benefits to these 'constraints' -- if you can stick with mostly what's exposed already in the default nix packages your configuration will be very clean and your system will be built in a reproducible way from the start.
jallmann 10y ago
I'm using a NixOS server in prod for an application. Works very nicely, makes deployment a breeze. I went from never using NixOS to having a production-ready deployment, including writing a nixpkg for Nignx configuration and my application, all in about a day. I love how an entire server can be configured with a single Nix file (configuration.nix) along with any custom packages, which are again flat-files. NixOS works best when you're 'all-in' with Nix, including using nixkpkgs over language package managers (or having such package managers be nix-aware). This is not something that's easily done right now, and precludes having a 'pure' Nix experience, especially when using it for dev purposes. In my case though, where the main application is a binary written in OCaml (which uses the opam package manager), NixOS makes it straightforward to build and distribute binary packages that are patched to the correct library paths on the system. Once that was set up, everything else was a breeze. Server maintenance is almost effortless.
rb808 10y ago
If you're a non-trivial shop you'll need to harden it, you're better off with CentOS/RHEL than Ubuntu.
- icebraining 10y ago
  Why?
  - joseph 10y ago
    Probably because mainstream distros like RHEL have third party tools available to verify that instances meet standards such as CIS. NixOS might be a great design, but that isn't going to matter to $InfoSecDepartment when they lack the ability to audit. Edit: Just noticed the parent was comparing RHEL to Ubuntu, not NixOS, so now I'm not sure :).
- ramblenode 10y ago
  NixOS recently got a big update to its Grsecurity module and will probably merge a large set of patches in the next month or so for compiler hardening. There is a lot of work being done on this front.

teh 10y ago

We run a fleet. Happy to answer questions here or shoot us a mail at team@wearewizards.io if it's sensitive.

tikhonj 10y ago
Do you have a blog post about it or something? I don't have specific questions, I'd just love to see how Nix looks in production.
- teh 10y ago
  No high level overview, just a fairly specific (and sadly slightly out of date) one here: https://blog.wearewizards.io/how-to-use-nixops-in-a-team https://blog.wearewizards.io/how-to-use-nixops-in-a-team It's still Linux but a lot of common problems go away. E.g. if you remove a user from your config then it's removed on the server (as opposed to Ansible). Or: Because you have a dependency DAG all services that need restarting after a change are known (no more Chef notifies :restart). Or: No more apt-get dist-upgrade. You can have a single package depend on an entirely new version of your OS and keep the old OS in place (except for the kernel version of course).
viraptor 10y ago
How's the binary security on NixOS these days? The official information I find are... worrying. Specifically: a) security updates are the same as all the other updates and may take days to get to you, do runtime replacements manually yourself (https://nixos.org/wiki/Security_Updates https://nixos.org/wiki/Security_Updates) and b) next to no hardening during compilation (https://nixos.org/wiki/Hardened_NixOS https://nixos.org/wiki/Hardened_NixOS)
- frio 10y ago
  Good news! Hardened NixOS isn't far: https://github.com/NixOS/nixpkgs/pull/12895 https://github.com/NixOS/nixpkgs/pull/12895 :). I've been watching/waiting for that for a while myself, and the progress on that ticket has been encouraging enough that I've started building out NixOS stuff in anticipation of it landing by 16.09. Now I'm somewhat slavering for it because I've come to like NixOS a lot more than I thought I would...
- teh 10y ago
  Both good points. In practice a) hasn't been an issue but for b) we had to apply a fair amount of patches ourselves. Luckily b is almost fixed, and if combined with e.g. grsecurity I think nix has a good story.

drvdevd 10y ago

I've used it as a server before and it's quite nice, it just requires getting used to the nix language a bit and possibly dealing with customization work when something you need isn't exposed already in /etc/nixos/configuration.nix. For example, imagine you needed some weird iptables functionality - you might end up writing some shell-ish rules using 'networking.firewall.extraCommands' [https://github.com/NixOS/nixos/blob/master/modules/services/networking/firewall.nix https://github.com/NixOS/nixos/blob/master/modules/services/...]. Of course other admins on your team might have to be on board with this learning curve too.

drvdevd 10y ago
I should add that there are the obvious benefits to these 'constraints' -- if you can stick with mostly what's exposed already in the default nix packages your configuration will be very clean and your system will be built in a reproducible way from the start.

jallmann 10y ago

I'm using a NixOS server in prod for an application. Works very nicely, makes deployment a breeze. I went from never using NixOS to having a production-ready deployment, including writing a nixpkg for Nignx configuration and my application, all in about a day. I love how an entire server can be configured with a single Nix file (configuration.nix) along with any custom packages, which are again flat-files. NixOS works best when you're 'all-in' with Nix, including using nixkpkgs over language package managers (or having such package managers be nix-aware). This is not something that's easily done right now, and precludes having a 'pure' Nix experience, especially when using it for dev purposes. In my case though, where the main application is a binary written in OCaml (which uses the opam package manager), NixOS makes it straightforward to build and distribute binary packages that are patched to the correct library paths on the system. Once that was set up, everything else was a breeze. Server maintenance is almost effortless.

rb808 10y ago

If you're a non-trivial shop you'll need to harden it, you're better off with CentOS/RHEL than Ubuntu.

icebraining 10y ago
Why?
- joseph 10y ago
  Probably because mainstream distros like RHEL have third party tools available to verify that instances meet standards such as CIS. NixOS might be a great design, but that isn't going to matter to $InfoSecDepartment when they lack the ability to audit. Edit: Just noticed the parent was comparing RHEL to Ubuntu, not NixOS, so now I'm not sure :).
ramblenode 10y ago
NixOS recently got a big update to its Grsecurity module and will probably merge a large set of patches in the next month or so for compiler hardening. There is a lot of work being done on this front.