7 ms·

I'm intrigued by the "atomic upgrades and rollbacks" claim. Let's say that deploying a package requires placing two files somewhere, e.g. vmlinuz and initrd. Ho

I'm intrigued by the "atomic upgrades and rollbacks" claim. Let's say that deploying a package requires placing two files somewhere, e.g. vmlinuz and initrd. How can this be done atomically?

sp332 11y ago
The package manager could just flock() a specific file before making any changes.
- millstone 11y ago
  From svn's red book: By atomic transaction, we mean simply this: either all of the changes happen in the repository, or none of them happens. Subversion tries to retain this atomicity in the face of program crashes, system crashes, network problems, and other users' actions. Does Nix mean the same thing? Is it atomic in the face of program or system crashes?
  - cwp 11y ago
    Yes. In many cases, switching to the new version of the package is nothing more than rewriting a symlink. Sometimes it's more involved, like restarting a service. Either way, the cut over happens after building the new package has completed successfully. If building the package fails, it has no effect on the running system. I suppose there is still a window in which an error could leave the system in an inconsistent state, but it's pretty narrow, and the error would have to be a kernel panic or something like that.
    - millstone 11y ago
      Err, are there any systems that try to cut over to a new package before it's been built successfully?
      - thrownaway122 11y ago
        I guess that traditional package managers could install a helper script or a config file early in the build system before the main package is finished.
      - cwp 11y ago
        Yes. Almost all of them, actually. By "build" I mean the whole package installation process, including moving the compiled files into their final destinations. If you're upgrading an existing package, and that involves overwriting one binary in /usr/bin and another in /usr/lib, a crash in the middle of that process leaves the package partially upgraded, and very likely broken. Nix never overwrites an existing package installation. Instead, it installs the new version in a separate directory, then overwrites a symlink that pointed to the old version with a symlink pointing to the new version.
  - Lethalman 11y ago
    Things you must be afraid when using nix: 1) If you install a bad grub, you cannot rollback it using nix. That means, not a wrong grub line, but if grub itself is broken. 2) Anything that is not under nix control, the data: like desktop configurations, databases, ecc.
    - davexunit 11y ago
      But after the first successful install you shouldn't have to worry about GRUB anymore. I don't know what it's like on NixOS, but on GuixSD we just change the grub.cfg.
      - Lethalman 11y ago
        You also upgrade grub versions I hope :) Or maybe improve the grub installation scripts.
- cwp 11y ago
  The contents of any package are immutable; once they're written they're never modified again. New versions of the package get installed in a different directory tree.
tomaac 11y ago
I am not sure if this is the same, but Redhat's Project Atomic does similar thing using rpm-ostree http://www.projectatomic.io/ http://www.projectatomic.io/
- Lethalman 11y ago
  It's a different approach. That's basically an implementation using ostree. Nix is way different than that. Neither is better in my opinion, but I use Nix happily.
jfoutz 11y ago
The handwavy answer is, each install (even later versions of the same package) gets installed into a unique directory. There is no /bin or /usr/bin. bit more info here: https://nixos.org/nixos/about.html https://nixos.org/nixos/about.html
- millstone 11y ago
  Literally, there is no /bin or /usr/bin? So most shebang scripts won't run at all? That is indeed a radical departure.
  - jfoutz 11y ago
    Sorta buried in that link "A big implication of the way that Nix/NixOS stores packages is that there is no /bin, /sbin, /lib, /usr, and so on. Instead all packages are kept in /nix/store. (The only exception is a symlink /bin/sh to Bash in the Nix store.) Not using ‘global’ directories such as /bin is what allows multiple versions of a package to coexist. Nix does have a /etc to keep system-wide configuration files, but most files in that directory are symlinks to generated files in /nix/store." So you get bash :)
  - Lethalman 11y ago
    There's a /bin/sh because libc needs it, but we're going to remove that one too in the future somehow. But /usr/bin/env is in nixos to make it easy for people to run scripts. But packages never use /usr/bin/env, it's just there for convenience.
    - millstone 11y ago
      Ok, if env is supplied then scripts can at least use it to locate Python, etc. which makes a lot of sense. The text made it sound like these directories did not exist at all.
  - deleted 11y ago
    [deleted]
  - deleted 11y ago
    [deleted]
mbrock 11y ago
They use atomic file operations involving symlinks. Moving a new symlink to overwrite an old symlink is one way to atomically switch out a directory for another. I don't know if there is any special handling of the kernel packages, but that's basically how it works as far as I know.

I'm intrigued by the "atomic upgrades and rollbacks" claim. Let's say that deploying a package requires placing two files somewhere, e.g. vmlinuz and initrd. How can this be done atomically?

sp332 11y ago

The package manager could just flock() a specific file before making any changes.

millstone 11y ago
From svn's red book: By atomic transaction, we mean simply this: either all of the changes happen in the repository, or none of them happens. Subversion tries to retain this atomicity in the face of program crashes, system crashes, network problems, and other users' actions. Does Nix mean the same thing? Is it atomic in the face of program or system crashes?
- cwp 11y ago
  Yes. In many cases, switching to the new version of the package is nothing more than rewriting a symlink. Sometimes it's more involved, like restarting a service. Either way, the cut over happens after building the new package has completed successfully. If building the package fails, it has no effect on the running system. I suppose there is still a window in which an error could leave the system in an inconsistent state, but it's pretty narrow, and the error would have to be a kernel panic or something like that.
  - millstone 11y ago
    Err, are there any systems that try to cut over to a new package before it's been built successfully?
    - thrownaway122 11y ago
      I guess that traditional package managers could install a helper script or a config file early in the build system before the main package is finished.
    - cwp 11y ago
      Yes. Almost all of them, actually. By "build" I mean the whole package installation process, including moving the compiled files into their final destinations. If you're upgrading an existing package, and that involves overwriting one binary in /usr/bin and another in /usr/lib, a crash in the middle of that process leaves the package partially upgraded, and very likely broken. Nix never overwrites an existing package installation. Instead, it installs the new version in a separate directory, then overwrites a symlink that pointed to the old version with a symlink pointing to the new version.
- Lethalman 11y ago
  Things you must be afraid when using nix: 1) If you install a bad grub, you cannot rollback it using nix. That means, not a wrong grub line, but if grub itself is broken. 2) Anything that is not under nix control, the data: like desktop configurations, databases, ecc.
  - davexunit 11y ago
    But after the first successful install you shouldn't have to worry about GRUB anymore. I don't know what it's like on NixOS, but on GuixSD we just change the grub.cfg.
    - Lethalman 11y ago
      You also upgrade grub versions I hope :) Or maybe improve the grub installation scripts.
cwp 11y ago
The contents of any package are immutable; once they're written they're never modified again. New versions of the package get installed in a different directory tree.

tomaac 11y ago

I am not sure if this is the same, but Redhat's Project Atomic does similar thing using rpm-ostree http://www.projectatomic.io/ http://www.projectatomic.io/

Lethalman 11y ago
It's a different approach. That's basically an implementation using ostree. Nix is way different than that. Neither is better in my opinion, but I use Nix happily.

jfoutz 11y ago

The handwavy answer is, each install (even later versions of the same package) gets installed into a unique directory. There is no /bin or /usr/bin. bit more info here: https://nixos.org/nixos/about.html https://nixos.org/nixos/about.html

millstone 11y ago
Literally, there is no /bin or /usr/bin? So most shebang scripts won't run at all? That is indeed a radical departure.
- jfoutz 11y ago
  Sorta buried in that link "A big implication of the way that Nix/NixOS stores packages is that there is no /bin, /sbin, /lib, /usr, and so on. Instead all packages are kept in /nix/store. (The only exception is a symlink /bin/sh to Bash in the Nix store.) Not using ‘global’ directories such as /bin is what allows multiple versions of a package to coexist. Nix does have a /etc to keep system-wide configuration files, but most files in that directory are symlinks to generated files in /nix/store." So you get bash :)
- Lethalman 11y ago
  There's a /bin/sh because libc needs it, but we're going to remove that one too in the future somehow. But /usr/bin/env is in nixos to make it easy for people to run scripts. But packages never use /usr/bin/env, it's just there for convenience.
  - millstone 11y ago
    Ok, if env is supplied then scripts can at least use it to locate Python, etc. which makes a lot of sense. The text made it sound like these directories did not exist at all.
- deleted 11y ago
  [deleted]
- deleted 11y ago
  [deleted]

mbrock 11y ago

They use atomic file operations involving symlinks. Moving a new symlink to overwrite an old symlink is one way to atomically switch out a directory for another. I don't know if there is any special handling of the kernel packages, but that's basically how it works as far as I know.