6 ms·

I'd say nix is the more deterministic way to get all, and only, the things you absolutely need. Docker has the same issue as vagrant that you'll get a different

by kreetx 3y ago

I'd say nix is the more deterministic way to get all, and only, the things you absolutely need. Docker has the same issue as vagrant that you'll get a different debian depending on the day you init the VM, run apt, and so on.

oneplane 3y ago
It depends. Depends on a whole lot of things, and Nix isn't special in this: Do you re-use an existing artefact or do you build a new one? You can put Nix in Docker and in Vagrant and get the same result every time. You an also not use Nix, create a Docker image or a Vagrant box, and whenever you instantiate one of those you get the same result every time. You an also involve Nix and not pin to a specific release and now you get a different result every time. That would be the same as creating a Dockerfile with a mutable tag or Vagrant file with a dynamic setup. Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need.". That is something that is probably only really useful in CI when you need reproducible builds. For everything else you probably want to be tracking a stable release for the version you target. Especially since most people aren't working to get a deterministic result, but a 'close enough' result to get the job done.
- kreetx 3y ago
  In nix you can pin your environment down to a commit in nixpkgs, from which whenever you build your system (i.e, artefact), the result will be the same. How do you achieve this in docker or vagrant? > Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need." IMO, you do, as otherwise you'll need to fix additional issues every time your CI runs, no?
  - oneplane 3y ago
    In docker and vagrant you do it the same way, the hash of the specific artefact. That can be a commit ID, but you can also do this without Git and the likes and use a content hash. Those are guaranteed to always refer to the same thing. >> Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need." >IMO, you do, as otherwise you'll need to fix additional issues every time your CI runs, no? That is why I wrote about the specific exception for CI, but since we're talking about humans experimenting in a box, we're not talking about CI.
    - kreetx 3y ago
      > In docker and vagrant you do it the same way, the hash of the specific artefact. No, in nix you pin the commit hash of nixpkgs, which gives you the exact same package set every time. But with a Docerfile, even if you add it to your project's repo then any `apt-get install something` will still give you a different something depending on when you build the image. In nix, you always get the same something. > That is why I wrote about the specific exception for CI I meant that you keep the dev environment the same as CI because if you develop in a different environment than the CI then whenever CI runs then you'll get new issues (because you weren't using the same environment).
      - oneplane 3y ago
        I wasn't referring to Nix, how Nix uses hashing, what specific hash, or source implementation it is using, or that it might be using an artefact hash. I was referring to the referencing method: a hash. This is the same method in the three mentioned products (nix, docker, vagrant): an immutable pointer to a specific resource. As for CI, dev environments, not relevant here. I only mentioned in passing because CI is a common use case for hardcoded immutable reference. We're talking about someone wanting a box to experiment in, not CI. Not even software engineering either.
        kreetx 3y ago
        What hashes do you use in docker or vagrant?
        oneplane 3y ago
        SHA256, generally. So for example, you might use 5c15a6e5c0bf02e6c0eaa939cb543c41d7725453064c920b9a4faeea7c357506. (This is a digest hash you'd use in Docker, as an example) You can both consume that specific one, but also reproduce it locally if you want to. Depending on your intent, bandwidth and trust you can pick the method you desire.
        3y ago
- k8svet 3y ago
  >create a Docker image or a Vagrant box, and whenever you instantiate one of those I'm passing X to doubt. This is just odd analysis. Nix is strictly superior to Docker, except the learning curve. You have to really work at it to make any modern Nix setup non-reproducible.
  - oneplane 3y ago
    You can pass X as much as you like, but Nix, like anything else, can be configured with dynamic references. Nix is also not related to Docker (neither superior nor inferior), and the learning curve makes it unsuitable for everything where Docker and Vagrant are suitable, even if it would technically do the same thing. Either way, you are missing the point: the problem here isn't having perfect reproduction. It's about having an environment you can learn in without having to worry about breaking things. That's why the author references vagrant since it's about as low a barrier to entry as disposable environments can be (and even still a barrier to high for some).
    - kreetx 3y ago
      I think it's you who is missing the point: the learning environment won't build as neither docker nor vagrant will give you reproducible environments, while nix will. Sure, you can make your nix environments dynamic too, but having determinism is its core feature. I don't think it is with docker nor vagrant, as the environment you get will be isolated, but not deterministic. You can, of course bolt some custom thing on top of those (or use nix), but that is going to be a pain. EDIT: Both vagrant and docker are useful tools, it's just that for creating any dev environments nix will be better due to determinism.
      - oneplane 3y ago
        The point is, and always was: someone wants a box to experiment in. Nix is not that box. It was never the box. It will never be the box. That is the point I am making and reflects the point the author was making. And just in case there is a language barrier: box does not refer to a specific technical implementation of a box, it's just a term to denote a border between the user's system (which they don't want to break) and "something else". Edit: and just in case the word 'someone' trips over a language barrier, I'm not referring to 'anyone' but to the persona (the 'someone') who might want to try something out because they saw something and thought it was cool. Not someone with package manager experience, not a software engineer, not a sysadmin. We're also not 'building a learning environment'. Learning environment is a proxy for disposable environment, which as a relatively simple concept is already a step too far for the average "how do I become a cool hacker like on TV" case (which is where the unreasonable effectiveness from the title comes in) where someone might take their first steps and wanting to try something out without breaking their current environment.
- deleted 3y ago
  [deleted]
nunez 3y ago
You can avoid that unpredictability if you don't use the latest tag, or, better, create images or start containers from SHA256 digests. Those are guaranteed to be immutable. I.e instead of: Docker run --rm debian:trixie You'd do Docker run --rm debian:trixie@sha256:0ee9224bb31d3622e84d15260cf759bf37f08d652cc0411ca7b1f785f87ac19c The only disadvantage of the digest approach is that you would need to manually resolve the digest that's correct for your processor arch. Using bare tags like "debian:trixie" can resolve to manifest lists (if so configured) that has Docker automatically find the right digest for your arch.
- kreetx 3y ago
  What if the environment needs more packages installed via apt, do you save those hashes somewhere, too?
  - codethief 3y ago
    Debian & Ubuntu provide daily snapshots of their repositories, see snapshot.debian.org and https://ubuntu.com/blog/ubuntu-snapshots-on-azure-ensuring-predictability-and-consistency-in-cloud-deployments https://ubuntu.com/blog/ubuntu-snapshots-on-azure-ensuring-p... See e.g. https://github.com/reproducible-containers/repro-sources-list.sh https://github.com/reproducible-containers/repro-sources-lis... for how to use them. Also, at least for Debian there are official images available that use snapshot.debian.org as package repository from the get-go – unfortunately those images are not published on a daily basis yet. It seems the overall topic if reproducibility is on the Docker team's to do list, though, see https://github.com/docker-library/official-images/issues/16044 https://github.com/docker-library/official-images/issues/160...
    - kreetx 3y ago
      But if you read those links, you'll see that it's not quite there yet. Docker in general can't solve reproducibility - it's the package manager within any container that does that.
      - 3y ago
hiAndrewQuinn 3y ago
I deem that you are correct! I have investigated NixOS in a Vagrant + Virtualbox combo very similar to what I did in this post, and yes, this is a much more deterministic solution to this idea. If the people you are writing a tutorial for can be expected to be familiar with Nix, this is probably a much more future-proofed solution. That's still a small camp of people at the moment, but it does include the creator of Vagrant himself, Mitchell Hashimoto. I maintain that a specific Debian version is a better "lingua franca" for a general audience - that may change in coming years, of course. I have begun writing a new tutorial called "NixOS: Three Big Ideas" to expose people like myself who haven't given Nix the time of day to explore its concepts in about 20 minutes or so.

by kreetx 3y ago

oneplane 3y ago

It depends. Depends on a whole lot of things, and Nix isn't special in this: Do you re-use an existing artefact or do you build a new one? You can put Nix in Docker and in Vagrant and get the same result every time. You an also not use Nix, create a Docker image or a Vagrant box, and whenever you instantiate one of those you get the same result every time. You an also involve Nix and not pin to a specific release and now you get a different result every time. That would be the same as creating a Dockerfile with a mutable tag or Vagrant file with a dynamic setup. Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need.". That is something that is probably only really useful in CI when you need reproducible builds. For everything else you probably want to be tracking a stable release for the version you target. Especially since most people aren't working to get a deterministic result, but a 'close enough' result to get the job done.

kreetx 3y ago
In nix you can pin your environment down to a commit in nixpkgs, from which whenever you build your system (i.e, artefact), the result will be the same. How do you achieve this in docker or vagrant? > Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need." IMO, you do, as otherwise you'll need to fix additional issues every time your CI runs, no?
- oneplane 3y ago
  In docker and vagrant you do it the same way, the hash of the specific artefact. That can be a commit ID, but you can also do this without Git and the likes and use a content hash. Those are guaranteed to always refer to the same thing. >> Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need." >IMO, you do, as otherwise you'll need to fix additional issues every time your CI runs, no? That is why I wrote about the specific exception for CI, but since we're talking about humans experimenting in a box, we're not talking about CI.
  - kreetx 3y ago
    > In docker and vagrant you do it the same way, the hash of the specific artefact. No, in nix you pin the commit hash of nixpkgs, which gives you the exact same package set every time. But with a Docerfile, even if you add it to your project's repo then any `apt-get install something` will still give you a different something depending on when you build the image. In nix, you always get the same something. > That is why I wrote about the specific exception for CI I meant that you keep the dev environment the same as CI because if you develop in a different environment than the CI then whenever CI runs then you'll get new issues (because you weren't using the same environment).
    - oneplane 3y ago
      I wasn't referring to Nix, how Nix uses hashing, what specific hash, or source implementation it is using, or that it might be using an artefact hash. I was referring to the referencing method: a hash. This is the same method in the three mentioned products (nix, docker, vagrant): an immutable pointer to a specific resource. As for CI, dev environments, not relevant here. I only mentioned in passing because CI is a common use case for hardcoded immutable reference. We're talking about someone wanting a box to experiment in, not CI. Not even software engineering either.
      - kreetx 3y ago
        What hashes do you use in docker or vagrant?
        oneplane 3y ago
        SHA256, generally. So for example, you might use 5c15a6e5c0bf02e6c0eaa939cb543c41d7725453064c920b9a4faeea7c357506. (This is a digest hash you'd use in Docker, as an example) You can both consume that specific one, but also reproduce it locally if you want to. Depending on your intent, bandwidth and trust you can pick the method you desire.
        3y ago
k8svet 3y ago
>create a Docker image or a Vagrant box, and whenever you instantiate one of those I'm passing X to doubt. This is just odd analysis. Nix is strictly superior to Docker, except the learning curve. You have to really work at it to make any modern Nix setup non-reproducible.
- oneplane 3y ago
  You can pass X as much as you like, but Nix, like anything else, can be configured with dynamic references. Nix is also not related to Docker (neither superior nor inferior), and the learning curve makes it unsuitable for everything where Docker and Vagrant are suitable, even if it would technically do the same thing. Either way, you are missing the point: the problem here isn't having perfect reproduction. It's about having an environment you can learn in without having to worry about breaking things. That's why the author references vagrant since it's about as low a barrier to entry as disposable environments can be (and even still a barrier to high for some).
  - kreetx 3y ago
    I think it's you who is missing the point: the learning environment won't build as neither docker nor vagrant will give you reproducible environments, while nix will. Sure, you can make your nix environments dynamic too, but having determinism is its core feature. I don't think it is with docker nor vagrant, as the environment you get will be isolated, but not deterministic. You can, of course bolt some custom thing on top of those (or use nix), but that is going to be a pain. EDIT: Both vagrant and docker are useful tools, it's just that for creating any dev environments nix will be better due to determinism.
    - oneplane 3y ago
      The point is, and always was: someone wants a box to experiment in. Nix is not that box. It was never the box. It will never be the box. That is the point I am making and reflects the point the author was making. And just in case there is a language barrier: box does not refer to a specific technical implementation of a box, it's just a term to denote a border between the user's system (which they don't want to break) and "something else". Edit: and just in case the word 'someone' trips over a language barrier, I'm not referring to 'anyone' but to the persona (the 'someone') who might want to try something out because they saw something and thought it was cool. Not someone with package manager experience, not a software engineer, not a sysadmin. We're also not 'building a learning environment'. Learning environment is a proxy for disposable environment, which as a relatively simple concept is already a step too far for the average "how do I become a cool hacker like on TV" case (which is where the unreasonable effectiveness from the title comes in) where someone might take their first steps and wanting to try something out without breaking their current environment.
deleted 3y ago
[deleted]

nunez 3y ago

You can avoid that unpredictability if you don't use the latest tag, or, better, create images or start containers from SHA256 digests. Those are guaranteed to be immutable. I.e instead of: Docker run --rm debian:trixie You'd do Docker run --rm debian:trixie@sha256:0ee9224bb31d3622e84d15260cf759bf37f08d652cc0411ca7b1f785f87ac19c The only disadvantage of the digest approach is that you would need to manually resolve the digest that's correct for your processor arch. Using bare tags like "debian:trixie" can resolve to manifest lists (if so configured) that has Docker automatically find the right digest for your arch.

kreetx 3y ago
What if the environment needs more packages installed via apt, do you save those hashes somewhere, too?
- codethief 3y ago
  Debian & Ubuntu provide daily snapshots of their repositories, see snapshot.debian.org and https://ubuntu.com/blog/ubuntu-snapshots-on-azure-ensuring-predictability-and-consistency-in-cloud-deployments https://ubuntu.com/blog/ubuntu-snapshots-on-azure-ensuring-p... See e.g. https://github.com/reproducible-containers/repro-sources-list.sh https://github.com/reproducible-containers/repro-sources-lis... for how to use them. Also, at least for Debian there are official images available that use snapshot.debian.org as package repository from the get-go – unfortunately those images are not published on a daily basis yet. It seems the overall topic if reproducibility is on the Docker team's to do list, though, see https://github.com/docker-library/official-images/issues/16044 https://github.com/docker-library/official-images/issues/160...
  - kreetx 3y ago
    But if you read those links, you'll see that it's not quite there yet. Docker in general can't solve reproducibility - it's the package manager within any container that does that.
    - 3y ago

hiAndrewQuinn 3y ago

I deem that you are correct! I have investigated NixOS in a Vagrant + Virtualbox combo very similar to what I did in this post, and yes, this is a much more deterministic solution to this idea. If the people you are writing a tutorial for can be expected to be familiar with Nix, this is probably a much more future-proofed solution. That's still a small camp of people at the moment, but it does include the creator of Vagrant himself, Mitchell Hashimoto. I maintain that a specific Debian version is a better "lingua franca" for a general audience - that may change in coming years, of course. I have begun writing a new tutorial called "NixOS: Three Big Ideas" to expose people like myself who haven't given Nix the time of day to explore its concepts in about 20 minutes or so.