11 ms·
When I first heard about Docker, I thought it did what Nix does. Then someone handed me a docker image saying, "I got this working on my laptop, deploy this. I
by rdtsc 10y ago
When I first heard about Docker, I thought it did what Nix does.
Then someone handed me a docker image saying, "I got this working on my laptop, deploy this. Isn't this great!". I had no idea how they arrived at that configuration. If something happened to them or that image, I wouldn't have known how to reproduce that "working state".
So I am happy to see Nix / Guix become popular. It is what I imagined package management to be.
Before that we just used rpm and deb packages. For all the hate they get, they are actually well thought out, tested, stable format. With {pre/post}-{install/uninstall} script support, dependency resolution and so on.
But Nix / Guix is a qualitatively different things. Hope it becomes popular.
- colemickens 10y agoI mean, that's exactly what a Dockerfile is for. I don't know of anyone seriously using Docker that isn't using Dockerfiles exclusively. I only know of a single time when it's valuable to "commit" a new image (recovering logs from a stopped container). None of which is to detract from Nix. Nix+Docker is a powerful combination. It takes Docker's declarative nature all the way down to the compiler used to build the bits placed in the container.
- hso1 10y agoThere's a post on building Docker images with Nix: https://news.ycombinator.com/item?id=11502989 https://news.ycombinator.com/item?id=11502989
- OJFord 10y agoI sort of agree, but I also think there's no reason to view Docker commits any differently to Git commits. Are you screwed if you lose a working .git project, or its maintainer? Well, yeah. But we don't actively worry about that happening all that much because we routinely take precautions like distributing it across developer's machines and servers. Maybe a commit does break, but that's why we didn't just overwrite the previous image. But also: > I only know of a single time when it's valuable to > "commit" a new image (recovering logs from a stopped > container) Not everything's open source.
- colemickens 10y agoIt doesn't have anything to do with a commit breaking anything. It has to do with reproducability. As others have mentioned, a Dockerfile isn't perfect (I'd argue Dockerfile+Nix is pretty close) but at least you don't have to ask "Oh god, what did OJ Ford run manually before commiting the image we use in Prod." That's truly horrific and simply isn't a problem if you use a Dockerfile. >Not everything's open source. I don't know what that means. Whether the source is closed or not has zero impact on the use-cases I'm discussing for Docker. Even if the source of the base layer isn't available, it doesn't stop you from using that base layer in a new Dockerfile, at all.
- benley 10y agoThe big difference is that nothing about Dockerfiles implies that the processes they perform to build an image are deterministic or repeatable. Will a build step that runs `curl https://github.com/something/whatever` https://github.com/something/whatever` do the same thing in six months that it does today? Docker doesn't help with that at all. Nix improves determinism by (almost) guaranteeing that if you build the same nix expression six months from now, you'll get exactly the same dependencies, all the way down to glibc. Either that or the build will just fail, if remote dependencies aren't downloadable anymore, which is always tons of fun.
- pacala 10y agoIn practice, one can check the Dockerfile lineage of a given Docker image and assess the quality of the commands. Good practices are to use package managers with explicit versions, for example "apt-get install subversion-tools=1.3.2-5~bpo1". Or, if you so desire, just pipe NixOS outputs into Docker, as somebody else in this thread mentioned.
- thejosh 10y agoYES. Finally someone understands what really annoys me when I say that docker isn't 100% reproducible if you aren't using version pinning or something similar. If you have 2 developers, and one of them does a build the next day, you could have them with two different versions of a package when the version went up.
- koverstreet 10y agoThat isn't even the bad part - the bad part is that give some big complicated docker image (or any build in general!), you really have no practical way of knowing what's different. That should scare people.
- mgkimsal 10y agosounds like 'emerge hell' from gentoo 12 years ago.
- 10y ago
- cstrahan 10y agoAs another use points out, there was some relevant discussion earlier on HN about using Nix+Docker: https://news.ycombinator.com/item?id=11502989 https://news.ycombinator.com/item?id=11502989 Copying a comment I left on that blog post: --- Q: "What does building a a Docker image using Nix give you over creating a regular Dockerfile to build a Docker image?" A: * Better abstraction (e.g. the example of a function that produces docker images) * The Hydra build/CI server obviates the need for paying for (or administering a self hosted) docker registry, and avoids the imperative push and pull model. Because a docker image is just another Nix package, you get distributed building, caching and signing for free. * Because Nix caches intermediate packages builds, building a Docker image via Nix will likely be faster than letting Docker do it. * Determinism. With Docker, you're not guaranteed that you'll build the same image across two machines (imagine the state of package repositories changing -- it's trivial to find different versions of packages across two builds of the same Dockerfile). With Nix, you're guaranteed that you have the same determinism that any other Nix package has (e.g. everything builds in a chroot without network access (unless you provide a hash of the result, for e.g. tarball downloads)) --- IMO, NixOS almost subsumes Docker. Where Docker gives you a way to guarantee that the contents of an image will be the same across two machines, it does little to ensure that building that image from a given Dockerfile is a deterministic. One of Docker's neat tricks is using layers to try to save on disk util -- but even then, Nix beats Docker; if you start two different containers using Nix packages, the common packages are shared across both containers, whereas two different Docker images with an uncommon base would not share the common files. Docker mostly looks like someone set out to answer two questions: 1. How can we work around all of the problems inherent in conventional package management so we can have a smidgen of determinism? 2. How can we have a relatively nice/cohesive UI around launching/inspecting containers and managing networking/bind-mounts? #1 can be side-stepped entirely with a package manager that doesn't cause all of those problems in the first place (Nix/Guix). #2 is still quite useful, but would have been better served as layering on top of something like Nix or Guix (not that I really approve of the "wannabe systemd" daemon approach that Docker employs (all while being, ironically, rather difficult to use with systemd: https://lwn.net/Articles/676831/ https://lwn.net/Articles/676831/ )). Edit: grammar
- zenlikethat 10y agoTangential, but if you want to recover logs from a stopped container, why not use 'docker cp'?
- zwischenzug 10y agoThere's a little company called 'RedHat'... https://github.com/openshift/source-to-image https://github.com/openshift/source-to-image
- geocar 10y ago> I wouldn't have known how to reproduce that "working state". So what? You don't need to reproduce that working state: The work has already been done! Think about how the interactive approach Lisp and Smalltalk programmers use is superior to the compile/run/rerun approach used by C++ and Java programmers -- by exploring the problem in a nonlinear way, you can find your way to a solution more quickly. Docker package management is like that: You make some changes, you commit often, fork your instance and try lots of different things. Eventually you find something that works and you tag it. You share your tags with other people. This is an extremely productive way to get work done.
- oconnore 10y agoWorking with Lisp images: it's horrible when you lose track of what runtime change allows your code to continue working. You typically keep a working compilation between code and your intended environment, and use the run time to experiment in between changes.
- DigitalJack 10y agoI think this all the time. I have to refresh my clojure repl because I forget what I've eval'd or not and the cognitive load of mentally tracking that state is not something I want to bother with.
- geocar 10y agoClojure is unusually leaky compared to Unix/Linux or (for example) Erlang. If we design systems out of a collection of processes, that we can replace at will, then what we actually have is a collection of checkpoints and branches that we can try simultaneously. Replacing individual branches allows us to try out changes there, without having to rebuild (and wait) the entire system. Another way to think about this, is that Clojure is more like subversion than like git: It might be better than RCS, but you're still stuck with a linear revision history, and new changes can't completely conflict with old objects and old changes that are created. And yet another way, is to imagine all of those old class definitions that old objects hold onto are actually global variables, because they have global (time) extent, and yet are unreachable, so it's difficult to find out whether your new code is being affected by those old settings. Smalltalk, and even other (esp. Commercial) Lisps don't have this problem. It has nothing to do with interactivity, but with hidden state.
- nextos 10y agoAlso people forget that the Nix philosophy doesn't end in the package manager Nix, there's also NixOS which takes a functional approach to configuration management of the whole OS. And many other things.
- zimbatm 10y agoAnd nixops which allows you to build everything locally (or delegated to a build machine) and push configuration to multiple machines.
- drewcrawford 10y agoBut how do you reproduce the dpkg packages? When your friend hands you a deb, instead of a docker image, what have you accomplished as far as knowing how they arrived at their configuration? Debs are also binary blobs, just more of them, and separately. From what I can see, there are two philosophically pure approaches to software packaging. Either we can build everything from source, so that we have reproducibility (see e.g. emerge for a practical system, although it is still not bit-for-bit reproducible) or else we can ship everything as a blob so we have total consistency (e.g. Docker, Go). The other approaches, apt and nix, seem to me to be compromise positions. They may even be practical or useful compromises. But they are ultimately about the degree to which we allow binary blobs, not whether we do. But it seems to me that once you have accepted installing binaries via apt then installing them from Docker is a question of degree, not of kind. It seems very hard to me to argue that distributing binaries via Docker is really worse than via nix.
- colemickens 10y agoHm. Can you elaborate on why nix is a compromise? Nix is effectively building from source, but with lots of caching in place to avoid end users constantly having to rebuild every last package from source. Nix and Apt are very different in this regard. >But they are ultimately about the degree to which we allow binary blobs, not whether we do. I don't agree with this assesment of Nix and Nix packages at all. My local system evaluates the package definition (a description of how to build it from source) and merely is smart enough to retrieve a prebuilt binary with the exact same source inputs and configuration. Sure, I'm still trusting the remote party to have built it as described, but that's true of any system where pre-built binaries are distributed. For another explanation, I could disable the NixOS Hydra Cache and could theoretically rebuild my current system state without their binary cache and still wind up with the exact same bits. At least, that's the idea behind Nix.
- pmahoney 10y ago> Either we can build everything from source, so that we have reproducibility (see e.g. emerge for a practical system, although it is still not bit-for-bit reproducible) or else we can ship everything as a blob so we have total consistency (e.g. Docker, Go). Nix is really both of these combined, and is not like apt/dpkg in how it separates source and binary packages. A Nix package is really instructions (including all compiler flags) of how to build something from source [1] and nothing more. Due to being pure functions, given the binary inputs (e.g. a zlib dependency previously built) and the instructions, any two people will always get the same binary output upon executing those instructions [2]. Furthermore, without executing the instructions, I can generate the unique identifier of the resulting binary package by hashing the inputs and the instructions. This unique identifier can be used to determine if the package already exists locally in /nix/store, and optionally be used to check if some trusted binary cache (such as Nix's binary cache) already has the binary package. There is no binary package in the sense that apt/dpkg have source packages and many derived binary packages which are installed without need of the source package. For example (contrived and simplified): gcc-uniqueGccHash = bootstrappingProblem() libc-uniqueLibcHash = buildLibc(gcc-uniqueGccHash) zlib-uniqueZlibHash = buildZlib(gcc..., libc...) myprog-uniqueMyProgHash = buildMyProg(gcc..., libc..., zlib...) To download "myprog" from a binary cache, I can't calculate uniqueMyProgHash using any old versions of libc and zlib; I must have exactly the same hashes that were used by the system that built "myprog", all the way up the dependency chain. And I'll only ever have that if the builds are reproducible and/or I downloaded them all from the same binary cache. [1] In practice, there's nothing stopping the instructions from saying "download this binary blob" instead of "download this source tarball". [2] The problem of build reproduciblity is not solved, but many outside of Nix also see this as important, see https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds