12 ms·
Nix is a better Docker image builder than Docker's image builder
- anarchogeek 3y agoDid they ever figure out how to run nix on top of MacOS? When I see a project uses Nix I know the creators aren't serious. It's kind of like seeing something is written in Lisp or Scheme... somebody decided that functional systems and running code aren't worth it. Instead they'll build a system which is perfect in all ways in theory but nobody can or will ever use.
- pxc 3y agoNix has run on macOS for more than a decade.
- msteffen 3y agoReally random, but the illustrator who made the slide art for this (Annie Ruygt) also made logos and brand art for two YC startups, RethinkDB (rethinkdb.com) and Pachyderm (pachyderm.io) (where I worked, and which was founded by former RethinkDB engineers). She does great work!
- deleted 3y ago[deleted]
- madjam002 3y agoDoes anyone here have any experience using https://github.com/pdtpartners/nix-snapshotter https://github.com/pdtpartners/nix-snapshotter ? I build a lot of Docker images using Nix, and while yes it’s generally more pleasant than using Dockerfiles, the 128 layer limit is really annoying and easy to hit when you start building images with Nix. The workaround of grouping store paths makes poor use of storage and bandwidth.
- mikepurvis 3y agoI haven't tried it yet as I need to produce containers that can work on public cloud k8s, but it definitely looks like the way to go. All the existing methods for grouping store paths into layers are finnicky, brittle, and non-optimal.
- hinshun 3y agoAuthor of nix-snapshotter here. Yes, one of the main downsides of Docker images using Nix is the 128 layer limit. It means we have to use a heuristic to combine packages into the same layer and losing Nix’s package granularity. When building containers with Nix packages already on a Nix binary cache you also have to transform the Nix packages into layer tarballs effectively doubling the storage requirements. Nix-snapshotter brings native understanding of Nix packages to the container ecosystem so the runtime prepares the container root filesystem directly from the Nix store. This means docker pull == Nix substitution and also at Nix package granularity. It goes a bit further with Kubernetes integration that you can read about in the repo. Let me know if you have any other questions!
- k8svet 3y agoWhat's the state of deployment for something like nix-snapshotter nowadays (with the realization that the answer depends on which of N k8s install methods might be in use)? I assume it's mostly in the field of ... "you're making a semi-large investment on this enough that you're doing semi-custom kubernetes deployments with custom containerd?" Or maybe the thought is nix-snapshotter users are running k8s/kubelet with nixos anyway so its not a big deal to swapout/add containerd config?
- hinshun 3y agoYes it’s going to depend on which k8s distribution you’re using. We have work in-progress for k3s to natively support nix-snapshotter: https://github.com/k3s-io/k3s/pull/9319 https://github.com/k3s-io/k3s/pull/9319 For other distributions, nix-snapshotter works with official containerd releases so it’s just a matter of toml configuration and a systemd unit for nix-snapshotter. We run Kubernetes outside of NixOS, but yes the NixOS modules provided by the nix-snapshotter certainly make it simple.
- k8svet 3y agoSorry to bug you with more questions but I literally dreamed of nix-snapshotter for years, so I'm excited. Do you know how this (installation burden) translates in the real world (GKS? AKS? etc)? Can one even get away with abusing DaemonSets/hostDir/privileged on hosted clusters to modify their own installations? Or is `nix-snapshotter` just sort of out of the question on those provided solutions?
- mikepurvis 3y agoNo discussion about Nix-built containers is complete without mentioning nix2container: https://github.com/nlewo/nix2container https://github.com/nlewo/nix2container It is truly magical for handling large, multi-layered containers. Instead of building the container archives themselves and storing them in the nix store, it builds a JSON manifest that is consumed by a lightly patched version of skopeo that streams the layers directly to either your local container engine or the registry. This means you never rebuild or reupload a container layer that is unchanged. Disclosure: I contributed a change in nix2container that allows cheaply pulling non-Nix layers into the build, just using the content hashes from their registry manifests.
- Nullabillity 3y agoNixpkgs' streamDockerImage does something similar, instead of storing a multi-layer tarball it produces a script that cats them all together on demand, ready to feed into `docker load` or whatever.
- takeda 3y agoIn that case streamDockerImage produces script that you run then you pipe output to skopeo or docker. nix2container wraps all of that and it automatically runs it in behind the scenes when you call nix run. The whole image generation is much more efficient as well. In the standard streamDockerImage you get a script that generates a docker layers and the image. In nix2container all layers are stored in nix cache so subsequent run doesn't regenerate them. I believe that was the main goal behind this solution. Another benefit (and I think it is even better than the caching) is that it also allows you manually specify what dependencies go to each layer. The automatic way that the code offers, is nice in theory, but because docker has a limit of 128 layers, in practice it starts nicely then the last layer is a clump of remaining dependencies that didn't fit in previous layers. With nix2container I managed for example to create the first layer which contains just Python and all of its dependencies, then the next layer are my application dependencies (Python packages) the last layer is my application. With this approach a simple bugfix in the application only replaces the last layer that's the size of a few kb. Only a change of Python version will rebuild the whole thing.
- tetris11 3y agoGuix is also pretty good at this, only lacking up to date packages that one would want to build an image with
- djaouen 3y agoThis is my one gripe with Guix. Alas, we (apparently) can't have it all!
- Zambyte 3y agoIt is very easy to overload package versions locally for your needs, and it's quite easy to push that upstream to Guix so that others may benefit as well :)
- djaouen 3y agoCan you explain (or point to an explanation) of exactly how to do that? The Guix versions of Erlang and Elixir are way out of date, and I would like to push a fix.
- Zambyte 3y agoSee here[0] for pushing a fix, here[1] for the anatomy of a package definition (often you only need to bump the version number and update the hash, but compilers may be a bit more involved). It may be useful to define package variants[2], which is what I do for some packages locally. You can also see this page[3] for using ad-hoc package variants using command line flags. Hope this helps :) [0] https://guix.gnu.org/manual/devel/en/html_node/Contributing.html https://guix.gnu.org/manual/devel/en/html_node/Contributing.... [1] https://guix.gnu.org/manual/devel/en/html_node/Defining-Packages.html https://guix.gnu.org/manual/devel/en/html_node/Defining-Pack... [2] https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html https://guix.gnu.org/manual/devel/en/html_node/Defining-Pack... [3] https://guix.gnu.org/manual/devel/en/html_node/Package-Transformation-Options.html https://guix.gnu.org/manual/devel/en/html_node/Package-Trans...
- 3y ago
- operator-name 3y agoThis is great if you've already adopted Nix, and I'd love for nothing than more declarative package management solutions like Nix or Guix to take off. If you're already using Docker but want to gradually adopt Nix, there is an alternative approach outlined by this talk: https://youtu.be/l17oRkhgqHE https://youtu.be/l17oRkhgqHE. Instead of migrating both the configuration AND container building to Nix straight away, you can keep the Dockerfile to build the nix configuration. The biggest downside is that you don't take advantage of layers at all, but the upside is that you can gradually adapt your Dockerfiles, and reuse any Docker infrastructure or automation you already use.
- FiberBundle 3y ago[1] also uses this approach. [1] https://mitchellh.com/writing/nix-with-dockerfiles https://mitchellh.com/writing/nix-with-dockerfiles
- AtlasBarfed 3y agoSo one of the pillars of the article is that docker builds aren't reproducible, but Nix is. But... is a lot of that irreproducibiity (apologies for that word) because there's no guarantee one of the docker layers will be available? And... does Nix have some guarantee to the end of the universe that package versions will stay in the repository?
- operator-name 3y agoI'd give this article a read, as it can explain it more clearly than I can: https://serokell.io/blog/what-is-nix https://serokell.io/blog/what-is-nix But to briefly answer your specific questions: Docker files are commonly not reproducible because they contain arbitary stateful commands like `apt-get update`, `curl`, etc. For a layer with these kinds of commands to be reproducible you would need a mechanism to version and verify the result. Nix provides such a mechanism, and a community package repository with versioned dependancies between packages. These are defined in a domain specific language called Nix (text files) and kept into a git repository. This should be familiar if you've used a package manager with lock files before. You can guarentee the package version will stay in the repository by pinning your build to an exact commit hash in the repository.
- debuggerpk 3y agohorrible, horrible font on the website!
- mmh0000 3y agoWhat a weird thing to complain about. Just override the font if you dislike it that much. That's the great thing about the interwebs, it's all just text, you can format it however you want. Assuming you're using something based off chrome: https://chromewebstore.google.com/detail/font-changer/obgkjikcnonokgaiablbenkgjcdbknna https://chromewebstore.google.com/detail/font-changer/obgkji... Or Firefox: https://addons.mozilla.org/en-US/firefox/addon/refont/ https://addons.mozilla.org/en-US/firefox/addon/refont/
- eichin 3y agoIt's actually a smooth readable (but tall-and-skinny) font on my personal laptop, and a pixelated mess on my work laptop; I've never figured out why... Oh, huh, they're using a custom font: https://xeiaso.net/blog/iaso-fonts/ https://xeiaso.net/blog/iaso-fonts/ so that should actually be consistent. (After the weekend I'll poke at it on the other laptop and see what's up.) What I'm getting at is that they might (unintentionally) be complaining about a rendering bug and not the actual font...
- djaouen 3y agoI just wanted to chime in here and say that Guix also has a nice and easy-to-use Docker option with "guix pack -f docker" [1]. Guix also has the advantage of using an already-used language (Guile/Scheme) rather than its own bespoke one. :) [1] https://guix.gnu.org/manual/en/html_node/Invoking-guix-pack.html https://guix.gnu.org/manual/en/html_node/Invoking-guix-pack....
- Nonoyesnoyes 3y agoThe article lost me somewhere. Long intro and than just assuming too much. That's just possible due to nix being more granular? Is that right? Can I really build nix from 5 years ago? No src gone? No cache server gone? Nothing? I mean yeah Ubuntu as a base is shitty.
- xena 3y agoThe text is written to be spoken, it works better when I present it. I'll have the video edited next week, that may flow better for you.
- Nonoyesnoyes 3y agoI will check it out then. Would you answer my assumption? Is this easier with nix because it's more granular? Like if I have a cacerts alone I can select it? And can I really build stuff from 5 years ago?
- chaxor 3y agoI don't like Ubuntu or Debian as a base image for docker, but it's typically my go-to if I need glibc stuff or browser emulation. Is there a better alternative like Alpine but with glibc that isn't Debian?
- imp0cat 3y agoI think Debian is a solid choice, mostly everything is either present or available. Image sizes can get out of hand tho.
- j-bos 3y agoI like the article but had a hard time following the specifics of the configs and commamds. Feels like it's more meant for people already familiar with nix, or sufficiently interested to study up while reading
- earthling8118 3y agoThat's very interesting, because as someone familiar with nix my take was that this was information I considered to be aimed at people who weren't familiar.
- febed 3y agoI didn’t fully grok how this works - what is the base image for the generated image? Also wouldn’t the image size be large if the glibc is copied over again
- aidenn0 3y ago> what is the base image for the generated image? Default is none (i.e. like "FROM scratch" in a Dockerfile); you can specify a baseImage if needed, but I haven't had to yet. It works by copying parts of the nix store into the image as needed, but see also below. > wouldn’t the image size be large if the glibc is copied over again The original Nix docker-tools buildImage did suffer from poor reuse of common dependencies. Docker already has a way to reuse parts of images (e.g. if you build 7 images where the first N lines of a Dockerfile are the same, the 7 images will use a shared store for the results of running the first N lines). There are several backends for Docker storage that accomplish this in various ways (e.g. FS overlays, tricks with ZFS/btrfs snapshots). Nix docker-tools now has a "buildLayeredImage" that uses this ability of Docker to share much of the storage for the dependencies, so if you build several images that all rely on glibc, you only pay the cost of storing glibc in docker once.
- febed 3y agoThanks, that made the article clearer for me
- xlii 3y agoI spent last 2-3 days trying to get Docker images built on Darwin and I feel that this article is a universe making fun of me. Nix is absolutely the best tool for what I want to achieve but it has those dark forsaken corners that just suck your soul out dry. I love it but sometimes it feels like being a Morty on Rick’s adventure to the compilerland.
- amouat 3y agoI'm using this quote: "I love it but sometimes it feels like being a Morty on Rick’s adventure to the compilerland."
- renewiltord 3y agoI use Orbstack and it works flawlessly to do this. Really good tool. I use Docker to cross-compile for {aarch64,amd64} x {linux,darwin} since not all the cross-compiling is super robust across our stacks (I'm using a specific glibc for one Linux part, etc.). Just a bunch of docker on my Darwin aarch64 and it compiles everything. Good experience.
- e40 3y agoI installed Orbstack and found that I didn't really need it, so I removed the directory in /Applications. Wow, for weeks and weeks I found remnants of it in a lot of places. Very disappointing that it left so much cruft around. They should have an uninstaller. It left a really bad taste and I'm unlikely to try it again. Before someone asks. I've been using macOS for a long time. I've never seen remnants like this from a program. Sure, there are often directories left in ~/Library/Application Support/, but this was more than that. Unfortunately, I didn't write down the details, but I ran across the bits in at least 3-4 places.
- cqqxo4zV46cp 3y agoI’ve found this to be the norm for ‘Docker Desktop alternatives’. Not to say that Orbstack isn’t uniquely messy.
- 3y ago
- whazor 3y agoThis blog post is missing the reasoning on why shared docker layers are useful. It is because of caching. The more images are sharing the same layers the better, as it allows you to cache more stuff. Better caching means faster startup of containers. Why is docker bad at this? In order to enjoy the caching benefit, each time you build a docker image you want it to output as much existing layers as possible. So running apt-get install python3 today should result in the exact same layer as yesterday, if there are no new updates. But this requires the all the files to be exactly the same, including the metadata like creation time. As docker layers are cached by hashing the files. Now, Nix already does storing dependencies by hash. So the layers will always be the same with the same version and same configuration.
- m1keil 3y agoIn Docker, if the layers are cached, the layer with apt-get won't be automatically invalidated (unless --no-cache or any changes to the upper layers).
- raffraffraff 3y agoBut that's what I would expect to happen. I don't see a problem.
- eichin 3y agoWon't get invalidated even if what "apt-get install python3" does changes - the cache is only based on the syntax of the RUN string plus the previous layer hash, IIRC. (COPY actually invalidates if the file being copied changes, so maybe there's a way to fetch a hash of the repo and stash it where copy will notice, or something, but then it seems you need external tooling to do that bit?)
- m1keil 3y agoI didn't claim there is a problem. The original comment made it sound as if docker will expire a cached layer because the (potential) result of apt-get is different, which isn't the case.
- tuananh 3y agoas platform engineer, i want to like nix. but it's not easy for everyone else. and the dx is still pretty bad IMO. for example, i prefer devbox DX just because i can add pkg like this `devbox add python@3.11`. also, looking at 120 lines flake.nix. it's not exactly "easier" https://github.com/Xe/douglas-adams-quotes/blob/main/flake.nix https://github.com/Xe/douglas-adams-quotes/blob/main/flake.n...
- klntsky 3y agoThese 120 lines do quite a lot more, don't they?
- Cyph0n 3y agoThat's kind of an unfair comparison. The flake you linked: 1. Defines a Go binary (i.e., how to build it) 2. Defines a Docker image that uses said Go binary as an entry point 3. Defines a NixOS module that creates a systemd service that runs the Go binary (only relevant on NixOS) 4. Defines a NixOS test for the module that ensures that the NixOS module actually creates a systemd service that runs the Go binary as expected. The NixOS test framework is actually quite impressive - tests run in a QEMU VM that also runs NixOS :) Note that only (1) and (2) are relevant to the linked article (+ some of the surrounding boilerplate).
- hamandcheese 3y agoI agree that it's not a fair comparison, but I will add that this is a big barrier to newcomers. Everyone experienced with nix builds their own ivory tower of a nix flake (myself included), so it's hard to find actually good examples of how to do basic things without wading through a bunch of other bullshit.
- Cyph0n 3y agoI'm a beginner myself and have been trying my best to keep things simple. But I do agree that the complexity creep is quite tempting with Nix. Not sure why though..
- jrockway 3y agoI spent a half a day or so relatively recently trying to build our CI base image with Nix (at the recommendation of our infra team), but it was huge, and some stuff didn't work because of linking issues. One issue that really bugged me was to build multi-arch images, it actually wants to execute stuff as the other architecture, and only supports using qemu with hardware virtualization for that. My build machine (and workstation) is a VM, so I don't have that. I do have binfmt-misc, though, so if you just happened to fork and exec the arm64 "mkdir" to "mkdir /tmp", it would have worked. Of course, this implementation is a travesty when docker layers are just tar files, and you can make the directory like this: echo "tmp uid=0 gid=0 time=0 mode=0755 type=dir" | bsdtar -cf - @- (As an aside, I'm sure this exact layer already exists somewhere. So users probably don't even have to download it.) Every time I try nix, I feel like it's just a few months away from being something I'd use regularly. nixpkgs has a lot of packages, everything you could ever want. They all install OK onto my workstation. But "I need bash, python, build-essential, and Bazel" doesn't seem like something they're targeting the docker image builder at. I guess people just want to put their go binary in a docker image and ... you don't need nix for that. Pull distroless, stick your application in a tar file, and there's your container. (I personally use `rules_oci` with Bazel... but that's all it does behind the scenes. It just has some smarts about knowing how to build different binaries for different architectures and assembling and image index yaml file to push to your registry.)
- viraptor 3y ago> to build multi-arch images, it actually wants to execute stuff as the other architecture You should be able to cross compile binaries for other architectures without actually running them. As long as the package's build files support it of course. > and only supports using qemu with hardware virtualization for that That doesn't sound right. You can use qemu for architectures that be only software emulated too. The minimal example is discussed here: https://discourse.nixos.org/t/how-do-i-get-a-shell-nix-with-cross-compiler-and-qemu/7658 https://discourse.nixos.org/t/how-do-i-get-a-shell-nix-with-... I don't want to say it should be as simple as using pkgCross (https://nix.dev/tutorials/cross-compilation.html https://nix.dev/tutorials/cross-compilation.html), but... are some specific issues with the usual process that you're running into?
- 3y ago
- denysvitali 3y agoUnfortunately the result of a Nix Docker image is an image that is 100+ MB for no particular reason :(
- TeeMassive 3y agoThere are ways to properly build a nix container image so this kind of things doesn't happen. You'll find plenty of projects on GitHub dedicated to only that.
- denysvitali 3y agoCan you please provide an example? Everything I've tried ended up being way bigger than I think it should
- l0b0 3y agoWhere on Earth are you getting that result from? [1] gives an 11 MB image. [1] https://nix.dev/tutorials/nixos/building-and-running-docker-images.html#build-your-first-container https://nix.dev/tutorials/nixos/building-and-running-docker-...
- denysvitali 3y agoFrom experience - but I'm more than happy to be proven wrong as I would love to build all the Docker images with Nix. What you linked is the equivalent of: FROM scratch COPY hello-world /hello-world Of course that's small (hello-world is statically linked). Try to add coreutils (or any other small package) and you'll see what I mean. In my experience the size of a Docker image built with some nix packages is greater than the Debian counterpart. I don't know why though.
- georgyo 3y agoNo, that dockerfile is not equivalent because the hello-world is not statically built in the nix version. However I'll give you that it could be smaller in more complex examples. For example glibcLocales is for all locales which is quite chunky but your application only needs one locale.
- Thaxll 3y agoI keep reading about Nix and I still don't understand what it does better than Docker, all the example in the post are trivial to do in a Dockerfile so where is the added value? Docker build are deterministic and easily reproductible, you use a tagged image, that's it, it set in stone. The 0.01% of Dockerfile that don't work, what does it even means, what does not work? The other thing is about that buildGoModule module so now you need somehow a third party tool to use or build Go in a Docker image, when using a Dockerfile you just use regular Go commands such as go build and you know exactly what is going on and what args you use to build the binary. As for the thing about using Ubuntu 18 which is out of date and not finding it, most orgs have docker image cache especially since docker hub closed access to large downloads, but more importantly there is a reason that's its not there anymore, it's not secure to use it, it's like wanting to use the JVM 6, you should not use something that is out of date security wise.
- TeeMassive 3y agoDocker builds are not deterministic, I don't get where you get that idea. I can't count the hours lost because the last guy who left one year ago built the image using duck tape and sed commands everywhere. The image is set in stone, but so is a zip file, there's nothing special here. Building an image using nix solves many problems regarding not only reproducible environments that can be tested outside a container but also fully horizontal dependency management where each dependency gets a layer that's not stacked on one another like a typical apt/npm/cargo/pip command. And I don't have to reverse engineer the world just to see what files changed in the filesystem since everything has its place and has a systematic BOM.
- jhanoncomm 3y agoSo is it right, to make docker reproducible it needs to either build dependencies from source from say a git hash or use other package managers that are reproducible or rely on base images that are reproducible. And that all relies on discipline. Just like using a dynamically typed programming language can in theory have no type errors at run time, if you are careful enough.
- jossephus01 3y agoMy experience with building Docker images for Java applications using Nix wasn't very pleasant though. After the deprecation of gradle2nix, there doesn't seem to be a clear alternative method for building Docker images for Gradle-based Java applications. I challenged a friend to create the smallest possible Docker image for a simple Spring Boot application some time ago. While I was using Nix, the resulting image was twice the size of the image built without Nix. You can check out the code for yourself here: https://github.com/jossephus/Docker_challenge/blob/main/flake.nix https://github.com/jossephus/Docker_challenge/blob/main/flak... .
- tadfisher 3y agoThat's because you're including two JDKs, zulu and the one that gradle includes via its jdk argument. Look for gradleGen in nixpkgs to see what I mean. And sorry for gradle2nix, I'm working on an improvement that's less of a hack.
- jossephus01 3y agoThanks tadfisher, I will check it out. This is by no means meant to be a dunk on gradle2nix. Love your work on android-nixpkgs and I will be looking for the alternative. Thanks.
- rapnie 3y ago> And sorry for gradle2nix, I'm working on an improvement that's less of a hack. Don't be. Thanks for your work. Excited to learn about the improvement. Can you tell more about what you have in mind?
- tripdout 3y agoHey, also wanted to thank you for android-nixpkgs - it's great
- yjftsjthsd-h 3y ago> While I was using Nix, the resulting image was twice the size of the image built without Nix. I would be very interested to know where the difference is; is nix including things it doesn't need to? Is the non-nix build not including things it should?
- paulddraper 3y agoI really like Bazel (rules_oci) for building containers. It's the way building containers ""should"" be. Here's my base image, here's my files, here's my command, write the into an image.
- takeda 3y agoThat's pretty much how building on nix works, except you don't need base image or your application file, you specify what command to run from which package and it will be placed in the container with all runtime dependencies automatically. Of course you can customize the container further if needed.
- paulddraper 3y agoYes, similar idea. With the objective of reproducible software. (Also, you don't need a base image for rules_oci either; most people choose to start with one.)
- kmarc 3y agoAs many bazel rules, rules_oci's predecessor (rules_docker) was an unmaintained spagetti of hell, now we are pushed to rules_oci, and it's rpmtree recommended way of installing rpms which then in turn don't support post install script... All this bazel-is-our-savior complex burns down when we want to build a tiny bit complicated thing with it. And we unfortunately do try to do that (our devbox image), which with full caching takes long minutes to an hour, and it's a freaking thousands pine mess instead of using a lined dockerfioe with pinned versions. I absolutely hate bazel and its broken unmaintained Google-abandoned rulesets and I wish we either used either Docker or Buck2 for everything.
- paulddraper 3y ago> unmaintained spagetti of hell, rules_docker was fundamentally flawed -- and overreaching in scope -- in ways that rules_oci is not. > broken unmaintained Google-abandoned rulesets rules_oci last commit was 12 hours ago, and it's been actively maintained for years. (This criticism is even weirder from someone pushing Buck2. Like, it's a great tool. But, apparently it warranted a complete rewrite too, eh?) > instead of using a lined dockerfioe with pinned versions You'll pin every transitive version? > All this bazel-is-our-savior complex burns down when we want to build a tiny bit complicated thing with it. Let me rephrase. I would not use Bazel to build images from rpm or deb packages. But....would you use Nix for installing rpm or deb packages????? I believe you've lost the thread.
- solatic 3y agoHaving the author do this for a service written in Go is a mistake. Your first address for containerizing Go services should be ko: https://ko.build/ https://ko.build/ , and similar solutions like Jib in the Java ecosystem: https://github.com/GoogleContainerTools/jib https://github.com/GoogleContainerTools/jib . No need to require everyone to install something heavy like Nix, no need for privileged containers in CI to connect to a Docker daemon so that actual commands can be executed to determine filesystem contents, just the absolute bare minimum of a manifest defining a base layer + the compiled artifacts copied into the tarball at the correct positions. More languages should support this kind of model - when you see that pnpm's recipe (https://pnpm.io/docker https://pnpm.io/docker), ultimately, is to pick a pre-existing node base image, then copy artifacts in and set some manifest settings, there's really no technical reason why something like "pnpm build-container-image", without a dependency on a Docker daemon, hasn't been implemented yet. Using nix, or Dockerfile, or similar systems are, today, fundamentally additional complications to support building containerized systems that are not pure Go or pure Java etc. So we should stop recommending them as the default.
- kstenerud 3y agoI've tried again and again to like Nix, but at this point I have to throw in the towel. I have 2 systems running Nix, and I'm afraid to touch them. I've already broken both of them enough that I had to reinstall from scratch in the past (yes yes - it's supposed to be impossible I know), and now I've forgotten most of it. In theory, Nix is idempotent and deterministic, but the problem is "deterministic in what way?" Unless you intimately understand what every dependent part is doing, you're going to get strange results and absolutely bizarre and unhelpful errors (or far more likely: nothing at all, with no feedback). Nix feels more like alchemy than science. Like trying to get random Lisp packages to play nice together. Documentation is just plain AWFUL (as in: complete and technically accurate, but maddeningly obtuse), and tutorials only get you part of the way. The moment you step off the 80% path, you're in for a world of hurt, because the underlying components are just not built to support anything else. Sure, you can always "build your own", but this requires years of experiential knowledge and layers upon layers of frustration that I just don't want to deal with anymore (which is also why I left Gentoo all those years ago). And woe unto you if you want to use a more modern version than the distribution supports! The strength of Docker is the chaos itself. You can easily build pretty much anything, without needing much more than a cursory understanding of the shell and your distro's package manager. Or you can mix and match whatever the hell you want! When things break, it's MUCH easier to diagnose and fix the problems because all of the tooling has been around for decades, which makes it mature enough to handle edge cases (and breakage is almost ALWAYS about edge cases). Nix is more like Emacs: It can do absolutely anything if you have the patience for it and the deep, arcane knowledge to keep it from exploding in a brilliant flash of octarine. You either go full-in and drink the kool aid, or you keep it at arm's length - smiling and nodding as you back slowly towards the door whenever an enthusiast speaks.
- janjongboom 3y agoI've gone down the same path. I love deterministic builds, and I think Docker's biggest fault is that to the average developer a Dockerfile _looks_ deterministic - and it even is for a while (build a container twice in a row on the same machine => same output), but then packages get updated in the package manager, base images get updated w/ the same tag, and when you rebuild a month later you get something completely different. Do that times 40 (the number of containers my team manages) and now fixing containers is a significant part of your job. So in theory Nix would be perfect. But it's not, because it's so different. Get a tool from a vendor => won't work on Nix. Get an error => impossible to quickly find a solution on the web. Anyway, out of that frustration I've funded https://www.stablebuild.com https://www.stablebuild.com. Deterministic builds w/ Docker, but with containers built on Ubuntu, Debian or Alpine. Currently consists of an immutable Docker Hub pull-through cache, full daily copies of the Ubuntu/Debian/Alpine package registries, full daily copies of most popular PPAs, daily copies of the PyPI index (we do a lot of ML), and arbitrary immutable file/URL cache. So far it's been the best of both worlds in my day job: easy to write, easy to debug, wide software compatibility, and we have seen 0 issues due to non-determinism in containers that we moved over to StableBuild in my day job.
- _cenw 3y agoLittle known (possibly unintended) feature, but you can put the `toplevel` attribute of a nixosSystem into docker image `contents`, which lets you use NixOS modules to set things up. Just be sure to import the minimal preset, because those images get large. Unfortunately booting the entire system with /init is largely broken, especially without --privileged. This would be an amazing feature if it didn't require so much extra tinkering.
- yencabulator 3y agoIf you skip the docker and use systemd-nspawn automated by NixOS, that's just `containers.foo = { autoStart=true; ...; config = {config,pkgs}: { just another nixos config here }; };`
- andrewstuart 3y agoI like nspawn over docker because it doesn't use the layered file system thing. Instead, it's just a simple root directory placed somewhere and you run the container on that. Much more straightforward.
- conradludgate 3y agoI wanted to love nix. It seems like something I would like. I tried to compile rust using nix on my mac. Didn't work, known bug. I reinstalled my desktop to use nixos. I got lost between flakes, nixpkgs, homemanager. I managed to get vscode installed but when I added the nix extension (declared in nix) it would refuse to run vscode... It's just not a good experience so I reinstalled arch
- ildjarn 3y agoWhat the post misses is that lots of packages are not available on Nix, but everything is available on Docker automatically. If what you need is available, however, then it can be so much better.
- georgyo 3y agoWhat is automatic about docker? Do you mean other people have already put in the work? Or do you mean that it's more trivial to pip/npm/cargo install stuff?
- pluto_modadic 3y agoNix as package management, yes - you're waiting on someone else to make upstream work. Where Debian/PIP/NPM/Cargo already do work. Nix as an instruction set, no, you still have to declare what you want, same as a dockerfile. it's just... Docker will do what a shell script will. Nix is like hoping someone's done the upstream finagling to get your particular dependencies happy.
- ildjarn 3y agoIf something works in a distro, it automatically works in a Dockerfile based on that distro. There is no extra packaging effort required.
- woile 3y agoI've been using nix to build docker containers (from a Mac). I would like to skip docker as well, but I wouldn't know how. On the server, I use docker swarm, with traefik as load balancer, in a very small machine, which I can later grow. It works pretty well for me. Nix on the CI has never fail for anything but mistakes of my own.
- benreesman 3y agoNix and NixOS are in something like the state git was in before GitHub: the fundamental idea is based on more serious computer science than the status quo (SVN, Docker), the plumbing still has some issues but isn’t worse, and the porcelain and docs are just not there for mainstream adoption. I think that might have changed with the release of flox: https://flox.dev https://flox.dev, it’s basically seamless (and that’s not surprising coming from DE Shaw). Nix doesn’t really make sense without flakes and nix-command, those things are documented as experimental and defaulted off. The documentation story is getting better, but it’s not there. nixlang is pretty cool once you learn it well, but it’s never going to be an acceptable barrier to entry at the mainstream. It’s not really the package manager it’s advertised as, nix-env -iA foo is basically never what you want. It’s completely unsurprising that it’s still a secret weapon of companies with an appetite for bleeding-edge shit that requires in-house expertise. flox addresses all of this for the “try it out and immediately have a better time” barrier. Nix/NixOS or something like it is going to send Docker to the same dustbin Subversion is in now, but that’s not going to happen until it has the GitHub moment, and then it’ll happen all at once. Most of the complaints with Nix in this thread are technically false, but eminently understandable and more importantly (repeat after me Nix folks): it’s never the users fault. I’m aware that I’m part of a shrinking cohort who ever knew a world without git/GitHub, so I know this probably sounds crazy to a large part of the readership, but listen to Linus explaining to a room full of people who passed the early Google hiring bar why they should care about a tool they feel is too complicated for them: https://youtu.be/MjIPv8a0hU8?si=QC0UnHXRdMpp2tI4 https://youtu.be/MjIPv8a0hU8?si=QC0UnHXRdMpp2tI4
- rfoo 3y agoI believe for a developer tool to success, for the most common thing to do there has to be at least three ways an engineer may misuse your tool and still get it "done" (by leaving non-obvious tech debts behind). This is true for git, but not so true yet for Nix, so I'm not sure a GitHub-like moment helps.
- benreesman 3y agoIn a full-metal-jacket NixOS setting it’s bloody hard to bash (no pun intended) your way through to the next screen by leaving behind tech debt (Python comes to mind, I made the mistake of trying to use Nix to manage Python once, never again). But anywhere else you just brew/apt/rpm install whatever and nix develop —impure, which is easier than most intermediate git stuff and plenty of beginner stuff. git and Nix are almost the same data structure if you start poking around in .git or /nix/store, I might not understand what you mean without examples. But all my guesses about what you might mean are addressed well by flox.
- torcete 3y agoCoincidentally, Two days ago I was trying to adapt a flake to include a docker derivation. I came across with xelaso's page and inspired by the example provided (and after a few tries) I manage to compose a docker image. That was very cool! BTW: Thanks Xelaso.
- verdverm 3y agoI've been using Dagger, it's awesome. It's the second take by the creators of Docker. It accomplishes most of what nix does for this problem Write your pipelines and more in languages you already use. It unlocks the advanced features of BuildKit, like the layer graph and advanced caching. nit: this post makes a point about deterministic builds and then uses "latest" for source image tags, which is not deterministic. I've always appreciated Kelsey's comment that "latest" is not a version
- mdaniel 3y agoSince there are a plethora of dagger projects, lazyweb: https://github.com/dagger/dagger#readme https://github.com/dagger/dagger#readme They also recently released their "github actions" replacement <https://news.ycombinator.com/item?id=39550431 https://news.ycombinator.com/item?id=39550431> but holy hell their documentation is just aggressively bad
- shykes 3y ago> holy hell their documentation is just aggressively bad Anything in particular you'd like to see improved in the docs?
- mdaniel 3y agoThe fact that I couldn't point to one page on the docs that shows the tl;dr or the what problem is this solving https://docs.dagger.io/quickstart/562821/hello https://docs.dagger.io/quickstart/562821/hello just emits "Hello, world!" which is fantastic if you're writing a programming language but less helpful if you're trying to replace a CI/CD pipeline. Then, https://docs.dagger.io/quickstart/292472/arguments https://docs.dagger.io/quickstart/292472/arguments doubles down on that fallacy by going whole hog into "if you need printf in your pipline, dagger's got your back". The subsequent pages have a lot of english with little concrete examples of what's being shown. I summarized my complaint in the linked thread as "less cowsay in the examples" but to be honest there are upteen bazillion GitHub Actions out in the world, not the very least of which your GHA pipelines use some https://github.com/dagger/dagger/blob/v0.10.2/.github/workflows/engine-and-cli.yml#L60-L63 https://github.com/dagger/dagger/blob/v0.10.2/.github/workfl... https://github.com/dagger/dagger/blob/v0.10.2/.github/workflows/engine-and-cli.yml#L70-L71 https://github.com/dagger/dagger/blob/v0.10.2/.github/workfl... so demonstrate to a potential user how they'd run any such pipeline in dagger, locally, or in Jenkins, or whatever by leveraging reusable CI functions that setup go or run trivy Related to that, I was going to say "try incorporating some of the dagger that builds dagger" but while digging up an example, it seems that dagger doesn't make use of the functions yet <https://github.com/dagger/dagger/tree/v0.10.2/ci#readme https://github.com/dagger/dagger/tree/v0.10.2/ci#readme> which is made worse by the perpetual reference to them as their internal codename of Zenith. So, even if it's not invoked by CI yet, pointing to a WIP PR or branch or something to give folks who have CI/CD problems in their head something concrete to map into how GHA or GitLabCI or Jenkins or something would go a long way
- qxxx 3y agoI tried nix (for spinning up dev environments). It is very slow compared to docker.
- cue_the_strings 3y agoNix falls into a category where it's incompatible enough with upstream that it requires significant packaging effort, for most libraries. This is especially so when you have complex C or C++ libs with imperfect Cmake and Autotools build scripts or janky Makefiles. Wrapping everything for Nix becomes a significant portion of the work. Frequently it's not packaged or some version was but the upstream build system changed an you have to do it yourself. If you don't either benefit significantly from Nix's upsides (which most projects really don't), or you don't enjoy this and do it for fun, it's not a productive use for your time. If you can convince your employer to pay you to shave that yak, it's a net waste.