5 ms·
I've seen countless attempts to replace "docker build" and Dockerfile. They often want to give tighter control to the build, sometimes tightly binding to a pack
by bmitch3020 7mo ago
I've seen countless attempts to replace "docker build" and Dockerfile. They often want to give tighter control to the build, sometimes tightly binding to a package manager. But the Dockerfile has continued because of its flexibility. Starting from a known filesystem/distribution, copying some files in, and then running arbitrary commands within that filesystem mirrored so nicely what operations has been doing for a long time. And as ugly as that flexibility is, I think it will remain the dominant solution for quite a while longer.
- whateveracct 7mo agoNix is exceptionally good at making docker containers.
- mikepurvis 7mo agoEspecially if you use nix2container to take control over the layer construction and caching.
- Spivak 7mo agoYes but then you're committed to using Nix which doesn't work so well the moment you need some software not packaged by Nix. Want to throw a requirements.txt in there? No no, why would you even ask that? Meanwhile docker says yeah sure just run pip install, why should I care?
- okso 7mo agoLLMs are getting very good at packaging software using Nix.
- CuriouslyC 7mo agoThis. I wouldn't have touched Nix when you needed someone who was really good at Nix to keep it working, but agents make it viable to use in a number of place.
- mort96 7mo agoThen you're committing to maintaining a package for that software. Like all LLM boosters, you've ignored the fact that the largest time sink in many kinds of software is not initial development, but perpetual maintenance.
- xyzzy_plugh 7mo agoIt's not materially any different from maintaining lines in a Dockerfile.
- mort96 7mo agoIt is mateirially different compared to "maintaining" the line 'RUN apt-get -y install foobar'
- SOLAR_FIELDS 7mo agoIs it though? If the way that I’m going to edit those files is by typing the same natural language command into Claude code, and the edit operation to maintain it takes 20 seconds instead of 10, to me that seems pretty materially the same
- mort96 7mo agoYes, it is
- SOLAR_FIELDS 7mo agoHow so?
- whateveracct 7mo agoI use software from pretty much every language with Nix. And I package it myself too when needed. Including Python often :)
- nothrabannosir 7mo agoNix doesn't make sense if all you're going to use it for is building Docker images. It only makes sense if you're all in in the first place. Then Docker images are free.
- gnull 7mo agoPackaging for nix is exceptionally easy once you learn it. And once something is packaged, it's solved for all, it's not going to randomly break. If you care about getting it to work with minimal effort right now more thar about it being sustainable later, then sure.
- hamandcheese 7mo agoI don't in ow if I'd say it's "easy". The Python ecosystem in particular is quite hard to get working in a hermetic way (Nix or otherwise). Multiple attempts at getting Python easy to package with Nix have come and gone over the years.
- saghm 7mo ago> Packaging for nix is exceptionally easy once you learn it Most of the complaints I've seen about Nix about around documentation, so "once you learn it" might be the larger issue.
- ghthor 7mo agoPacking software with nix is easier than any other system TBH and just seems to be just getting easier.
- stabbles 7mo agoDoes Nix do one layer per dependency? Does it run into >=128 layers issues? In Spack [1] we do one layer per package; it's appealing, but I never checked if besides the layer limit it's actually bad for performance when doing filesystem operations. [1] https://spack.readthedocs.io/en/latest/containers.html https://spack.readthedocs.io/en/latest/containers.html
- hamandcheese 7mo agoThis post has a great overview: https://grahamc.com/blog/nix-and-layered-docker-images/ https://grahamc.com/blog/nix-and-layered-docker-images/ tl;dr it will put one package per layer as much as possible, and compress everything else into the final layer. It uses the dependency graph to implement a reasonable heuristic for what is fine grained and what get combined.
- ghthor 7mo agoThat layering algorithm is also configurable, though I couldn’t really understand how to configure it and just wrote my own post processing to optimize layering for my internal use case. I believe I can open source this w/o much work. The layer layout is just a json file so it can be post processed w/o issue before passing to the nix docker builders
- zbentley 7mo ago> the Dockerfile has continued because of its flexibility I wish we had standardized on something other than shell commands, though. Puppet or terraform or something more declarative would have been such a better alternative to “everyone cargo cults ‘RUN apt-get upgrade’ onto the top of their dockerfiles”. Like, the layer/stage/caching behavior is fine. I just wish the actual execution parts had been standardized using something at a higher level of abstraction than shell.
- avsm 7mo agoDocker broke out the build layer into a separate component called BuildKit (see HN discussion recently https://news.ycombinator.com/item?id=47166264 https://news.ycombinator.com/item?id=47166264). However, Dockerfiles are so popular because they run shell commands and permit 'socially' extending someone else shell commands; tacking commands onto the end of someone else's shell script is a natural process. /bin/sh is unreasonably effective at doing anything you need to a filesystem, and if the shell exposes a feature, it has probably been used in a Dockerfile somewhere. Every other solution, especially declarative ones, tend to come up short when _layering_ images quickly and easily. However, I agree they're good if you control the entire declarative spec.
- mihaelm 7mo agoI'd say LLB is the "standard", Dockerfile is just one of human-friendly frontends, but you can always make one yourself or use an alternative. For example, Dagger uses BuildKit directly for building its containers instead of going through a Dockerfile.
- bheadmaster 7mo ago> Puppet or terraform or something more declarative would have been such a better alternative Until you need to do something that isn't covered with its DSL, and you extend it with an external command execution declaration... At which point people will just write bash scripts anyway and use your declarative language as a glorified exec.
- sofixa 7mo ago
- miladyincontrol 7mo agoThe lack of docker registry-like solutions really does seem to be the chokepoint for many alternatives. Personally I love using mkosi and while it has all the composability and deployment options I'd care for, its clear not everyone wants to build starting only with a blank set of OS templates.
- nunez 7mo agoThere are lots of alternative container image registries: quay, harbor, docker's open sourced one, the cloud providers, github, etc. Or do you mean a replacement for docker hub?
- __MatrixMan__ 7mo agoThere are some hurdles preventing that flow from achieving reproducible builds. As the bad guys get more sophisticated, it's going to become more and more important that one party can say "we trust this build hash" and a separate party to say "us too". That's not going to work if both parties get different hashes when they build the image, which won't happen as long as file modification timestamps (and other such hazards) are part of what gets hashed.
- bmitch3020 7mo agoRecent versions of buildkit have added support for SOURCE_DATE_EPOC. I've been making the images reproducible before that with my own tooling, regctl image mod [1] to backdate the timestamps. It's not just the timestamps you need to worry about. Tar needs to be consistent with the uid vs username, gzip compression depends on implementations and settings, and the json encoding can vary by implementation. And all this assumes the commands being run are reproducible themselves. One issue I encountered there was how alpine tracks their package install state from apk, which is a tar file that includes timestamps. There are also timestamps in logs. Not to mention installing packages needs to pin those package versions. All of this is hard, and the Dockerfile didn't make it easy, but it is possible. With the right tools installed, reproducing my own images has a documented process [2]. [1]: https://regclient.org/cli/regctl/image/mod/ https://regclient.org/cli/regctl/image/mod/ [2]: https://regclient.org/install/#reproducible-builds https://regclient.org/install/#reproducible-builds
- hsbauauvhabzb 7mo agoDoes any of that matter if you’re not auditing the packages you install? I’m more concerned about sources being poisoned over the build processes. Xz is a great example of this.
- __MatrixMan__ 7mo agoBoth are needed, but you get more bang for your buck focusing on build security than on audited sources. If the build is solid then it forces attackers to work in the open where all auditors can work together towards spoiling the attack. If you flip it around and instead have magically audited source but a shaky build, then perhaps a diligent user can protect themself, but they do so by doing their own builds, which means they're unaware that the attack even exists. This allows the attacker to just keep trying until they compromise somebody who is less diligent. Getting caught requires a user who analyses downloaded binaries in something like ghidra... who does that when it's much easier to just build them from sources instead? (answer: vanishingly few people). And even once the attacker is found out, they can just hide the same payload a bit differently, and the scanners will stop finding it again. Also, "maybe the code itself is malicious" can only ever be solved the hard way, whereas we have a reasonable hope of someday providing an easy solution to the "maybe the build is malicious" problem.
- phplovesong 7mo agoYou can pretty much replace "docker build" with "go build". But as long as people want to use scripting languages (like php, python etc) i guess docker is the neccessary evil.
- speedgoose 7mo agoIt doesn't sound like Golang is going to dominate and replace everything else, so Docker is there to stay.
- osigurdson 7mo agoIn some situations, yes, others no. For instance if you want to control memory or cpu using a container makes sense (unless you want to use cgroups directly). Also if running Kubernetes a container is needed.
- matrss 7mo agoYou have to differentiate container images, and "runtime" containers. You can have the former without the latter, and vice versa. They are entirely orthogonal things. E.g. systemd exposes a lot of resource control as well as sandboxing options, to the point that I would argue that systemd services can be very similar to "traditional" runtime containers, without any image involved.
- osigurdson 7mo agoWell, I did mention "or use cgroups" above.
- matrss 7mo agoAnd what I've said is that there are more options. You don't have to use cgroups directly, there are other tools abstracting over them (e.g. systemd) that aren't also container runtimes.
- aobdev 7mo agoWasn’t this the same argument for .jar files?
- kccqzy 7mo ago> But the Dockerfile has continued because of its flexibility. The flip side is that the world still hasn’t settled on a language-neutral build tool that works for all languages. Therefore we resort to running arbitrary commands to invoke language-specific package managers. In an alternate timeline where everyone uses Nix or Bazel or some such, docker build would be laughed out of the window.
- brightball 7mo agoReminds me of the “Electric cars in reverse” video where the guy envisions a world where all vehicles are electric and tries to make the argument for gas engines.
- anal_reactor 7mo agoLink?
- fittom 7mo agotry searching for 'Rory Sutherland: What If Petrol Cars Were Invented In 2025'
- brightbeige 7mo agoActual article was in the evening standard, but like all things Rory Sutherland, it’s worth to watch him tell the story: https://youtu.be/OTOKws45kCo?si=jbTdx3YCGkZv3Akb https://youtu.be/OTOKws45kCo?si=jbTdx3YCGkZv3Akb For those who want more of him, check out his classic TED talk from decades ago: “Lessons from an ad man” https://www.ted.com/talks/rory_sutherland_life_lessons_from_an_ad_man https://www.ted.com/talks/rory_sutherland_life_lessons_from_...
- muvlon 7mo agoAs a Nix evangelist, I have to say: Nix is really not capable of replacing languag-specific package managers. > running arbitrary commands to invoke language-specific package managers. This is exactly what we do in Nix. You see this everywhere in nixpkgs. What sets apart Nix from docker is not that it works well at a finer granularity, i.e. source-file-level, but that it has real hermeticity and thus reliable caching. That is, we also run arbitrary commands, but they don't get to talk to the internet and thus don't get to e.g. `apt update`. In a Dockerfile, you can `apt update` all you want, and this makes the build layer cache a very leaky abstraction. This is merely an annoyance when working on an individual container build but would be a complete dealbreaker at linux-distro-scale, which is what Nix operates at.
- osigurdson 7mo agoI'm not sure if this is what you mean but in some ways it would be nice to have tighter coupling with a registry. Docker build is kind of like a multiplexer - pull from here or there and build locally, then tag and push somewhere else. Most of the time all pulls are from public registries, push to a single private one and the local image is never used at all. It seems overly orthogonal for the typical use case but perhaps just not enough of an annoyance for anyone to change it.
- alex_dev42 7mo ago[dead]
- Ameo 7mo ago^ this account has posted nothing but AI generated spam since it was created 6 hours ago