3 ms·

> But the Dockerfile has continued because of its flexibility. The flip side is that the world still hasn’t settled on a language-neutral build tool that works

by kccqzy 8mo ago

> But the Dockerfile has continued because of its flexibility. The flip side is that the world still hasn’t settled on a language-neutral build tool that works for all languages. Therefore we resort to running arbitrary commands to invoke language-specific package managers. In an alternate timeline where everyone uses Nix or Bazel or some such, docker build would be laughed out of the window.

brightball 8mo ago
Reminds me of the “Electric cars in reverse” video where the guy envisions a world where all vehicles are electric and tries to make the argument for gas engines.
- anal_reactor 8mo ago
  Link?
  - fittom 8mo ago
    try searching for 'Rory Sutherland: What If Petrol Cars Were Invented In 2025'
    - brightbeige 8mo ago
      Actual article was in the evening standard, but like all things Rory Sutherland, it’s worth to watch him tell the story: https://youtu.be/OTOKws45kCo?si=jbTdx3YCGkZv3Akb https://youtu.be/OTOKws45kCo?si=jbTdx3YCGkZv3Akb For those who want more of him, check out his classic TED talk from decades ago: “Lessons from an ad man” https://www.ted.com/talks/rory_sutherland_life_lessons_from_an_ad_man https://www.ted.com/talks/rory_sutherland_life_lessons_from_...
muvlon 8mo ago
As a Nix evangelist, I have to say: Nix is really not capable of replacing languag-specific package managers. > running arbitrary commands to invoke language-specific package managers. This is exactly what we do in Nix. You see this everywhere in nixpkgs. What sets apart Nix from docker is not that it works well at a finer granularity, i.e. source-file-level, but that it has real hermeticity and thus reliable caching. That is, we also run arbitrary commands, but they don't get to talk to the internet and thus don't get to e.g. `apt update`. In a Dockerfile, you can `apt update` all you want, and this makes the build layer cache a very leaky abstraction. This is merely an annoyance when working on an individual container build but would be a complete dealbreaker at linux-distro-scale, which is what Nix operates at.
- kccqzy 8mo ago
  Fundamentally speaking, the key point is really just hermeticity and reliable caching. Running arbitrary commands is never the problem anyways. What makes gcc a blessed command but the compiler for my own language an "arbitrary" command anyways? And in languages with insufficient abstraction power like C and Go, you often need to invoke a code generation tool to generate the sources; that's an extremely arbitrary command. These are just non-problems if you have hermetic builds and reliable caching.
  - xhcuvuvyc 8mo ago
    I mean, I guess at a theoretical level. In practice, it's just not a large problem.
- poly2it 8mo ago
  Well, arbitrary granularity is possible with Nix, but the build systems of today simply do not utilise it. I've for example written an experimental C build system for Nix which handles all compiler orchestration and it works great, you get minimal recompilations and free distributed builds. It would be awesome if something like this was actually available for major languages (Rust?). Let me know if you're working on or have seen anything like this!
  - vatsachak 8mo ago
    crane https://crane.dev/getting-started.html https://crane.dev/getting-started.html
    - poly2it 8mo ago
      I use crane, but it does not have arbitrary granularity. The end goal would be something which handled all builds in Nix.
  - nh2 8mo ago
    A problem with that is that Nix is slow. On my nixos-rebuild, building a simple config file for in /etc takes much longer than a common gcc invocation to compile a C file. I suspect that is due to something in Nix's Linux sandbox setup being slow, or at least I remember some issue discussions around that; I think the worst part of that got improved but it's still quite slow today. Because of that, it's much faster to do N build steps inside 1 nix build sandbox, than the other way around. Another issue is that some programming languages have build systems that are better than the "oneshot" compilation used by most programming languages (one compiler invocation per file producing one object file, e.g. ` gcc x.c x.o`). For example, Haskell has `ghc --make` which compiles the whole project in one compiler invocation, with very smart recompilation avoidance (pet-function, comment changes don't affect compilation, etc) and avoidance of repeat steps (e.g. parsing/deserialising inputs to a module's compilation only once and keeping them in memory) and compiler startup cost. Combining that with per-file general-purpose hermetic build systems is difficult and currently not implemented anywhere as far as I can tell. To get something similar with Nix, the language-specific build system would have to invoke Nix in a very fine-grained way, e.g. to get "avoidance of codegen if only a comment changed", Nix would have to be invoked at each of the parser/desugar/codegen parts of the compiler. I guess a solution to that is to make the oneshot mode much faster by better serialisation caching.
_the_inflator 8mo ago
There is some truth to it, however in production it is simple: There is a working deployment or not. Therefore I would rephrase your remarks as upside: let others continue scratch their head while others deploy working code to PROD. I am glad there is a solution like Docker - with all it flaws. Nothing is flawless, there is always just yet another sub-optimal solution weighting out the others by a large margin.
- kccqzy 8mo ago
  Popularity of a technology usually isn’t perfectly correlated with how good it is. > let others continue scratch their head while others deploy working code to PROD. You make it sound like when docker build arrived on the scene, a cross-language hermetic build tool was still a research project. That’s just untrue.
whurley23 8mo ago
This calls for https://xkcd.com/927/ https://xkcd.com/927/.

by kccqzy 8mo ago

brightball 8mo ago

Reminds me of the “Electric cars in reverse” video where the guy envisions a world where all vehicles are electric and tries to make the argument for gas engines.

anal_reactor 8mo ago
Link?
- fittom 8mo ago
  try searching for 'Rory Sutherland: What If Petrol Cars Were Invented In 2025'
  - brightbeige 8mo ago
    Actual article was in the evening standard, but like all things Rory Sutherland, it’s worth to watch him tell the story: https://youtu.be/OTOKws45kCo?si=jbTdx3YCGkZv3Akb https://youtu.be/OTOKws45kCo?si=jbTdx3YCGkZv3Akb For those who want more of him, check out his classic TED talk from decades ago: “Lessons from an ad man” https://www.ted.com/talks/rory_sutherland_life_lessons_from_an_ad_man https://www.ted.com/talks/rory_sutherland_life_lessons_from_...

muvlon 8mo ago

As a Nix evangelist, I have to say: Nix is really not capable of replacing languag-specific package managers. > running arbitrary commands to invoke language-specific package managers. This is exactly what we do in Nix. You see this everywhere in nixpkgs. What sets apart Nix from docker is not that it works well at a finer granularity, i.e. source-file-level, but that it has real hermeticity and thus reliable caching. That is, we also run arbitrary commands, but they don't get to talk to the internet and thus don't get to e.g. `apt update`. In a Dockerfile, you can `apt update` all you want, and this makes the build layer cache a very leaky abstraction. This is merely an annoyance when working on an individual container build but would be a complete dealbreaker at linux-distro-scale, which is what Nix operates at.

kccqzy 8mo ago
Fundamentally speaking, the key point is really just hermeticity and reliable caching. Running arbitrary commands is never the problem anyways. What makes gcc a blessed command but the compiler for my own language an "arbitrary" command anyways? And in languages with insufficient abstraction power like C and Go, you often need to invoke a code generation tool to generate the sources; that's an extremely arbitrary command. These are just non-problems if you have hermetic builds and reliable caching.
- xhcuvuvyc 8mo ago
  I mean, I guess at a theoretical level. In practice, it's just not a large problem.
poly2it 8mo ago
Well, arbitrary granularity is possible with Nix, but the build systems of today simply do not utilise it. I've for example written an experimental C build system for Nix which handles all compiler orchestration and it works great, you get minimal recompilations and free distributed builds. It would be awesome if something like this was actually available for major languages (Rust?). Let me know if you're working on or have seen anything like this!
- vatsachak 8mo ago
  crane https://crane.dev/getting-started.html https://crane.dev/getting-started.html
  - poly2it 8mo ago
    I use crane, but it does not have arbitrary granularity. The end goal would be something which handled all builds in Nix.
- nh2 8mo ago
  A problem with that is that Nix is slow. On my nixos-rebuild, building a simple config file for in /etc takes much longer than a common gcc invocation to compile a C file. I suspect that is due to something in Nix's Linux sandbox setup being slow, or at least I remember some issue discussions around that; I think the worst part of that got improved but it's still quite slow today. Because of that, it's much faster to do N build steps inside 1 nix build sandbox, than the other way around. Another issue is that some programming languages have build systems that are better than the "oneshot" compilation used by most programming languages (one compiler invocation per file producing one object file, e.g. ` gcc x.c x.o`). For example, Haskell has `ghc --make` which compiles the whole project in one compiler invocation, with very smart recompilation avoidance (pet-function, comment changes don't affect compilation, etc) and avoidance of repeat steps (e.g. parsing/deserialising inputs to a module's compilation only once and keeping them in memory) and compiler startup cost. Combining that with per-file general-purpose hermetic build systems is difficult and currently not implemented anywhere as far as I can tell. To get something similar with Nix, the language-specific build system would have to invoke Nix in a very fine-grained way, e.g. to get "avoidance of codegen if only a comment changed", Nix would have to be invoked at each of the parser/desugar/codegen parts of the compiler. I guess a solution to that is to make the oneshot mode much faster by better serialisation caching.

_the_inflator 8mo ago

There is some truth to it, however in production it is simple: There is a working deployment or not. Therefore I would rephrase your remarks as upside: let others continue scratch their head while others deploy working code to PROD. I am glad there is a solution like Docker - with all it flaws. Nothing is flawless, there is always just yet another sub-optimal solution weighting out the others by a large margin.

kccqzy 8mo ago
Popularity of a technology usually isn’t perfectly correlated with how good it is. > let others continue scratch their head while others deploy working code to PROD. You make it sound like when docker build arrived on the scene, a cross-language hermetic build tool was still a research project. That’s just untrue.

whurley23 8mo ago

This calls for https://xkcd.com/927/ https://xkcd.com/927/.