12 ms·

Nixery – Docker images on the fly with Nix

201 points by _fnqu 4y ago

audiodude 4y ago
This is amazing because it takes the promise of "caching Docker layers", which doesn't usually work out well in practice, and actually delivers on it.
spockz 4y ago
This looks really nice, it saves making the Dockerfile yourself. And just sometimes you want an image but with a bit of extra in it for debugging. If normally you pull that image directly with this you don’t need to setup a build to build and push the custom image only to scuttle it again later. What is interesting though is that nix is all about reproducible builds but I don’t see a way to specific packaged versions here.
- clhodapp 4y ago
  I belive that if you host your own nixery instance, you can pass a nixpkgs commit hash as a docker tag
exdsq 4y ago
Oh great! More indirection. Now when I want to deploy my web app I can check my private nixery.dev deployment is properly configured in Nix to build my Docker images so I can deploy my containers to the cloud so someone can access me Rest API. And the cost of guaranteeing builds will probably work? Running your own nixery service, learning Nix, and learning Docker. I would love someone to do a cost-benefit analysis of these sorts of tools against the time of using (and sometimes debugging) Make and/or Bash. I’m so cynical of Nix lol - no disrespect to the OP/Author, I just want to work on problems but most of my time is spent on building and deploying stuff.
- hamandcheese 4y ago
  Do you actually use nix and have an experience to share?
  - WJW 4y ago
    Not OP, but the evangelism about "never having build problems anymore" does perplex me a bit. In the languages I have been programming in (Haskell, Ruby, some Python and elm and JS and Rust), I can't recall having any significant build problems in the last ~8 years or so anyway. What does everyone do that their build keeps breaking?
    - steeleduncan 4y ago
      Those languages all have sensible package managers built in, nix adds more value with languages like C/C++, or when you have multiple binaries interacting. For example, I had a C++ project break catastrophically when upgrading from Ubuntu 19.10 to 20.04. It built fine, but wouldn't boot. Undoubtedly the root cause was my fault, but I couldn't trace it and the timing was terrible, so I had a hack to build inside a 19.10 container. Nix would have saved me a lot of pain in that case by pinning the compiler or whatever dependency broke it. Nix is also useful in that it pins the entire ecosystem in the case that your project isn't fully contained in one language. To take your Haskell example, if you use Stack it will pin every piece of Haskell code you might import, but I've had problems where an external tool changes its output, requiring changes to my tool. In that case nix is like Stack, but for your entire OS, building that tool in a reproducible manner.
    - ananthakumaran 4y ago
      Probably you are lucky to not run into the nokogiri compilation issue. For production codebases, usually, this is not an issue because the build breakages are fixed one at a time by upgrading gem, etc. This is a big issue for personal projects, I am no longer able to run many old projects (last commit made before 5 years), because they depend on older library version (like libglew), which is not supported by any recent distros.
    - drewm1980 4y ago
      Pull in a lot of transitive dependencies, without a system in place that automatically and strictly vendors or pins the versions of everything, and the probably of your build succeeding will converge to zero over time. Humans screw up semver all the time even when they're aware of hyrum's law and are doing their very best not to break user code. I would say this starts becoming an issue when you're around 20-30 devs maintaining software that's a few years old. All that's just for rebuilding when their are no changes to your code... pulling in security updates is a whole additional mess if your software is exposed to adversaries.
- d4mi3n 4y ago
  This really isn't too different than what is already done, but it does as you say add more layers of indirection. Consider: Prior to Docker, you'd typically go to AWS or a virtual host provider, spin up your OS of choice, install any relevant system dependencies, language runtimes, set up a CI/CD pipeline, and finally deploy. The only real difference between what I just described and Docker / Nix / additional layers is that we (as an industry/profession) have not yet built sufficiently ergonomic tooling to make this trivial. AWS and similar cloud providers did away with much of the server and network setup. Docker has done away with some of the application environment setup. All that said, Nix does seem to be trying to replace something we already have a workable answer to (host/app config). Whether or not the additional overhead is worthwhile even after ergonomics have caught up will probably depend on your own use-cases. I can see it being useful for high-trust environments (finance, medicine, anything else regulated). It could also do a lot to improve the general security of the OSS ecosystem by giving projects a path forward to truly reproducible binaries. Outside of those contexts, you probably don't care until tooling gets to the point where you can opt-in and get those guarantees "for free".
- unicornmama 4y ago
  Nix makes a lot of sense if you really understand build system. But makes less sense if you understand operations, or management. Real world engineering is about trade-offs, and Nix has no wiggle room for compromise. It’s optimizes on one dimension: Reproducible builds. But an organisation won’t succeed when it places the needs of build system engineers on a pedestal.
  - zimbatm 4y ago
    You can always set `__noChroot = true;` on your derivations and forgo the sandbox. Then it's not more difficult than a Dockerfile really.
  - pjkundert 4y ago
    It adds some effort and complexity to the "build" dimension of your multi-dimension optimization, yes. But it also removes a whole bunch of complexity from every other dimension, by removing variability from the equation. If you cannot rely on /what/ you are running, then what, exactly, are you testing? Do you really know? I've found that most "managers" (in fact, most programmers) don't seem to appreciate this. The "well, I don't know what happened -- maybe reboot the system, and it'll work?" approach is insane.
  - JamesSwift 4y ago
    Id disagree its uni-dimensional. It optimizes for reproducibility and hermeticity without using virtualization (i.e. better performance).
- drewm1980 4y ago
  Is the extra layer of indirection you don't like Docker or this nix -> Docker integration? Compared to just installing all your nix packages in one Docker layer, this does introduce build complexity. But it's analogous to the complexity of a compiler for a static language... the thing that comes out is not more complex than what went in, so at least the complexity doesn't propagate. The images produced by this should be interchangeable with their single-layer counterparts; the caching will just be better when the code is rebuilt and re-distributed. If you're all-in on nix, does the container ecosystem even add value? The author of this software thought so, at least when he posted in 2018: "Tying in to the schedulers, orchestration, and monitoring is very valuable" Note, I have not used this; I'm just also frustrated by software development getting eaten by incidental complexity.
- chriswarbo 4y ago
  > I would love someone to do a cost-benefit analysis of these sorts of tools against the time of using (and sometimes debugging) Make and/or Bash. Nix does essentially the same job as Make. The differences are: - Make embeds a shell code interpreter, whilst Nix just execs a binary; given its path, a list of args and a set of env vars. (Note that almost all Nix definitions use bash as their binary!) - Make does meta-programming with a mixture of "automatic variables" ('$<', '$^', etc.), 'eval', macros, etc. whilst Nix uses a programming language. - Make relies on timestamps to figure out whether to re-use existing outputs; Nix relies on the hash of the definition (this works recursively, since hashes are included in filenames; hence changing a reference will alter all the hashes up the dependency tree). - Make runs commands in the directory where 'make' was invoked, Nix runs commands in a temp folder (and optionally restricts network and filesystem access) - Make runs commands with the same environment it was invoked with, Nix specifies the environment of commands in the build definition Nix also has a killer feature that Make can't do, called "import from derivation". This lets us define a build process, like 'fetch this git repo', then import and use Nix definitions from its result. In comparison, Makefiles can't (reliably) fetch and import each other; e.g. my C project's Makefile can't fetch the GCC source tarball, and depend on its Makefile's "install" rule to provide a compiler. My hypothesis is that this deficiency of Make is the reason for a whole bunch of unneeded complexity in the software world (e.g. "package managers", "OS distributions", "configuration managers", etc.) From a practical point of view, Nix is almost always used as a wrapper on top of something else (Make/Ant/Maven/Cabal/etc.); but that's just because most projects benefit from those "ecosystems". Note that we could just as well wrap such Ant/Maven/Cabal/etc. projects in a layer of Make instead of Nix, but nobody does since it wouldn't give us any benefit ;) If you're happy to ignore those "ecosystems" and just have a simple "bash + Make" project, you could instead have a simple "bash + Nix" project and avoid all the layers of Make/Ant/Maven/Cabal/etc. (as well as any Docker, Ansible, Apt/RPM, etc. that others might also decide to layer on top!)
  - dboreham 4y ago
    Thank you for this post. This is the kind of authoritative, insightful, contextually relevant information that makes HN so valuable.
- goodpoint 4y ago
  > I would love someone to do a cost-benefit analysis of these sorts of tools against the time of using (and sometimes debugging) Make and/or Bash Spot on! But HN is so biased towards new or flashy stuff...
lmeyerov 4y ago
Is there a clean way to do reuse this for multistage builds? ``` FROM nixery.dev/shell/git/node14/python3.8 as debug_extras FROM our/production:1.2.3 COPY --from=debug_extras /nixstuff /ubuntu/stuff RUN python -c "print('nice!')" ```
- hamandcheese 4y ago
  You could copy over the nix store, though your path wouldn’t be set up correctly to find the programs you want.
- JamesSwift 4y ago
  It depends on what you are trying to get out of the builder pattern. I think it wouldn't provide any benefit to you but hard to say without knowing exactly what you want to achieve. e.g. if you want to remove all the stuff that you aren't using then nix already does that with a GC
hamandcheese 4y ago
This is really cool, but I don’t know if I see the appeal for actual nix users — if you are a nix user and have it set up in CI, you can easily build docker images yourself using buildLayeredImage. And then if you aren’t a nix user, why would you use this? Installing packages with, say, apt, is decidedly not where my pains with docker have arose.
- rajivmr 4y ago
  That's correct. I recently ended up using `buildLayerImage` (actually `buildLayerImageWithNixDb`) for CI, not only to run a single process, but also `systemd` and multiple processes. `podman` comes with built-in support for `systemd`. [Here](https://github.com/fdb-rs/fdb/blob/fdb-0.2.2/nix/ci/flake.nix#L34 https://github.com/fdb-rs/fdb/blob/fdb-0.2.2/nix/ci/flake.ni...) is relevant code.
- denvrede 4y ago
  If I need an image with a specific set of tools it's cumbersome to build a whole workflow to build, store and maintain these images. Having a service that can receive a custom list of Nix packages and returns an image that I can instantly use would be really, really, really nice.
  - tazjin 4y ago
    That's what most users of nixery.dev (i.e. the public instance) do, afaict. Ad-hoc images for CI, and for debugging purposes.
    - machinerychorus 4y ago
      yep, this is exactly how I use it. Nixery is so convenient for getting random one-off containers. Thank you for building and hosting it!
nickysielicki 4y ago
I don’t know if it’s just because I hopped on the bandwagon in the past few months, but it really is starting to feel like Nix is gaining momentum. You can use it on Mac, WSL, Ubuntu multiuser, Docker, in an nspawn container, or through NixOS on a drive. You can build a livecd iso or a Raspberry Pi image from the same flake that you use for everything else. I work in embedded with Yocto every day, and I can’t help but think that Nix is going to eat their lunch in the next decade. There’s really never been anything (usable) that’s like Nix. I think it’s inevitable that it takes over everything.
- mateuszf 4y ago
  Not tried that but I've heard it's even possible to build custom Android distros.
  - nickysielicki 4y ago
    I know that there’s a very active matrix channel for Nix on ARM that is mostly dedicated to pinephone/smartphone projects, so I imagine there’s some android tooling there but I don’t think it’s “android” in the AOSP sense.
    - yakak 4y ago
      There is an AOSP/etc dist building on nix project called robotnix. It has little to do with nixOS on Arm, so it is rarely mentioned on that channel.
  - zimbatm 4y ago
    samueldr has been doing a lot of good work in that direction. See https://mobile.nixos.org/ https://mobile.nixos.org/ and https://github.com/samueldr/ https://github.com/samueldr/
    - samueldr 4y ago
      Just noting, using Nix it is also possible to build an actual real deal Android image using Robotnix: - https://github.com/danielfullmer/robotnix/ https://github.com/danielfullmer/robotnix/ This is different from a non-Android Linux on Mobile devices, which is what Mobile NixOS aims to achieve :).
civodul 4y ago
In the same spirit but in the form of a readily-usable command (rather than a service), 'guix pack' can produce application bundles in a reproducible fashion, in the Docker format as well as in other formats: https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-pack.html https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix... Hopefully the smart layering strategy that Nixery uses will eventually make it into 'guix pack'!
- tazjin 4y ago
  > Hopefully the smart layering strategy that Nixery uses will eventually make it into 'guix pack'! I've been meaning to extract it into a more standalone tool that can output a layer distribution, that way it could also be used in things like Nix's `dockerTools.buildLayeredImage`. The main annoyance is that creating the popularity data inside of a build is not easily possible for the entirety of the package set. Still working on that one ... I'm not sure how much Guix internals have diverged since they forked Nix, but if the dependency analysis of store paths can be done the same way then this should also be straightforward to port to Guix.
  - lewo 4y ago
    With https://github.com/nlewo/nix2container https://github.com/nlewo/nix2container, I'm trying to make a more standalone tool. Basically, a Go binary takes a reference graph and produces a JSON file describing a container image. This JSON file is then ingested by a Skopeo fork (it adds a new `transport`) to produce images (to file, registries,...). Currently, it supports the dockerTools layering algorithm and is designed to work with Guix [1] as well;) [1] https://github.com/nlewo/nix2container/blob/065e5b108650ee4c2dc80fe1f4952eff50922e6c/nix/utils.go#L55 https://github.com/nlewo/nix2container/blob/065e5b108650ee4c...
    - tazjin 4y ago
      Ah, I've actually seen this before. Since it's written in Go, you might be able to pretty much copy&paste the Nixery layering strategy into it. I wouldn't mind!
  - 4y ago
stabbles 4y ago
I like how they optimize for layer reuse, but at the same time, nix doesn't really fit docker layer caching because it's better. With a unique prefix per package, you don't need stacking of layers. You can just download a bunch of packages in parallel, extract them in parallel, and finally "merge" the prefixes to get those combined bin/ and lib/ dirs.
- frafra 4y ago
  > You can just download a bunch of packages in parallel, extract them in parallel, and finally "merge" the prefixes to get those combined bin/ and lib/ dirs. docker pull achieves the same result: layers are fetched in parallel, and they are extracted using pgiz (parallel gzip). It just uses a pre-defined order, which does not harm performances, but it is not useful either in case nixery is used.
  - stabbles 4y ago
    The point is not about parallelization, it's that nixery has to optimize for cache reuse, which is an artificial problem created by docker. If you have two layers installing an individual packages like /nix/store/x and /nix/store/y, stacking them as [x, y] and [y, x] would result in the same docker image contents, but docker will generate different hashes.
    - frafra 4y ago
      Thanks for clarify your point. > If you have two layers installing an individual packages like /nix/store/x and /nix/store/y, stacking them as [x, y] and [y, x] would result in the same docker image contents This is an assumption which is valid for nix, but not for most of the package managers. Whenever such assumption can be considered correct, Dockerfiles can achieve similar results using multiple stages, but you would probably need a pre-processor to have a stage for each package. Something like an `INCLUDE` directive could help too: https://github.com/moby/moby/issues/3378 https://github.com/moby/moby/issues/3378.
ilovefood 4y ago
This is absolutely fantastic, I can't count the times where I needed to debug some sidecar/envoy/tls thing or some network connectivity between two different systems and needed a specific tool (nmap, telnet, ..) within a docker container to debug that and couldn't (or didn't want to) rebuild the container with the missing dependency or package. Really hits a sweet spot and might make life a bit easier for me. Thanks for sharing it!
frafra 4y ago
Would it be possible to implement something like similar using different distributions? I am thinking of Fedora with rpm-ostree, for example.

201 points by _fnqu 4y ago

This is amazing because it takes the promise of "caching Docker layers", which doesn't usually work out well in practice, and actually delivers on it.

spockz 4y ago

This looks really nice, it saves making the Dockerfile yourself. And just sometimes you want an image but with a bit of extra in it for debugging. If normally you pull that image directly with this you don’t need to setup a build to build and push the custom image only to scuttle it again later. What is interesting though is that nix is all about reproducible builds but I don’t see a way to specific packaged versions here.

clhodapp 4y ago
I belive that if you host your own nixery instance, you can pass a nixpkgs commit hash as a docker tag

exdsq 4y ago

Oh great! More indirection. Now when I want to deploy my web app I can check my private nixery.dev deployment is properly configured in Nix to build my Docker images so I can deploy my containers to the cloud so someone can access me Rest API. And the cost of guaranteeing builds will probably work? Running your own nixery service, learning Nix, and learning Docker. I would love someone to do a cost-benefit analysis of these sorts of tools against the time of using (and sometimes debugging) Make and/or Bash. I’m so cynical of Nix lol - no disrespect to the OP/Author, I just want to work on problems but most of my time is spent on building and deploying stuff.

hamandcheese 4y ago
Do you actually use nix and have an experience to share?
- WJW 4y ago
  Not OP, but the evangelism about "never having build problems anymore" does perplex me a bit. In the languages I have been programming in (Haskell, Ruby, some Python and elm and JS and Rust), I can't recall having any significant build problems in the last ~8 years or so anyway. What does everyone do that their build keeps breaking?
  - steeleduncan 4y ago
    Those languages all have sensible package managers built in, nix adds more value with languages like C/C++, or when you have multiple binaries interacting. For example, I had a C++ project break catastrophically when upgrading from Ubuntu 19.10 to 20.04. It built fine, but wouldn't boot. Undoubtedly the root cause was my fault, but I couldn't trace it and the timing was terrible, so I had a hack to build inside a 19.10 container. Nix would have saved me a lot of pain in that case by pinning the compiler or whatever dependency broke it. Nix is also useful in that it pins the entire ecosystem in the case that your project isn't fully contained in one language. To take your Haskell example, if you use Stack it will pin every piece of Haskell code you might import, but I've had problems where an external tool changes its output, requiring changes to my tool. In that case nix is like Stack, but for your entire OS, building that tool in a reproducible manner.
  - ananthakumaran 4y ago
    Probably you are lucky to not run into the nokogiri compilation issue. For production codebases, usually, this is not an issue because the build breakages are fixed one at a time by upgrading gem, etc. This is a big issue for personal projects, I am no longer able to run many old projects (last commit made before 5 years), because they depend on older library version (like libglew), which is not supported by any recent distros.
  - drewm1980 4y ago
    Pull in a lot of transitive dependencies, without a system in place that automatically and strictly vendors or pins the versions of everything, and the probably of your build succeeding will converge to zero over time. Humans screw up semver all the time even when they're aware of hyrum's law and are doing their very best not to break user code. I would say this starts becoming an issue when you're around 20-30 devs maintaining software that's a few years old. All that's just for rebuilding when their are no changes to your code... pulling in security updates is a whole additional mess if your software is exposed to adversaries.
d4mi3n 4y ago
This really isn't too different than what is already done, but it does as you say add more layers of indirection. Consider: Prior to Docker, you'd typically go to AWS or a virtual host provider, spin up your OS of choice, install any relevant system dependencies, language runtimes, set up a CI/CD pipeline, and finally deploy. The only real difference between what I just described and Docker / Nix / additional layers is that we (as an industry/profession) have not yet built sufficiently ergonomic tooling to make this trivial. AWS and similar cloud providers did away with much of the server and network setup. Docker has done away with some of the application environment setup. All that said, Nix does seem to be trying to replace something we already have a workable answer to (host/app config). Whether or not the additional overhead is worthwhile even after ergonomics have caught up will probably depend on your own use-cases. I can see it being useful for high-trust environments (finance, medicine, anything else regulated). It could also do a lot to improve the general security of the OSS ecosystem by giving projects a path forward to truly reproducible binaries. Outside of those contexts, you probably don't care until tooling gets to the point where you can opt-in and get those guarantees "for free".
unicornmama 4y ago
Nix makes a lot of sense if you really understand build system. But makes less sense if you understand operations, or management. Real world engineering is about trade-offs, and Nix has no wiggle room for compromise. It’s optimizes on one dimension: Reproducible builds. But an organisation won’t succeed when it places the needs of build system engineers on a pedestal.
- zimbatm 4y ago
  You can always set `__noChroot = true;` on your derivations and forgo the sandbox. Then it's not more difficult than a Dockerfile really.
- pjkundert 4y ago
  It adds some effort and complexity to the "build" dimension of your multi-dimension optimization, yes. But it also removes a whole bunch of complexity from every other dimension, by removing variability from the equation. If you cannot rely on /what/ you are running, then what, exactly, are you testing? Do you really know? I've found that most "managers" (in fact, most programmers) don't seem to appreciate this. The "well, I don't know what happened -- maybe reboot the system, and it'll work?" approach is insane.
- JamesSwift 4y ago
  Id disagree its uni-dimensional. It optimizes for reproducibility and hermeticity without using virtualization (i.e. better performance).
drewm1980 4y ago
Is the extra layer of indirection you don't like Docker or this nix -> Docker integration? Compared to just installing all your nix packages in one Docker layer, this does introduce build complexity. But it's analogous to the complexity of a compiler for a static language... the thing that comes out is not more complex than what went in, so at least the complexity doesn't propagate. The images produced by this should be interchangeable with their single-layer counterparts; the caching will just be better when the code is rebuilt and re-distributed. If you're all-in on nix, does the container ecosystem even add value? The author of this software thought so, at least when he posted in 2018: "Tying in to the schedulers, orchestration, and monitoring is very valuable" Note, I have not used this; I'm just also frustrated by software development getting eaten by incidental complexity.
chriswarbo 4y ago
> I would love someone to do a cost-benefit analysis of these sorts of tools against the time of using (and sometimes debugging) Make and/or Bash. Nix does essentially the same job as Make. The differences are: - Make embeds a shell code interpreter, whilst Nix just execs a binary; given its path, a list of args and a set of env vars. (Note that almost all Nix definitions use bash as their binary!) - Make does meta-programming with a mixture of "automatic variables" ('$<', '$^', etc.), 'eval', macros, etc. whilst Nix uses a programming language. - Make relies on timestamps to figure out whether to re-use existing outputs; Nix relies on the hash of the definition (this works recursively, since hashes are included in filenames; hence changing a reference will alter all the hashes up the dependency tree). - Make runs commands in the directory where 'make' was invoked, Nix runs commands in a temp folder (and optionally restricts network and filesystem access) - Make runs commands with the same environment it was invoked with, Nix specifies the environment of commands in the build definition Nix also has a killer feature that Make can't do, called "import from derivation". This lets us define a build process, like 'fetch this git repo', then import and use Nix definitions from its result. In comparison, Makefiles can't (reliably) fetch and import each other; e.g. my C project's Makefile can't fetch the GCC source tarball, and depend on its Makefile's "install" rule to provide a compiler. My hypothesis is that this deficiency of Make is the reason for a whole bunch of unneeded complexity in the software world (e.g. "package managers", "OS distributions", "configuration managers", etc.) From a practical point of view, Nix is almost always used as a wrapper on top of something else (Make/Ant/Maven/Cabal/etc.); but that's just because most projects benefit from those "ecosystems". Note that we could just as well wrap such Ant/Maven/Cabal/etc. projects in a layer of Make instead of Nix, but nobody does since it wouldn't give us any benefit ;) If you're happy to ignore those "ecosystems" and just have a simple "bash + Make" project, you could instead have a simple "bash + Nix" project and avoid all the layers of Make/Ant/Maven/Cabal/etc. (as well as any Docker, Ansible, Apt/RPM, etc. that others might also decide to layer on top!)
- dboreham 4y ago
  Thank you for this post. This is the kind of authoritative, insightful, contextually relevant information that makes HN so valuable.
goodpoint 4y ago
> I would love someone to do a cost-benefit analysis of these sorts of tools against the time of using (and sometimes debugging) Make and/or Bash Spot on! But HN is so biased towards new or flashy stuff...

lmeyerov 4y ago

Is there a clean way to do reuse this for multistage builds? ``` FROM nixery.dev/shell/git/node14/python3.8 as debug_extras FROM our/production:1.2.3 COPY --from=debug_extras /nixstuff /ubuntu/stuff RUN python -c "print('nice!')" ```

hamandcheese 4y ago
You could copy over the nix store, though your path wouldn’t be set up correctly to find the programs you want.
JamesSwift 4y ago
It depends on what you are trying to get out of the builder pattern. I think it wouldn't provide any benefit to you but hard to say without knowing exactly what you want to achieve. e.g. if you want to remove all the stuff that you aren't using then nix already does that with a GC

hamandcheese 4y ago

This is really cool, but I don’t know if I see the appeal for actual nix users — if you are a nix user and have it set up in CI, you can easily build docker images yourself using buildLayeredImage. And then if you aren’t a nix user, why would you use this? Installing packages with, say, apt, is decidedly not where my pains with docker have arose.

rajivmr 4y ago
That's correct. I recently ended up using `buildLayerImage` (actually `buildLayerImageWithNixDb`) for CI, not only to run a single process, but also `systemd` and multiple processes. `podman` comes with built-in support for `systemd`. [Here](https://github.com/fdb-rs/fdb/blob/fdb-0.2.2/nix/ci/flake.nix#L34 https://github.com/fdb-rs/fdb/blob/fdb-0.2.2/nix/ci/flake.ni...) is relevant code.
denvrede 4y ago
If I need an image with a specific set of tools it's cumbersome to build a whole workflow to build, store and maintain these images. Having a service that can receive a custom list of Nix packages and returns an image that I can instantly use would be really, really, really nice.
- tazjin 4y ago
  That's what most users of nixery.dev (i.e. the public instance) do, afaict. Ad-hoc images for CI, and for debugging purposes.
  - machinerychorus 4y ago
    yep, this is exactly how I use it. Nixery is so convenient for getting random one-off containers. Thank you for building and hosting it!

nickysielicki 4y ago

I don’t know if it’s just because I hopped on the bandwagon in the past few months, but it really is starting to feel like Nix is gaining momentum. You can use it on Mac, WSL, Ubuntu multiuser, Docker, in an nspawn container, or through NixOS on a drive. You can build a livecd iso or a Raspberry Pi image from the same flake that you use for everything else. I work in embedded with Yocto every day, and I can’t help but think that Nix is going to eat their lunch in the next decade. There’s really never been anything (usable) that’s like Nix. I think it’s inevitable that it takes over everything.

mateuszf 4y ago
Not tried that but I've heard it's even possible to build custom Android distros.
- nickysielicki 4y ago
  I know that there’s a very active matrix channel for Nix on ARM that is mostly dedicated to pinephone/smartphone projects, so I imagine there’s some android tooling there but I don’t think it’s “android” in the AOSP sense.
  - yakak 4y ago
    There is an AOSP/etc dist building on nix project called robotnix. It has little to do with nixOS on Arm, so it is rarely mentioned on that channel.
- zimbatm 4y ago
  samueldr has been doing a lot of good work in that direction. See https://mobile.nixos.org/ https://mobile.nixos.org/ and https://github.com/samueldr/ https://github.com/samueldr/
  - samueldr 4y ago
    Just noting, using Nix it is also possible to build an actual real deal Android image using Robotnix: - https://github.com/danielfullmer/robotnix/ https://github.com/danielfullmer/robotnix/ This is different from a non-Android Linux on Mobile devices, which is what Mobile NixOS aims to achieve :).

civodul 4y ago

In the same spirit but in the form of a readily-usable command (rather than a service), 'guix pack' can produce application bundles in a reproducible fashion, in the Docker format as well as in other formats: https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-pack.html https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix... Hopefully the smart layering strategy that Nixery uses will eventually make it into 'guix pack'!

tazjin 4y ago
> Hopefully the smart layering strategy that Nixery uses will eventually make it into 'guix pack'! I've been meaning to extract it into a more standalone tool that can output a layer distribution, that way it could also be used in things like Nix's `dockerTools.buildLayeredImage`. The main annoyance is that creating the popularity data inside of a build is not easily possible for the entirety of the package set. Still working on that one ... I'm not sure how much Guix internals have diverged since they forked Nix, but if the dependency analysis of store paths can be done the same way then this should also be straightforward to port to Guix.
- lewo 4y ago
  With https://github.com/nlewo/nix2container https://github.com/nlewo/nix2container, I'm trying to make a more standalone tool. Basically, a Go binary takes a reference graph and produces a JSON file describing a container image. This JSON file is then ingested by a Skopeo fork (it adds a new `transport`) to produce images (to file, registries,...). Currently, it supports the dockerTools layering algorithm and is designed to work with Guix [1] as well;) [1] https://github.com/nlewo/nix2container/blob/065e5b108650ee4c2dc80fe1f4952eff50922e6c/nix/utils.go#L55 https://github.com/nlewo/nix2container/blob/065e5b108650ee4c...
  - tazjin 4y ago
    Ah, I've actually seen this before. Since it's written in Go, you might be able to pretty much copy&paste the Nixery layering strategy into it. I wouldn't mind!
- 4y ago

stabbles 4y ago

I like how they optimize for layer reuse, but at the same time, nix doesn't really fit docker layer caching because it's better. With a unique prefix per package, you don't need stacking of layers. You can just download a bunch of packages in parallel, extract them in parallel, and finally "merge" the prefixes to get those combined bin/ and lib/ dirs.

frafra 4y ago
> You can just download a bunch of packages in parallel, extract them in parallel, and finally "merge" the prefixes to get those combined bin/ and lib/ dirs. docker pull achieves the same result: layers are fetched in parallel, and they are extracted using pgiz (parallel gzip). It just uses a pre-defined order, which does not harm performances, but it is not useful either in case nixery is used.
- stabbles 4y ago
  The point is not about parallelization, it's that nixery has to optimize for cache reuse, which is an artificial problem created by docker. If you have two layers installing an individual packages like /nix/store/x and /nix/store/y, stacking them as [x, y] and [y, x] would result in the same docker image contents, but docker will generate different hashes.
  - frafra 4y ago
    Thanks for clarify your point. > If you have two layers installing an individual packages like /nix/store/x and /nix/store/y, stacking them as [x, y] and [y, x] would result in the same docker image contents This is an assumption which is valid for nix, but not for most of the package managers. Whenever such assumption can be considered correct, Dockerfiles can achieve similar results using multiple stages, but you would probably need a pre-processor to have a stage for each package. Something like an `INCLUDE` directive could help too: https://github.com/moby/moby/issues/3378 https://github.com/moby/moby/issues/3378.

ilovefood 4y ago

This is absolutely fantastic, I can't count the times where I needed to debug some sidecar/envoy/tls thing or some network connectivity between two different systems and needed a specific tool (nmap, telnet, ..) within a docker container to debug that and couldn't (or didn't want to) rebuild the container with the missing dependency or package. Really hits a sweet spot and might make life a bit easier for me. Thanks for sharing it!

frafra 4y ago

Would it be possible to implement something like similar using different distributions? I am thinking of Fedora with rpm-ostree, for example.