3 ms·
Looks like this requires Docker to be available? Personally I've been making OCI images using Nix, just by running tar, sha256sum and jq: - We can get a Nix d
by chriswarbo 4y ago
Looks like this requires Docker to be available?
Personally I've been making OCI images using Nix, just by running tar, sha256sum and jq:
- We can get a Nix derivation's dependencies using `writeDependenciesToFile`
- We can put these dependencies (alongside anything else we like) into an OCI "layer" using GNU tar: the `--hard-dereference` and `--mode=755` options work well, and we can include directories using `-C /foo --filesFrom=foo.txt` where foo.txt comes from `cd /foo && find . -maxdepth 1`, with the leading `./` removed.
- A "digest" is just a JSON file like `{size: 123, digest: "abc"}`, e.g. generated using `stat` and `sha256sum`
- Those sha256sums can be referenced in a config.json, like `{"rootfs": {"type": "layers", "diff_ids": [...]}}`; along with the application-specific config (EntryPoint, WorkingDir, etc.)
- Finally, a "manifest" is just another JSON file; with `{"mediaType": "application/vnd.oci.image.config.v1+json"}` for the config digest, and `{"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip"}` for each layer digest.
Unlike Docker, these commands can all be run directly (from the "host", although it's not really hosting anything, since we don't need any containers or VMs to run the above); their contents are completely declarative (since we're just zipping up pre-existing files, e.g. built/fetched by Nix; rather than mutating a filesystem in-place); we can do it on any OS (e.g. no need for a Linux VM on macOS); etc.
- ris 4y agoUm.. `dockerTools`/`ociTools`? https://nixos.org/manual/nixpkgs/stable/#chap-images https://nixos.org/manual/nixpkgs/stable/#chap-images
- chriswarbo 4y ago> dockerTools "Tags", "registries", "base images", "extraCommands"... no thanks > ociTools IIRC, this generates a runnable container (i.e. a directory, with accompanying config); not a container image, e.g. suitable for uploading to AWS ECS.
- kam 4y ago> "Tags", "registries", "base images", "extraCommands"... no thanks All of which are optional. It's just doing the same thing as your tar commands.
- chriswarbo 4y ago> All of which are optional True, but they're dangerous enough that I'd still at least want a wrapper function which doesn't accept them at all. > It's just doing the same thing as your tar commands. Not quite: I came up with those commands by reading the OCI specifications. In contrast, the output of the dockerTools functions does not quite follow those specifications, and appears to be more Docker-specific.
- pxc 4y agoNix 3 has a dead simple, builtin CLI frontend for doing this with built packages: https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-bundle.html https://nixos.org/manual/nix/unstable/command-ref/new-cli/ni... It uses dockerTools: https://github.com/NixOS/bundlers/blob/master/flake.nix https://github.com/NixOS/bundlers/blob/master/flake.nix maybe a bundler can be added that uses OCI tools, thus providing such a wrapper and giving a nice CLI for it
- Nullabillity 4y ago> "Tags", "registries", "base images", "extraCommands"... no thanks Usually you wouldn't use any of that, unless you have some external organizational reason to depend on a base image. A typical image would just be: dockerTools.streamLayeredImage { name = "mything"; config.Cmd = "${mything}/bin/mything"; }
- chriswarbo 4y agoHaving equivalents like ociTools.buildImage and ociTools.streamLayeredImage functions would certainly be nice; but at the moment there's only ociTools.buildContainer, which doesn't use images. I'll see if Nixpkgs is receptive to having such functionality, as a unification of what I've done with what exists in dockerTools (although I need to check that my employer's happy for it to be upstreamed first!)
- justjake 4y agoThere's a lot of cool things in here, some we're excited to explore and some that I'm first hearing about right now We could jam back and forth on this but generally we're not dogmatic about the "middle" of the process, and we suspect the implementation will change over time for a variety of reasons (even faster builds, smaller images, less dependencies, etc). The only thing we're dogmatic about is the experience and the fact that it emits an OCI compliant image (that can indeed be run on any orchestrator). If you're interested in contributing or jamming on this wanna shoot me a DM on twitter (JustJake) or email at jake at railway dot app?
- benreesman 4y agoThis is a low value comment and I deserve any downvotes I get: but I Fucking Love (TM) when people just slam out working shit like this without any red tape. Well played.
- madjam002 4y agoDo you have more info on this `writeDependenciesToFile` function? I can't seem to find any information about it.
- chriswarbo 4y agoWhoops, it's actually called `writeReferencesToFile`; sorry! https://nixos.org/manual/nixpkgs/stable/#trivial-builder-writeReferencesToFile https://nixos.org/manual/nixpkgs/stable/#trivial-builder-wri...
- madjam002 4y agoAh thanks for the pointer!
- senorsmile 4y agoDo you have any working examples you could link to? Or better yet, a script that can be run to perform the aforementioned steps?