12 ms·

We've moved our complete CI/CD pipeline to Nix. It's building all our Go, Rust and Node projects and their Docker images.

cal85 3y ago
Interested to know any technical details you're willing to share about how you have this set up. I'm a nix noob trying to get my head round its various parts, and interested to hear real world examples of how people are using it in CI/CD pipelines. My experience is limited to playing a bit with 'nix shell' locally.
- dindresto 3y ago
  I'm actually working on a blog article about our setup (or possibly a series of articles, depending on how much longer it gets), it'll be published on https://korz.dev https://korz.dev once its done. In the meantime, here's the rough summary: - Go projects are built with https://github.com/nix-community/gomod2nix https://github.com/nix-community/gomod2nix. We generate a list of internal packages a project depends on using `go list -json` that is then passed to gomod2nix's `buildGoApplication`. - Rust projects are built with https://github.com/cargo2nix/cargo2nix https://github.com/cargo2nix/cargo2nix. We chose cargo2nix to get incremental builds, meaning that dependency builds can be shared between our Rust projects and that not all dependencies have to be rebuilt when adding/updating/removing dependencies from a project. - Node projects are far trickier if you want them to be able to share dependencies and depend on one another. We have found Yarn 4 (currently release candidate, not stable yet) in combination with https://github.com/madjam002/yarnpnp2nix https://github.com/madjam002/yarnpnp2nix to work best for this. Unfortunately we have to patch some package hashes of packages that contain platform-specific binaries (such as esbuild). - Container images are built with https://github.com/nlewo/nix2container https://github.com/nlewo/nix2container instead of Nix's built-in docker tools, because nix2container allows configuration of layers and thus makes it easier to share layers between images and improve layer caching. As for integrating it into our CI pipeline, this is pretty much CI system agnostic as all you have to do is run `nix build ...` inside the pipeline, generating a `result` directory or file as an artifact that, in the case of nix2container, can be pushed to your container image registry.
- jefflinwood 3y ago
  You could checkout Nixpacks - https://nixpacks.com/docs/getting-started https://nixpacks.com/docs/getting-started - if you want a solution to build images as part of a CI/CD pipeline.

cal85 3y ago

Interested to know any technical details you're willing to share about how you have this set up. I'm a nix noob trying to get my head round its various parts, and interested to hear real world examples of how people are using it in CI/CD pipelines. My experience is limited to playing a bit with 'nix shell' locally.

dindresto 3y ago
I'm actually working on a blog article about our setup (or possibly a series of articles, depending on how much longer it gets), it'll be published on https://korz.dev https://korz.dev once its done. In the meantime, here's the rough summary: - Go projects are built with https://github.com/nix-community/gomod2nix https://github.com/nix-community/gomod2nix. We generate a list of internal packages a project depends on using `go list -json` that is then passed to gomod2nix's `buildGoApplication`. - Rust projects are built with https://github.com/cargo2nix/cargo2nix https://github.com/cargo2nix/cargo2nix. We chose cargo2nix to get incremental builds, meaning that dependency builds can be shared between our Rust projects and that not all dependencies have to be rebuilt when adding/updating/removing dependencies from a project. - Node projects are far trickier if you want them to be able to share dependencies and depend on one another. We have found Yarn 4 (currently release candidate, not stable yet) in combination with https://github.com/madjam002/yarnpnp2nix https://github.com/madjam002/yarnpnp2nix to work best for this. Unfortunately we have to patch some package hashes of packages that contain platform-specific binaries (such as esbuild). - Container images are built with https://github.com/nlewo/nix2container https://github.com/nlewo/nix2container instead of Nix's built-in docker tools, because nix2container allows configuration of layers and thus makes it easier to share layers between images and improve layer caching. As for integrating it into our CI pipeline, this is pretty much CI system agnostic as all you have to do is run `nix build ...` inside the pipeline, generating a `result` directory or file as an artifact that, in the case of nix2container, can be pushed to your container image registry.
jefflinwood 3y ago
You could checkout Nixpacks - https://nixpacks.com/docs/getting-started https://nixpacks.com/docs/getting-started - if you want a solution to build images as part of a CI/CD pipeline.