4 ms·

Way to get NVM working in CI/CD systems

22 points by EntICOnc 3y ago

bin_bash 3y ago
A lot of people are questioning the value of using nvm in CI, but I think the main reason is just that the .nvmrc file is shared between dev/CI. It's a minor benefit, but still nice. That said, rtx is so much simpler for this: curl https://rtx.pub/rtx-latest-linux-x64 > /usr/bin/rtx rtx install rtx x -- npm install rtx x -- npm run test it'll automatically use the .nvmrc/.node-version file. If developers want to use nvm locally, they still can.
- codethief 3y ago
  +1 for rtx / asdf. We have been using asdf across all our projects and pipelines.
- koolba 3y ago
  You can also cat the .nvmrc file and use that as an input to the action to setup nodejs. It’s literally a single line with the version number and there’s nothing nvm specific about it other than the name.
ehutch79 3y ago
If it’s in a docker container why do you need nvm? Wouldn’t you just pin the container for that node version as a base image?
- ehutch79 3y ago
  Apparently this is t a ci/cd thing, but a Jenkins thing…
  - atomicfiredoll 3y ago
    I haven't used Jenkins in a couple years, but I don't even think it's necessarily a Jenkins thing. Iirc one solution is that the NodeJS tooling can make multiple versions available, the job specifies the required version, and the build would automatically be run on a Jenkins node/agent with the requested version. That said, I believe there are several ways to handle it depending on how you manage agents (and desired tooling.)
- politelemon 3y ago
  I think the author doesn't mention why they've gone down this path at all and assumed it's self evident. There was nothing I could clean except for your conclusion, which is the standard way of building images. Just use the official docker image for that node version. Nvm is not a build tool.
  - SOLAR_FIELDS 3y ago
    If you want to only build and maintain a single runner image that works for multiple projects in your organization you would be interested in doing something like this. Why? Some generic CI/CD setups are simplified if you share a single runner image. One example that I’m using is actions-runner-controller. It wants to use a single image per instance, and it’s simpler/easier to manage one installation of actions-runner-controller. Maintaining security and dependencies is easier for a single image as well.
    - ehutch79 3y ago
      This feels weird. Unless you're running that image in production as well? Thats sort of the point? Production matches the test environment. It's still early here, it's very possible I'm not really with it at the moment
- codethief 3y ago
  Here's one possible reason: https://news.ycombinator.com/item?id=36002117 https://news.ycombinator.com/item?id=36002117
ghuntley 3y ago
Not worth the pain. Just use nix or a wrapper such as https://devenv.sh https://devenv.sh
- dindresto 3y ago
  We've moved our complete CI/CD pipeline to Nix. It's building all our Go, Rust and Node projects and their Docker images.
  - cal85 3y ago
    Interested to know any technical details you're willing to share about how you have this set up. I'm a nix noob trying to get my head round its various parts, and interested to hear real world examples of how people are using it in CI/CD pipelines. My experience is limited to playing a bit with 'nix shell' locally.
    - dindresto 3y ago
      I'm actually working on a blog article about our setup (or possibly a series of articles, depending on how much longer it gets), it'll be published on https://korz.dev https://korz.dev once its done. In the meantime, here's the rough summary: - Go projects are built with https://github.com/nix-community/gomod2nix https://github.com/nix-community/gomod2nix. We generate a list of internal packages a project depends on using `go list -json` that is then passed to gomod2nix's `buildGoApplication`. - Rust projects are built with https://github.com/cargo2nix/cargo2nix https://github.com/cargo2nix/cargo2nix. We chose cargo2nix to get incremental builds, meaning that dependency builds can be shared between our Rust projects and that not all dependencies have to be rebuilt when adding/updating/removing dependencies from a project. - Node projects are far trickier if you want them to be able to share dependencies and depend on one another. We have found Yarn 4 (currently release candidate, not stable yet) in combination with https://github.com/madjam002/yarnpnp2nix https://github.com/madjam002/yarnpnp2nix to work best for this. Unfortunately we have to patch some package hashes of packages that contain platform-specific binaries (such as esbuild). - Container images are built with https://github.com/nlewo/nix2container https://github.com/nlewo/nix2container instead of Nix's built-in docker tools, because nix2container allows configuration of layers and thus makes it easier to share layers between images and improve layer caching. As for integrating it into our CI pipeline, this is pretty much CI system agnostic as all you have to do is run `nix build ...` inside the pipeline, generating a `result` directory or file as an artifact that, in the case of nix2container, can be pushed to your container image registry.
__MatrixMan__ 3y ago
I'd rather just declare the version of node that this project depends on in a flake.nix and then use the same thing for dev and CI and production.
RecycledEle 3y ago
CI / CD = Continuous Integration and Continuous Delivery (for those who were wondering)
manojlds 3y ago
Why do you need nvm in CI. Just use containers and one node version.
- codethief 3y ago
  In theory you're right. However, you might have dozens of Dockerfiles and probably don't want to hard-code the Node version in each of them. Besides, when developing locally, you typically don't fire up your web app from inside a container.

22 points by EntICOnc 3y ago

A lot of people are questioning the value of using nvm in CI, but I think the main reason is just that the .nvmrc file is shared between dev/CI. It's a minor benefit, but still nice. That said, rtx is so much simpler for this: curl https://rtx.pub/rtx-latest-linux-x64 > /usr/bin/rtx rtx install rtx x -- npm install rtx x -- npm run test it'll automatically use the .nvmrc/.node-version file. If developers want to use nvm locally, they still can.

codethief 3y ago
+1 for rtx / asdf. We have been using asdf across all our projects and pipelines.
koolba 3y ago
You can also cat the .nvmrc file and use that as an input to the action to setup nodejs. It’s literally a single line with the version number and there’s nothing nvm specific about it other than the name.

ehutch79 3y ago

If it’s in a docker container why do you need nvm? Wouldn’t you just pin the container for that node version as a base image?

ehutch79 3y ago
Apparently this is t a ci/cd thing, but a Jenkins thing…
- atomicfiredoll 3y ago
  I haven't used Jenkins in a couple years, but I don't even think it's necessarily a Jenkins thing. Iirc one solution is that the NodeJS tooling can make multiple versions available, the job specifies the required version, and the build would automatically be run on a Jenkins node/agent with the requested version. That said, I believe there are several ways to handle it depending on how you manage agents (and desired tooling.)
politelemon 3y ago
I think the author doesn't mention why they've gone down this path at all and assumed it's self evident. There was nothing I could clean except for your conclusion, which is the standard way of building images. Just use the official docker image for that node version. Nvm is not a build tool.
- SOLAR_FIELDS 3y ago
  If you want to only build and maintain a single runner image that works for multiple projects in your organization you would be interested in doing something like this. Why? Some generic CI/CD setups are simplified if you share a single runner image. One example that I’m using is actions-runner-controller. It wants to use a single image per instance, and it’s simpler/easier to manage one installation of actions-runner-controller. Maintaining security and dependencies is easier for a single image as well.
  - ehutch79 3y ago
    This feels weird. Unless you're running that image in production as well? Thats sort of the point? Production matches the test environment. It's still early here, it's very possible I'm not really with it at the moment
codethief 3y ago
Here's one possible reason: https://news.ycombinator.com/item?id=36002117 https://news.ycombinator.com/item?id=36002117

ghuntley 3y ago

Not worth the pain. Just use nix or a wrapper such as https://devenv.sh https://devenv.sh

dindresto 3y ago
We've moved our complete CI/CD pipeline to Nix. It's building all our Go, Rust and Node projects and their Docker images.
- cal85 3y ago
  Interested to know any technical details you're willing to share about how you have this set up. I'm a nix noob trying to get my head round its various parts, and interested to hear real world examples of how people are using it in CI/CD pipelines. My experience is limited to playing a bit with 'nix shell' locally.
  - dindresto 3y ago
    I'm actually working on a blog article about our setup (or possibly a series of articles, depending on how much longer it gets), it'll be published on https://korz.dev https://korz.dev once its done. In the meantime, here's the rough summary: - Go projects are built with https://github.com/nix-community/gomod2nix https://github.com/nix-community/gomod2nix. We generate a list of internal packages a project depends on using `go list -json` that is then passed to gomod2nix's `buildGoApplication`. - Rust projects are built with https://github.com/cargo2nix/cargo2nix https://github.com/cargo2nix/cargo2nix. We chose cargo2nix to get incremental builds, meaning that dependency builds can be shared between our Rust projects and that not all dependencies have to be rebuilt when adding/updating/removing dependencies from a project. - Node projects are far trickier if you want them to be able to share dependencies and depend on one another. We have found Yarn 4 (currently release candidate, not stable yet) in combination with https://github.com/madjam002/yarnpnp2nix https://github.com/madjam002/yarnpnp2nix to work best for this. Unfortunately we have to patch some package hashes of packages that contain platform-specific binaries (such as esbuild). - Container images are built with https://github.com/nlewo/nix2container https://github.com/nlewo/nix2container instead of Nix's built-in docker tools, because nix2container allows configuration of layers and thus makes it easier to share layers between images and improve layer caching. As for integrating it into our CI pipeline, this is pretty much CI system agnostic as all you have to do is run `nix build ...` inside the pipeline, generating a `result` directory or file as an artifact that, in the case of nix2container, can be pushed to your container image registry.

__MatrixMan__ 3y ago

I'd rather just declare the version of node that this project depends on in a flake.nix and then use the same thing for dev and CI and production.

RecycledEle 3y ago

CI / CD = Continuous Integration and Continuous Delivery (for those who were wondering)

manojlds 3y ago

Why do you need nvm in CI. Just use containers and one node version.

codethief 3y ago
In theory you're right. However, you might have dozens of Dockerfiles and probably don't want to hard-code the Node version in each of them. Besides, when developing locally, you typically don't fire up your web app from inside a container.