3 ms·

Nice, really hope I'll find some sweet way to cache similarish with GitLab-CI. Also kinda been thinking about how cool it'd be to run Kubernetes with Nix native

by miduil 3y ago

Nice, really hope I'll find some sweet way to cache similarish with GitLab-CI. Also kinda been thinking about how cool it'd be to run Kubernetes with Nix natively (so instead of a docker layer registry you have nix paths mounted together to overlayfs)

grhmc 3y ago
I think it should be pretty straightforward to make the Magic Nix Cache work on GitLab, too. They have a similar caching API. We'll take a look!
- randomblast 3y ago
  Is the same true for Azure Pipelines? Given Actions forked from Pipelines I'd imagine it would be straightforward.
  - grhmc 3y ago
    I'm not sure... want to open a ticket? :)
- ivanbakel 3y ago
  I would be thrilled to see this ported to GitLab CI. This is an incredibly (for lack of a better word) sexy tool on top of Nix, and would perfectly fit a use case we have for using Nix for caching build artifacts.
- pxc 3y ago
  I would love to use this kind of thing with GitLab CI (and AzDO) at work!
- tracnar 3y ago
  Nice. The GitLab caching API is basically saving and restoring a folder, so if you allow pointing to a local folder instead of the GitHub API you should be good!
whateveracct 3y ago
I just spun up a gitlab-runner on NixOS (super easy due to how NixOS works)
- grhmc 3y ago
  Nice! Yeah, we're obviously big fans of NixOS over here :). In cases where build infrastructure is highly ephemeral, this sort of cache would make a lot of sense. We'd love to help get it working there!
- miduil 3y ago
  Yeah, you are just trading security with those runners though. I want the cache to be secure and not being able to take over other repos/branches/tags ci-jobs.
  - kevincox 3y ago
    It depends what level of security you need. My runner doesn't have root access and it talks to the nix-daemon on the host to do the building, so theoretically everything is safe. Of course the attack surface is quite large, so I wouldn't expose this to the public. But using this for my repos and trusted developers is fine with me. It is basically impossible to accidentally do harm. Also note that GitLab forks and Merge Requests from forks run in the author's repo, so they won't use your runners. So it is only people with push access that will use them.
takeda 3y ago
This is what I'm using with gitlab: https://github.com/takeda/nix-cde/blob/master/contrib/gitlab-runner/nix-builder.yaml https://github.com/takeda/nix-cde/blob/master/contrib/gitlab... It caches on two levels (instance's /nix/store on EBS and then also binary cache on S3).
kevincox 3y ago
My solution is here: https://kevincox.ca/2022/01/02/nix-in-docker-caching/ https://kevincox.ca/2022/01/02/nix-in-docker-caching/ I basically expose the daemon socket into the docker container so that it requests builds from the host. It means that everything is cached right on local disk. If you need more oomph than one machine will provide the cache won't be shared between different machines without extra effort but you can do a lot of building on a single machine (especially if a lot of stuff is using Nix so cached).

by miduil 3y ago

grhmc 3y ago

I think it should be pretty straightforward to make the Magic Nix Cache work on GitLab, too. They have a similar caching API. We'll take a look!

randomblast 3y ago
Is the same true for Azure Pipelines? Given Actions forked from Pipelines I'd imagine it would be straightforward.
- grhmc 3y ago
  I'm not sure... want to open a ticket? :)
ivanbakel 3y ago
I would be thrilled to see this ported to GitLab CI. This is an incredibly (for lack of a better word) sexy tool on top of Nix, and would perfectly fit a use case we have for using Nix for caching build artifacts.
pxc 3y ago
I would love to use this kind of thing with GitLab CI (and AzDO) at work!
tracnar 3y ago
Nice. The GitLab caching API is basically saving and restoring a folder, so if you allow pointing to a local folder instead of the GitHub API you should be good!

whateveracct 3y ago

I just spun up a gitlab-runner on NixOS (super easy due to how NixOS works)

grhmc 3y ago
Nice! Yeah, we're obviously big fans of NixOS over here :). In cases where build infrastructure is highly ephemeral, this sort of cache would make a lot of sense. We'd love to help get it working there!
miduil 3y ago
Yeah, you are just trading security with those runners though. I want the cache to be secure and not being able to take over other repos/branches/tags ci-jobs.
- kevincox 3y ago
  It depends what level of security you need. My runner doesn't have root access and it talks to the nix-daemon on the host to do the building, so theoretically everything is safe. Of course the attack surface is quite large, so I wouldn't expose this to the public. But using this for my repos and trusted developers is fine with me. It is basically impossible to accidentally do harm. Also note that GitLab forks and Merge Requests from forks run in the author's repo, so they won't use your runners. So it is only people with push access that will use them.

takeda 3y ago

This is what I'm using with gitlab: https://github.com/takeda/nix-cde/blob/master/contrib/gitlab-runner/nix-builder.yaml https://github.com/takeda/nix-cde/blob/master/contrib/gitlab... It caches on two levels (instance's /nix/store on EBS and then also binary cache on S3).

kevincox 3y ago

My solution is here: https://kevincox.ca/2022/01/02/nix-in-docker-caching/ https://kevincox.ca/2022/01/02/nix-in-docker-caching/ I basically expose the daemon socket into the docker container so that it requests builds from the host. It means that everything is cached right on local disk. If you need more oomph than one machine will provide the cache won't be shared between different machines without extra effort but you can do a lot of building on a single machine (especially if a lot of stuff is using Nix so cached).