3 ms·

Is there a system for running a package cache for nix? I love how Nix stops dependency churn from breaking my builds. What worries me is that it introduces quit

Is there a system for running a package cache for nix? I love how Nix stops dependency churn from breaking my builds. What worries me is that it introduces quite a few dependencies on remote servers, both for any global repository, and any tarballs referenced in the nix file. With golang, you can setup your own package proxy, point GOPROXY environment to that, and then be guaranteed that even if the original host is no longer available, you can build your code. Is there an equivalent for nix?

southerntofu 5y ago
I'm unaware of the details (haven't done that myself) but you can run your own build server and repository. The problem is more about archiving upstream sources, but if your own build server runs behind a caching proxy it should not be a problem once initial build of all your packages is done.
KingMachiavelli 5y ago
Yes, the nix store essentially acts as a cache. Nix can access "substituters" (e.g. remote nix stores/caches) over SSH. If you run nix-serve that's what cache.nixos.org is using (nix store over HTTP). [1] If you enable the "keep-outputs" and "keep-derivations" in your Nix config then your local nix store should keep anything it used until you manually clean out old derivations (assuming you aren't accessing the tarballs via some ---impure method). An easier option is to use a tool/service like cachix [2] to run your own repository which can be used just like the normal cache.nixos.org. [1] https://nixos.wiki/wiki/Binary_Cache https://nixos.wiki/wiki/Binary_Cache [2] https://www.cachix.org/ https://www.cachix.org/
tomberek 5y ago
This is an often unappreciated killer feature in Nix. (perhaps not well explained or documented?) copy runtime requirements: nix copy --to s3://some-bucket ./result copy full build description: nix copy --to s3://some-bucket ./result --derivation copy runtime of the buildtime: nix path-info -r nixpkgs#hello --derivation | xargs -n1 nix path-info | nix copy --to s3://some-bucket Then you set your substituter to s3://some-bucket either in nix.conf or in CLI with --substituters Or if you don't like s3, point it to https, ssh, or a simple local file directory.

southerntofu 5y ago

I'm unaware of the details (haven't done that myself) but you can run your own build server and repository. The problem is more about archiving upstream sources, but if your own build server runs behind a caching proxy it should not be a problem once initial build of all your packages is done.

KingMachiavelli 5y ago

Yes, the nix store essentially acts as a cache. Nix can access "substituters" (e.g. remote nix stores/caches) over SSH. If you run nix-serve that's what cache.nixos.org is using (nix store over HTTP). [1] If you enable the "keep-outputs" and "keep-derivations" in your Nix config then your local nix store should keep anything it used until you manually clean out old derivations (assuming you aren't accessing the tarballs via some ---impure method). An easier option is to use a tool/service like cachix [2] to run your own repository which can be used just like the normal cache.nixos.org. [1] https://nixos.wiki/wiki/Binary_Cache https://nixos.wiki/wiki/Binary_Cache [2] https://www.cachix.org/ https://www.cachix.org/

tomberek 5y ago

This is an often unappreciated killer feature in Nix. (perhaps not well explained or documented?) copy runtime requirements: nix copy --to s3://some-bucket ./result copy full build description: nix copy --to s3://some-bucket ./result --derivation copy runtime of the buildtime: nix path-info -r nixpkgs#hello --derivation | xargs -n1 nix path-info | nix copy --to s3://some-bucket Then you set your substituter to s3://some-bucket either in nix.conf or in CLI with --substituters Or if you don't like s3, point it to https, ssh, or a simple local file directory.