3 ms·

Exactly this. It would be really great if for example when building Rust in Nix, Nix could own each of the artifacts, in order to share artifacts between tools

Exactly this. It would be really great if for example when building Rust in Nix, Nix could own each of the artifacts, in order to share artifacts between tools that use the same revisions of dependencies, as well as to avoid rebuilding artifacts unnecessarily on every change to a derivation.

digikata 7y ago
How does Nix handle areas with extensive package ecosystems? Python, Node et al? It seems like every distro/super packager has different compromises for this question.
- lilyball 7y ago
  I haven't investigated Python, but for Node it uses a tool node2nix that fetches the metadata for the entire dependency tree and constructs a Nix expression for each one, such that each expression represents the source for that specific dependency. It then takes the top-level package and makes a call to `nodeEnv.buildNodePackage`, which contains the source for that top-level package and declares the entire recursive dependency tree. buildeNodePackage itself rebuilds the entire node_modules folder from the declared dependencies and reconstructs the lockfile. And then it asks npm to rebuild the package. The end result is this bypasses npm's downloading and managing of node_modules, does it all in a fully declarative fashion, and then hands control over to npm to build the package. Also, each dependency's source is cached independently by Nix, so if multiple packages use the exact same dependency, it doesn't have to be redownloaded (unlike Rust where the entire dependency tree is vendored separately per package).
  - steveklabnik 7y ago
    > (unlike Rust where the entire dependency tree is vendored separately per package). To be clear, Cargo downloads the source globally. The compiled output is per-project, however.
    - lilyball 7y ago
      Nix uses `cargo vendor` to literally vendor the source of all dependencies as a separate derivation. Maybe cargo keeps a global cache of the source as well when doing `cargo vendor`, I don't know, but Nix does construct a derivation that contains just the vendored dependency sources and then provides that as input when building the full crate.

digikata 7y ago

How does Nix handle areas with extensive package ecosystems? Python, Node et al? It seems like every distro/super packager has different compromises for this question.

lilyball 7y ago
I haven't investigated Python, but for Node it uses a tool node2nix that fetches the metadata for the entire dependency tree and constructs a Nix expression for each one, such that each expression represents the source for that specific dependency. It then takes the top-level package and makes a call to `nodeEnv.buildNodePackage`, which contains the source for that top-level package and declares the entire recursive dependency tree. buildeNodePackage itself rebuilds the entire node_modules folder from the declared dependencies and reconstructs the lockfile. And then it asks npm to rebuild the package. The end result is this bypasses npm's downloading and managing of node_modules, does it all in a fully declarative fashion, and then hands control over to npm to build the package. Also, each dependency's source is cached independently by Nix, so if multiple packages use the exact same dependency, it doesn't have to be redownloaded (unlike Rust where the entire dependency tree is vendored separately per package).
- steveklabnik 7y ago
  > (unlike Rust where the entire dependency tree is vendored separately per package). To be clear, Cargo downloads the source globally. The compiled output is per-project, however.
  - lilyball 7y ago
    Nix uses `cargo vendor` to literally vendor the source of all dependencies as a separate derivation. Maybe cargo keeps a global cache of the source as well when doing `cargo vendor`, I don't know, but Nix does construct a derivation that contains just the vendored dependency sources and then provides that as input when building the full crate.