4 ms·

Dream2nix – Automate reproducible packaging for various language ecosystems

peter_d_sherman 3y ago
The problem, broadly defined: You have and use and love NixOS. NixOS handles installing programs and their dependencies. But more importantly perhaps, it makes the recipes for such installation reproducible -- that is, a recipe for such production will reproduce that same environment on other machines. That's what Nix is and does, basically. But now let's add a language-related (sub)installer (like Python's 'pip', Node's 'npm', or Rust's 'cargo') -- to the mix... If we use one of those on NixOS to install a module/library/sub-dependency -- guess what? We may have just broken our Nix guaranteed reproducible environment! That's because Nix -- is unaware of what that (sub)installer (AKA "Package Manager") installed! That's also because what that (sub)installer installed may vary from one creation/instantiation of the environment on one machine, to the next, and/or if it was invoked at different points in time! So the goal here (AFAIK and it's a difficult, challenging goal!) -- is to make Nix aware of what a (sub)installer is/has installed, and bring that information into Nix such that a reproducible environment in Nix can still be guaranteed, even when a language-related (sub)installer like 'pip', 'npm', 'cargo' and others, are used! Is Dream2nix the best way to do that? I don't know! Does this software even work? I don't know -- I haven't tested it yet! But it's an interesting problem -- what to do when a necessary sub dependency is available in one of the language-based (sub)installer's repositories -- but not in Nix Packages... Installing that language-based sub-dependency on Nix is half the problem... Making what that sub-installer installed on Nix truly reproducible across Nix installations is the other half of the problem... Dream2nix would seem to be an up-and-coming project that has the most awareness of these Nix-related (sub)installer issues, thus far, so that is why I submit the link... Related: "dream2nix - A generic framework for 2nix tools": https://www.youtube.com/watch?v=jqCfHMvCsfQ https://www.youtube.com/watch?v=jqCfHMvCsfQ "Comparison of Programming Language Package Managers": https://news.ycombinator.com/item?id=12187888 https://news.ycombinator.com/item?id=12187888 https://docs.google.com/document/d/19HNnqMsETTdwwQd0I0zq2rg1IrJtaoFEA1B1OpJGNUg https://docs.google.com/document/d/19HNnqMsETTdwwQd0I0zq2rg1...

The problem, broadly defined: You have and use and love NixOS. NixOS handles installing programs and their dependencies. But more importantly perhaps, it makes the recipes for such installation reproducible -- that is, a recipe for such production will reproduce that same environment on other machines. That's what Nix is and does, basically. But now let's add a language-related (sub)installer (like Python's 'pip', Node's 'npm', or Rust's 'cargo') -- to the mix... If we use one of those on NixOS to install a module/library/sub-dependency -- guess what? We may have just broken our Nix guaranteed reproducible environment! That's because Nix -- is unaware of what that (sub)installer (AKA "Package Manager") installed! That's also because what that (sub)installer installed may vary from one creation/instantiation of the environment on one machine, to the next, and/or if it was invoked at different points in time! So the goal here (AFAIK and it's a difficult, challenging goal!) -- is to make Nix aware of what a (sub)installer is/has installed, and bring that information into Nix such that a reproducible environment in Nix can still be guaranteed, even when a language-related (sub)installer like 'pip', 'npm', 'cargo' and others, are used! Is Dream2nix the best way to do that? I don't know! Does this software even work? I don't know -- I haven't tested it yet! But it's an interesting problem -- what to do when a necessary sub dependency is available in one of the language-based (sub)installer's repositories -- but not in Nix Packages... Installing that language-based sub-dependency on Nix is half the problem... Making what that sub-installer installed on Nix truly reproducible across Nix installations is the other half of the problem... Dream2nix would seem to be an up-and-coming project that has the most awareness of these Nix-related (sub)installer issues, thus far, so that is why I submit the link... Related: "dream2nix - A generic framework for 2nix tools": https://www.youtube.com/watch?v=jqCfHMvCsfQ https://www.youtube.com/watch?v=jqCfHMvCsfQ "Comparison of Programming Language Package Managers": https://news.ycombinator.com/item?id=12187888 https://news.ycombinator.com/item?id=12187888 https://docs.google.com/document/d/19HNnqMsETTdwwQd0I0zq2rg1IrJtaoFEA1B1OpJGNUg https://docs.google.com/document/d/19HNnqMsETTdwwQd0I0zq2rg1...