7 ms·

> The host system and isolated environments will all be managed declaratively and reproducibly using Nix, a purely functional package manager Nix seems cool, b

> The host system and isolated environments will all be managed declaratively and reproducibly using Nix, a purely functional package manager Nix seems cool, but all of my forays into it have been so unpleasant as to be unworkable. It seems to work well enough if all of your dependencies are already in the Nix store and have been thoroughly tested, but as soon as one has to start writing Nix packages themselves it's a train wreck: the peculiarities of the Nix expression language, the chaos and lack of documentation in nixpkgs, the immense understanding one must have of the common low-level libraries used to create Nix packages, and you'll still spend tons of time trying to package obscure C dependencies with their own bespoke build systems. My constructive feedback is: 1. Add types to the Nix expression language so someone digging through the code can have some idea about what needs to be passed into various functions. This would probably help people traverse nixpkgs as well, since the client code will need to "import" the types of its arguments. 2. Make Nix more syntactically familiar. Familiarity here seems like it should be more important than innovating on programming language syntax. Thanks for not going full-Haskell on us, but it would be nice if looked more like JavaScript or Python or something that virtually any programmer could look at and recognize (I'm no great fan of either of those languages). 3. Very controversial, but the whole industry needs to minimize the number of C and C++ dependencies. Not only are these languages fundamentally insecure, but projects in these languages have their own bespoke build systems which assume dependencies are already installed at the correct versions and in the correct paths. Packaging these projects is painful, and it's largely the reason we have package maintainers who specialize not in building programs of a certain language, but in building certain dependencies.

tinco 5y ago
> Thanks for not going full-Haskell on us It would have been so much better if they went full Haskell. Haskell is better designed as a language, it has types and could support all Nix features without modification as far as I can tell. On top of that it's a common language that most have used at least in university and has loads of friendly tutorials online. Totally agree on the packaging of C/C++ projects is a drag of our entire industry and basically a complete embarrassment.
- SkyMarshal 5y ago
  I'm relatively new to NixOS but this is my sense of things too - it might have been better to use a subset of Haskell for the Nix package manager, instead of a new untyped language, for the reasons you give. One simple thing I like about Haskell is its readability. Instead of cramming both type signature and function definition into the same line, Haskell splits those two into separate lines. Having the type signature in its own line, as the first line of every function, is so much more readable. It's also interesting that you can outline/pseudo-code an entire program structure using only type signatures and no function definitions, check if the type signatures are all correct, and then fill in the function definitions later. That said, I'm still new to Nix, and there are probably good reasons it was done this way, buried in past discussions, the creator's PhD thesis, etc. that I haven't read yet. And I hear that types are being looked into for future upgrades to Nix.
  - mikepurvis 5y ago
    I'm not a Haskell user, but my assumption has been that the Nix language being something of a DSL has meant that it's able to include certain kinds of convenient optimizations such as directly referring to files by their relative path. Is this bogus, or would you imagine that a substantial number of additional wrapper/helper/noise functions would be needed to make a general-purpose language do what the Nix language does? I guess Guix is probably an opportunity to look at a practical instance of this first-hand.
    - momentoftop 5y ago
      Not the parent, but I think if you tried to do Nix in Haskell, the syntax would be seriously gross. For starters, you couldn't use Haskell's record syntax for Nix's sets, because they're not Haskell records. Types wouldn't help that much when you come to do lookups in these sets: if you get the name of a key wrong, it's a runtime error. The ability to use file paths in Nix gives you a very cheap mechanism for splitting code across files. In Nix, the expression `import <filepath>` will evaluate to the result of evaluating the file `<filepath>`. But in Haskell, every such import would need to be declared at the top of the file, and the module name would need to be duplicated at the use-site, almost certainly fully qualified to avoid nameclashes. I'm not overly bothered by type systems for things like Nix. In Nix, you're mostly just coding the construction of a single, reproducible value, which is trivial to exhaustively test: just construct it and see if it's right. If there's something wrong, you'll either 1) get the error immediately when you try to evaluate, and you might even be told helpful things like a list of strings was expected, not a list of numbers. 2) get an error when the derivation is built, due to an environment error that is unlikely to be detected by a type system. Splitting this into two phases, compile then evaluate, wouldn't help me much.
isr 5y ago
This doesn't address the dynamic vs static typing issue you brought up, but for the rest, you may be interested in exploring guix. Guix is essentially a reimplementation of nix packaging system using a pre-existing language - guile scheme. Some may react adversely to them using a lisp (<s>well, enlightenment is not for all ... </s>), but leaving that aside, guix has continued to evolve in slightly different ways to nix, but still the core (and the guarantees it provides) remains essentially the same
- mateuszf 5y ago
  The Lisp part is great, but sticking to only libre software makes it a hard pill to swallow.
  - spindle 5y ago
    Right, and I think (I hope I'm wrong) that guix makes it difficult to boot from ZFS, even though ZFS is libre. NixOS makes ZFS very easy.
    - rekado 5y ago
      I think you're wrong. Guix has a zfs package. The build farm just won't give you a pre-built binary due to license conflicts.
the_duke 5y ago
I switched to NixOS about a year ago. It's been amazing for me. Especially when combined with home-manager [1], which provides declarative user environments and flakes [2], which provide a even more declarative and more easily shareable package format. Getting my exact setup on a new device - including lots of GUI and terminal app customization - takes a single `nixos-rebuild-switch`, and works every time. Everything is configured in a single config file, from hardware and system service setup, over window manager setup, all the way to installed Vim/VS Code/Chrome plugins and lots of application configs. The ability to easily roll back to previous configurations and boot into old configs from the boot menu in case something breaks is also brilliant. You can also just install and run pretty much all software on demand, similar to `npx`, without polluting the system. ( nix run some-app). Declarative, reproducible development environments for each project are the cherry on top. That said, the onboarding experience is horrible. There are A LOT of things to learn, and the documentation is bad. I also agree that the language, while somewhat fine, is not great and very undiscoverable. The Nix ecosystem is a diamond in the rough. Sadly it would take a lot of effort to simplify everything and make it more polished. I'm afraid Nix will continue to remain very niche. [1] https://github.com/nix-community/home-manager https://github.com/nix-community/home-manager [2] https://nixos.wiki/wiki/Flakes https://nixos.wiki/wiki/Flakes
- a9h74j 5y ago
  Low skill myself at installing Linux, but I found installing a Guix to disk relatively straightforward. Managed to easily add available builds. Apparently-good support for virtual machines as well. http://guix.gnu.org/ http://guix.gnu.org/
- throwaway894345 5y ago
  Even trying to get VS Code working with a couple of plugins (and the help of the good, experienced people on the Nix discord) was impossible (I was just using the package manager, not NixOS--apparently VS Code requires a different incantation depending on which platform you use, which is another problem. > Sadly it would take a lot of effort to simplify everything and make it more polished. Yes, and while it would take a lot of effort to make most things nicer (improving the language, etc), other things require effort that scales linearly with the number of packages in the universe. For example, making the package definitions in the first place, adding documentation for the packages, testing packages, etc. Other things are even worse: testing various combinations of dependencies at different versions. > I'm afraid Nix will continue to remain very niche. Unless it gets some serious corporate support, I'm afraid I'll agree. The Nix folks have done some impressive work, but I get the impression that they're more interested in making something cool for themselves and other like-minded people at the expense of others (and there's nothing wrong with that! it's just incompatible with growing out of a niche).
vnlwq 5y ago
I'm not sure what kind of C/C++ projects you install, but most of them are still ./configure && make && make install. Cmake can be worse, Meson versionitis can be annoying. It is also annoying to install Meson and Python in stage 1 in Linuxfromscratch now. Yuck! The languages aren't fundamentally insecure, there are tons of reliable projects out there.
- sa1 5y ago
  For the kind of reproducibility nix needs, ./configure && make && make install is not enough. It needs to track and manage the artefacts created by the build system, not just run a script.
  - iammisc 5y ago
    Except, most of a nixos system is built by simple Autoconf scripts and some env cars being set correctly. I mean I built a whole nix distribution for embedded devices with runit as init and cross compiling and everything and nix made it easy and fun. And I learned that it's just Autoconf and shell all the way down. There's no magic to nix. The most voodoo non standard thing they do is patchelf stuff, but most of the nixos specific stuff is just setting paths to configure scripts or in the environment.
    - sa1 5y ago
      The builds are not guaranteed to be reproducible.
      - iammisc 5y ago
        Nix doesn't guarantee reproducible builds. That's why there's a whole signing framework
        sa1 5y ago
        There is no disagreement there. Reproducibility is definitely a goal though. I don't really understand your line of argument.
mikepurvis 5y ago
I started on my Nix journey a few months ago, and in some ways, I've had the worst-possible experience— I had basically no ramp-in period of being just a "Nix user" before attempting to write my first package, but was instead thrust immediately into a massive project templating out generated definitions for 1000+ interlinked packages with a bunch of oddball requirements, funky build tools, etc. I've needed to manage version and patch overrides, obscure linker conflicts, stuff that broke at runtime due to unpropagated buildInputs. I'm also a functional programming novice, so that recursive combinator magic which makes the overlay system work— also something completely new that I've had to wrap my head around. So yes, all this has been... somewhat harrowing. But overall I would say I'm quite pleased with the end result and impressed with what Nix has enabled. Despite its long evolutionary growth, many of the things that I've encountered feel thoughtfully considered and well made. The various ways that builds and binaries can be pushed around and shared is awesome, and I'm excited for the optimizations coming with the content-addressed store. I don't think that making the Nix language or package definitions feel more conventional would necessarily be a good thing— I think there's a risk that those kinds of changes obscure the underlying realities and lead to expectation mismatches. That said, one thing that I do feel is pretty unfortunate is the interplay between the shell and Nix in package definitions. Basically you pass a dict to mkDerivation, and all the stringy items in that dict become available to your build's environment as envvars. This means when writing out strings for your configurePhase, buildPhase, etc, you have to be hyper-aware that ${thing} will be templated by Nix in the Nix context, whereas $thing will be templated much later, by Bash. It's something that is learned quickly but my goodness is it ever a hell of a tripwire.
- iammisc 5y ago
  Nix needs a shell language integrated into nix. Nix should be usable as an os shell. That would make complex scripting so much easier than the weird passasfile bizarreness
  - rgoulter 5y ago
    What's the "weird passasfile bizarreness"?
maxdamantus 5y ago
> 1. Add types to the Nix expression language Interestingly, this was one of the first issues that Edolstra (creator of Nix) raised himself, but unfortunately he closed it again as unrealistic a number of years later [0]. [0] https://github.com/NixOS/nix/issues/14#issuecomment-378201908 https://github.com/NixOS/nix/issues/14#issuecomment-37820190...
- akavel 5y ago
  Thanks for linking this - reading through the thread made me discover a cool related project! -> https://github.com/tweag/nickel/ https://github.com/tweag/nickel/ See esp.: https://github.com/tweag/nickel/#related-projects-and-inspirations https://github.com/tweag/nickel/#related-projects-and-inspir...
ducktective 5y ago
About point 3: I don't think we should just ignore the immense technological knowledge encoded in C/C++ projects throughout the decades, only because they came from an era which having a package manager for every language was not fad of the day. Let's see how those "memory safe" languages would fare 20 years down the line...but I predict there would be a cool_lang and people would advocate for rewriting every algorithm known to man, yet again
- nix23 5y ago
  Not ignoring, but minimizing the surface, for example: Should the default "webserver" really be written in c? Or maybe just the one for embedded or high performance?
  - ducktective 5y ago
    Yeah, best language for the task...but the parent was talking about not using a dependency just because it is written in C or C++
    - nix23 5y ago
      No, he was talking about "minimizing" c/c++ dependencies.
- throwaway894345 5y ago
  I sympathize with this point, and I tried to be quite clear that I'm not suggesting we drop everything cavalierly today, but rather we decide that our aspiration is to deprecate C/C++ and we move our dependencies gradually from C-based to something else. Some dependencies will be more pernicious to move because they require a lot of specialty knowledge (e.g., Harfbuzz) or simply due to their size (e.g., Chromium) but many others could be moved more readily. It will be a long, hard road which is why we should start now. > Let's see how those "memory safe" languages would fare 20 years down the line...but I predict there would be a cool_lang and people would advocate for rewriting every algorithm known to man, yet again This just reads like you've taken personal offense. My argument is quite clearly not "C isn't cool enough" or "C isn't memory safe", it's "C lacks a standard, sane build system such that packaging C projects is a nightmare".
nix23 5y ago
To your point 3, i don't think it's too controversial, some tools are absolutely ok being written in c like all the core-utils, but a monolithic kernel, written in c is probably the biggest problem: https://www.cvedetails.com/product/47/Linux-Linux-Kernel.html?vendor_id=33 https://www.cvedetails.com/product/47/Linux-Linux-Kernel.htm...
olodus 5y ago
I haven't tried it myself yet but Dhall is able to output nix files which then makes it possible to write with types in Dhall and then output the nix files (without types annotations) when they would be used.
Chris2048 5y ago
> you'll still spend tons of time trying to package obscure C dependencies but my question would be: is this exactly the same issue that all other package maintainers have e.g. rpm, deb etc?

tinco 5y ago

> Thanks for not going full-Haskell on us It would have been so much better if they went full Haskell. Haskell is better designed as a language, it has types and could support all Nix features without modification as far as I can tell. On top of that it's a common language that most have used at least in university and has loads of friendly tutorials online. Totally agree on the packaging of C/C++ projects is a drag of our entire industry and basically a complete embarrassment.

SkyMarshal 5y ago
I'm relatively new to NixOS but this is my sense of things too - it might have been better to use a subset of Haskell for the Nix package manager, instead of a new untyped language, for the reasons you give. One simple thing I like about Haskell is its readability. Instead of cramming both type signature and function definition into the same line, Haskell splits those two into separate lines. Having the type signature in its own line, as the first line of every function, is so much more readable. It's also interesting that you can outline/pseudo-code an entire program structure using only type signatures and no function definitions, check if the type signatures are all correct, and then fill in the function definitions later. That said, I'm still new to Nix, and there are probably good reasons it was done this way, buried in past discussions, the creator's PhD thesis, etc. that I haven't read yet. And I hear that types are being looked into for future upgrades to Nix.
- mikepurvis 5y ago
  I'm not a Haskell user, but my assumption has been that the Nix language being something of a DSL has meant that it's able to include certain kinds of convenient optimizations such as directly referring to files by their relative path. Is this bogus, or would you imagine that a substantial number of additional wrapper/helper/noise functions would be needed to make a general-purpose language do what the Nix language does? I guess Guix is probably an opportunity to look at a practical instance of this first-hand.
  - momentoftop 5y ago
    Not the parent, but I think if you tried to do Nix in Haskell, the syntax would be seriously gross. For starters, you couldn't use Haskell's record syntax for Nix's sets, because they're not Haskell records. Types wouldn't help that much when you come to do lookups in these sets: if you get the name of a key wrong, it's a runtime error. The ability to use file paths in Nix gives you a very cheap mechanism for splitting code across files. In Nix, the expression `import <filepath>` will evaluate to the result of evaluating the file `<filepath>`. But in Haskell, every such import would need to be declared at the top of the file, and the module name would need to be duplicated at the use-site, almost certainly fully qualified to avoid nameclashes. I'm not overly bothered by type systems for things like Nix. In Nix, you're mostly just coding the construction of a single, reproducible value, which is trivial to exhaustively test: just construct it and see if it's right. If there's something wrong, you'll either 1) get the error immediately when you try to evaluate, and you might even be told helpful things like a list of strings was expected, not a list of numbers. 2) get an error when the derivation is built, due to an environment error that is unlikely to be detected by a type system. Splitting this into two phases, compile then evaluate, wouldn't help me much.

isr 5y ago

This doesn't address the dynamic vs static typing issue you brought up, but for the rest, you may be interested in exploring guix. Guix is essentially a reimplementation of nix packaging system using a pre-existing language - guile scheme. Some may react adversely to them using a lisp (<s>well, enlightenment is not for all ... </s>), but leaving that aside, guix has continued to evolve in slightly different ways to nix, but still the core (and the guarantees it provides) remains essentially the same

mateuszf 5y ago
The Lisp part is great, but sticking to only libre software makes it a hard pill to swallow.
- spindle 5y ago
  Right, and I think (I hope I'm wrong) that guix makes it difficult to boot from ZFS, even though ZFS is libre. NixOS makes ZFS very easy.
  - rekado 5y ago
    I think you're wrong. Guix has a zfs package. The build farm just won't give you a pre-built binary due to license conflicts.

the_duke 5y ago

I switched to NixOS about a year ago. It's been amazing for me. Especially when combined with home-manager [1], which provides declarative user environments and flakes [2], which provide a even more declarative and more easily shareable package format. Getting my exact setup on a new device - including lots of GUI and terminal app customization - takes a single `nixos-rebuild-switch`, and works every time. Everything is configured in a single config file, from hardware and system service setup, over window manager setup, all the way to installed Vim/VS Code/Chrome plugins and lots of application configs. The ability to easily roll back to previous configurations and boot into old configs from the boot menu in case something breaks is also brilliant. You can also just install and run pretty much all software on demand, similar to `npx`, without polluting the system. ( nix run some-app). Declarative, reproducible development environments for each project are the cherry on top. That said, the onboarding experience is horrible. There are A LOT of things to learn, and the documentation is bad. I also agree that the language, while somewhat fine, is not great and very undiscoverable. The Nix ecosystem is a diamond in the rough. Sadly it would take a lot of effort to simplify everything and make it more polished. I'm afraid Nix will continue to remain very niche. [1] https://github.com/nix-community/home-manager https://github.com/nix-community/home-manager [2] https://nixos.wiki/wiki/Flakes https://nixos.wiki/wiki/Flakes

a9h74j 5y ago
Low skill myself at installing Linux, but I found installing a Guix to disk relatively straightforward. Managed to easily add available builds. Apparently-good support for virtual machines as well. http://guix.gnu.org/ http://guix.gnu.org/
throwaway894345 5y ago
Even trying to get VS Code working with a couple of plugins (and the help of the good, experienced people on the Nix discord) was impossible (I was just using the package manager, not NixOS--apparently VS Code requires a different incantation depending on which platform you use, which is another problem. > Sadly it would take a lot of effort to simplify everything and make it more polished. Yes, and while it would take a lot of effort to make most things nicer (improving the language, etc), other things require effort that scales linearly with the number of packages in the universe. For example, making the package definitions in the first place, adding documentation for the packages, testing packages, etc. Other things are even worse: testing various combinations of dependencies at different versions. > I'm afraid Nix will continue to remain very niche. Unless it gets some serious corporate support, I'm afraid I'll agree. The Nix folks have done some impressive work, but I get the impression that they're more interested in making something cool for themselves and other like-minded people at the expense of others (and there's nothing wrong with that! it's just incompatible with growing out of a niche).

vnlwq 5y ago

I'm not sure what kind of C/C++ projects you install, but most of them are still ./configure && make && make install. Cmake can be worse, Meson versionitis can be annoying. It is also annoying to install Meson and Python in stage 1 in Linuxfromscratch now. Yuck! The languages aren't fundamentally insecure, there are tons of reliable projects out there.

sa1 5y ago
For the kind of reproducibility nix needs, ./configure && make && make install is not enough. It needs to track and manage the artefacts created by the build system, not just run a script.
- iammisc 5y ago
  Except, most of a nixos system is built by simple Autoconf scripts and some env cars being set correctly. I mean I built a whole nix distribution for embedded devices with runit as init and cross compiling and everything and nix made it easy and fun. And I learned that it's just Autoconf and shell all the way down. There's no magic to nix. The most voodoo non standard thing they do is patchelf stuff, but most of the nixos specific stuff is just setting paths to configure scripts or in the environment.
  - sa1 5y ago
    The builds are not guaranteed to be reproducible.
    - iammisc 5y ago
      Nix doesn't guarantee reproducible builds. That's why there's a whole signing framework
      - sa1 5y ago
        There is no disagreement there. Reproducibility is definitely a goal though. I don't really understand your line of argument.

mikepurvis 5y ago

I started on my Nix journey a few months ago, and in some ways, I've had the worst-possible experience— I had basically no ramp-in period of being just a "Nix user" before attempting to write my first package, but was instead thrust immediately into a massive project templating out generated definitions for 1000+ interlinked packages with a bunch of oddball requirements, funky build tools, etc. I've needed to manage version and patch overrides, obscure linker conflicts, stuff that broke at runtime due to unpropagated buildInputs. I'm also a functional programming novice, so that recursive combinator magic which makes the overlay system work— also something completely new that I've had to wrap my head around. So yes, all this has been... somewhat harrowing. But overall I would say I'm quite pleased with the end result and impressed with what Nix has enabled. Despite its long evolutionary growth, many of the things that I've encountered feel thoughtfully considered and well made. The various ways that builds and binaries can be pushed around and shared is awesome, and I'm excited for the optimizations coming with the content-addressed store. I don't think that making the Nix language or package definitions feel more conventional would necessarily be a good thing— I think there's a risk that those kinds of changes obscure the underlying realities and lead to expectation mismatches. That said, one thing that I do feel is pretty unfortunate is the interplay between the shell and Nix in package definitions. Basically you pass a dict to mkDerivation, and all the stringy items in that dict become available to your build's environment as envvars. This means when writing out strings for your configurePhase, buildPhase, etc, you have to be hyper-aware that ${thing} will be templated by Nix in the Nix context, whereas $thing will be templated much later, by Bash. It's something that is learned quickly but my goodness is it ever a hell of a tripwire.

iammisc 5y ago
Nix needs a shell language integrated into nix. Nix should be usable as an os shell. That would make complex scripting so much easier than the weird passasfile bizarreness
- rgoulter 5y ago
  What's the "weird passasfile bizarreness"?

maxdamantus 5y ago

> 1. Add types to the Nix expression language Interestingly, this was one of the first issues that Edolstra (creator of Nix) raised himself, but unfortunately he closed it again as unrealistic a number of years later [0]. [0] https://github.com/NixOS/nix/issues/14#issuecomment-378201908 https://github.com/NixOS/nix/issues/14#issuecomment-37820190...

akavel 5y ago
Thanks for linking this - reading through the thread made me discover a cool related project! -> https://github.com/tweag/nickel/ https://github.com/tweag/nickel/ See esp.: https://github.com/tweag/nickel/#related-projects-and-inspirations https://github.com/tweag/nickel/#related-projects-and-inspir...

ducktective 5y ago

About point 3: I don't think we should just ignore the immense technological knowledge encoded in C/C++ projects throughout the decades, only because they came from an era which having a package manager for every language was not fad of the day. Let's see how those "memory safe" languages would fare 20 years down the line...but I predict there would be a cool_lang and people would advocate for rewriting every algorithm known to man, yet again

nix23 5y ago
Not ignoring, but minimizing the surface, for example: Should the default "webserver" really be written in c? Or maybe just the one for embedded or high performance?
- ducktective 5y ago
  Yeah, best language for the task...but the parent was talking about not using a dependency just because it is written in C or C++
  - nix23 5y ago
    No, he was talking about "minimizing" c/c++ dependencies.
throwaway894345 5y ago
I sympathize with this point, and I tried to be quite clear that I'm not suggesting we drop everything cavalierly today, but rather we decide that our aspiration is to deprecate C/C++ and we move our dependencies gradually from C-based to something else. Some dependencies will be more pernicious to move because they require a lot of specialty knowledge (e.g., Harfbuzz) or simply due to their size (e.g., Chromium) but many others could be moved more readily. It will be a long, hard road which is why we should start now. > Let's see how those "memory safe" languages would fare 20 years down the line...but I predict there would be a cool_lang and people would advocate for rewriting every algorithm known to man, yet again This just reads like you've taken personal offense. My argument is quite clearly not "C isn't cool enough" or "C isn't memory safe", it's "C lacks a standard, sane build system such that packaging C projects is a nightmare".

nix23 5y ago

To your point 3, i don't think it's too controversial, some tools are absolutely ok being written in c like all the core-utils, but a monolithic kernel, written in c is probably the biggest problem: https://www.cvedetails.com/product/47/Linux-Linux-Kernel.html?vendor_id=33 https://www.cvedetails.com/product/47/Linux-Linux-Kernel.htm...

olodus 5y ago

I haven't tried it myself yet but Dhall is able to output nix files which then makes it possible to write with types in Dhall and then output the nix files (without types annotations) when they would be used.

Chris2048 5y ago

> you'll still spend tons of time trying to package obscure C dependencies but my question would be: is this exactly the same issue that all other package maintainers have e.g. rpm, deb etc?