8 ms·

Nix Language Primer

122 points by binaryphile 8y ago

Harkins 8y ago
Does anyone know the rationale for creating Nixlang? Guix's use of Scheme proves there isn't a novel feature unavailable elsewhere, so it seems like a lot of wasted effort to implement a language that will likely only ever be used for one suite of programs. (And tooling; though almost none exists now, making the choice even more expensive.) I've tried to find one, but "nix" is a difficult thing to google for given the couple decades people have used it as a catchall term for unix and unix-like operating systems.
- clhodapp 8y ago
  I believe the motivation is that it eschews a lot of convenient crutches that would defeat the purpose of using nix (e.g. impurity) and adds a lot of convenient crutches that don't (e.g. a lot of implicit coercion for derivations).
  - rekado 8y ago
    Purity is a property of the functional methodology, not of the language. You don't need to use an effect-free language when operating in an environment that renders most effects void.
- catern 8y ago
  The Nix thesis may give some insight: https://nixos.org/~eelco/pubs/phd-thesis.pdf https://nixos.org/~eelco/pubs/phd-thesis.pdf Guix's use of Guile is fundamentally equivalent to the Nix package manager's use of the Nix language, but the approach Guix takes to organizing packages is very different. IMO, the Nix language approach is cleaner and more elegant when it comes to describing packages, but it's not clear whether that's worth the cost of using a domain-specific language. So, it still remains to be seen what's the best approach.
  - rekado 8y ago
    In Guix there are first class package objects that have references to other first class packages. Together they form a lazily evaluated graph. At a lower level, package objects can be compiled into one or many derivations, which is where things start to look more similar to Nix again. In Nix the idea of "functional package management" is more visible as there are no packages but functions with arguments that would result in a package once evaluated.
    - catern 8y ago
      Yep, Guix package objects with references to other package objects, versus Nix functions with arguments, is what I was referring to.
- cwp 8y ago
  I think one of the keys features is lazy evaluation. Nixpkgs is one giant expression that evaluates to the complete set of packages that it provides. But since only some the attributes of that set are evaluated in any given invocation, it's still efficient. That can be vertical (eg, a package and it's dependencies) or horizontal (eg, the names of all the packages). Guix shows that this isn't the only way to do it, but it is a good way.
  - rekado 8y ago
    Guix packages are also evaluated lazily. Package objects (values of the `<package>` type) can have an arbitrary number of declared inputs, which are package objects as well. These are not evaluated eagerly, of course. Lazy evaluation does not have to be a language feature. In the case of Guix's `<package>` record, only some of the fields are delayed or thunked.
    - zimbatm 8y ago
      The difference is that the lazy evaluation needs to be baked in. There are many scenarios in the nixpkgs code-base where lazy evaluation is used on a not-package level. For example the NixOS module system depends heavily on it. I haven't used Guix to be able to compare but it saved my life many times. I suspect the advantage for Guix is that it forces things to be better structured.
- mbrock 8y ago
  I like Scheme, but I think a lot of people actually wouldn’t want to write their packages and OS configuration files as S-expressions. Nix is an extremely simple language with quite familiar syntax, a kind of JSON with functions and string interpolation. Note also that Guix uses Scheme a lot more deeply than Nix uses the Nix language, in the sense that Nix uses e.g. shell scripts where Guix uses Scheme statements. Actual “coding” in the Nix language is relatively rare.
  - akavel 8y ago
    For me personally, as a person who's tried Nix, it's helpful that the Nix language is simple, self contained and easy to learn & grok quickly. I never learnt Scheme, and its use in Guix, though extremely interesting and appealing in theory, also feels notably overwhelming to me. That I'd have to learn a whole (presumably huge) R7RS or something, just to be able to use Guix. And then still have to learn the Guix "API" or DSL over that to be able to actually use it. While the Nix language is small and fully described in the Nix manual, in surprisingly few words.
    - rekado 8y ago
      Disclaimer: as a Guix co-maintainer I'm totally biased. We don't use R7RS in Guix. You need to know about the Scheme syntax, obviously (including keyword arguments), and a couple of common procedures like `string-append`, but aside from that you don't really need to know much about Scheme at all. What comes in handy is the Guix DSL, which provides a convenient way to specify packages and download origins. Guix also has a bunch of utility procedures that are useful extensions to their Scheme counterparts, such as `mkdir-p` (which does what you think it does) or the `substitute*` form to substitute expressions in a file or list of files. One important difference between Nix and Guix is that Guix does not glue shell snippets together, but eventually compiles to Guile builder scripts, so it's Scheme all the way down.
      - mbrock 8y ago
        Does Guix let you write interactive programs in Scheme as part of the system configuration? For example, defining a systemd service that uses Guile libraries and so on, as a subexpression of your OS configuration file? While also referring to shared variables like the system's hostname, etc? I skimmed the paper on "Code Staging in Guix" and I think this should be very doable, but I haven't yet tried Guix for real. This ability seems like it would have huge implications for system development... I've dabbled with such experiments using Nix, but the lack of hygienic code staging makes it a bit icky.
- otabdeveloper2 8y ago
  Nobody wants an arbitrary complex program for describing some trivial build steps. A programming language here is an anti-feature. Ideal would be some DSL that isn't even Turing complete, but that's not practical at the moment. Maybe we'll get there some day.
  - k_bx 8y ago
    Dhall and Dhall-nix are definitely a step towards that.
  - mbrock 8y ago
    It's difficult to not be Turing complete. Makefiles are Turing complete: https://nullprogram.com/blog/2016/04/30/ https://nullprogram.com/blog/2016/04/30/
  - X6S1x6Okd1st 8y ago
    Really what no one wants is a build step that takes forever. I'm more upset about how long nix-env -i takes than what the computational complexity of the programming language is
    - Nullabillity 8y ago
      nix-env -i is slow because it has to evaluate every package definition. nix-env -iA is way faster.
  - joepie91_ 8y ago
    The ability to build abstractions is actually one of the key points of what makes Nix work, conceptually. So yes, a 'programming language' (how are you defining that anyway?) is absolutely a feature here.
- solatic 8y ago
  Do yourself a favor and watch Gabriel Gonzalez's talk "Nix: Under the Hood" ( https://youtu.be/GMQPzv3Sx58 https://youtu.be/GMQPzv3Sx58 ).
- deleted 8y ago
  [deleted]
- karmakaze 8y ago
  This is the first I've heard of Nix. A quick search reminded me that it has nothing to do with Nim, and turned up NixOS that I have heard of. Here's a good description of why I should care: https://yakking.branchable.com/posts/what-and-why-nix/ https://yakking.branchable.com/posts/what-and-why-nix/
r32a_ 8y ago
I've had the displeasure of being forced to use nix and to me it seems like an overengineered piece of garbage. I don't see why anyone should use this vs competitors
- kjeetgill 8y ago
  Huh. I think this is the first bad experience with NIX I've ever heard on HN. Having only ever heard about it in passing but never worked with it myself I'm curious, what were you building when you'd encountered it and what issues did you have. Was there a particular language you'd preferred to have worked in?
  - josteink 8y ago
    > Huh. I think this is the first bad experience with NIX I've ever heard on HN. Strictly speaking "Nix" may refer to two things, and I'm not sure which thing either of you are referring to, nor if you two are referring to the same thing. Personally I found the nix package-manager a bit odd, but to me that was not a deal-breaker for trying out NixOS. The bigeset problem for me was lacking packages... I was however willing to help the community out by trying to package the things I needed which wasn't already provided for. But then I had to use the Nix language. And that was a great displeasure, and I ended giving up NixOS all together because of this one pain point. And to address your comment: I think I've seen similar comments on HN at regular intervals. It's definitely not a one-off comment. If they had used another more standard language (like LISP!) I would probably still be on NixOS as my main OS. In the long run, I really hope Guix manages to get something good going. My biggest concern for Guix is that they will restrict the available packages by being overly zealous about package-licensing (like everything the FSF does).
    - danieldk 8y ago
      But then I had to use the Nix language. And that was a great displeasure, and I ended giving up NixOS all together because of this one pain point. Could you articulate why? I have only been using Nix for ~3 months, but so far I do not really have qualms with the Nix language. Maybe it is because I have written a fair bit of Haskell before. (Of course, I understand the argument that it would have been more comfortable if an existing language like Scheme was used.) My biggest concern for Guix is that they will restrict the available packages by being overly zealous a bout package-licensing (like everything the FSF does). Same here. When I got interested in NixOS/GuixSD I seriously considered using GuixSD, because Scheme is more familiar. But there are just some proprietary packages that I need for work (Skype) and at home. I didn't really feel like having to rely on third-party repositories, etc.
malkia 8y ago
There is also https://jsonnet.org/ https://jsonnet.org/
- arianvanp 8y ago
  Yep they're pretty similar in goals, (and syntax?) except for that nix has built in support for derivations as a language feature. Nix is a language for derivations first a language for configurations second. Jsonnet is only a configuration language.
spacesuitman2 8y ago
Why the need for another new language? Why not just use Guile or Scheme instead?
- k__ 8y ago
  Probably the same reason why Guile or Scheme didn't get as wide-spread as other languages in general.
  - adev_ 8y ago
    Because nobody want to program in (((( Guile )))), to the point that even GDB disable its support as extension language.
    - spacesuitman2 8y ago
      However undesirable it appears at first, it does prevent the creation of an infinite number of new, incompatible languages. Config languages tend toward turing completeness and just become turing tarpits. That's why I'm asking about using a Lisp which is extendable to its problem domain. Source on gdb disabling it? Just writing `gr` in gdb enters the guile repl.
      - adev_ 8y ago
        It has been disable by default and is not even compiled-in anymore on all major linux distributions ( RHEL, Debian, ubuntu ). >(gdb) gr Guile scripting is not supported in this copy of GDB.
        spacesuitman2 8y ago
        What's your version? I'm on 8.2
  - dorfsmay 8y ago
    And what would that be? Because it is lispy? Or because of the restrictive GNU licenses?
zimbatm 8y ago
A few correction on the otherwise good article: * Nix 2.0 includes the repl. Just run `nix repl` * The URL literal returns a string * It would be good to explain how the `<...>` notation resolves paths from the NIX_PATH. For example `<nixpkgs>` looks for the `nixpkgs` key in `NIX_PATH=nixpkgs=/path/to/channel`
- binaryphile 8y ago
  Thanks for the tips. We're still running on 1.11 because of problems with the recent glibc updates not matching the kernels in our deployment environment.
- binaryphile 8y ago
  Finally got around to updating the post with your suggestions. Thanks again.
deleted 8y ago
[deleted]
nwmcsween 8y ago
I don't understand why any languages exist for describing packages, its a layer above describing real dependencies like with make, ninja, etc. Just have an os written in something like premake but with a nicer DSL.
- lallysingh 8y ago
  Make & ninja don't support two key activities: - Versioning - Getting the package (the actual URL, deciding if you want the package, determining which variant -- sometimes two packages solve the same dependency, etc.)
- zimbatm 8y ago
  It allows to have some guarantees around the evaluation. The Nix expression language enforces things like not allowing arbitrary network access during evaluation. This is hard to do in a general-purpose language where network and file access are usually backed in.
  - nwmcsween 8y ago
    This is easily done by chain-loading and system()
rufugee 8y ago
I switched all of my machines to NixOS for a few months. It was a wonderful experience being able to keep all my environments in sync. Nix the language is one reason I reverted to Ubuntu a few weeks ago. It just didn't mesh well with me....so when the inevitable problem occurred, it was a pain to deal with. Other pain points: - the NixOS learning curve is steep. There are two many tools with too many obscure options with sometimes overlapping features. - small group of maintainers with no real documented security process...just an assurance that they path as quickly as possible - many packages are old if they're not mainstream I would love to see the concept of Nix reimplemented in a non-niche language without the fsf zealotry... I'd switch my company to something like this in a heartbeat. I think there's a real business opportunity for someone willing to take that on.
- zzzcpan 8y ago
  Imagine if all the right ideas from nix were taken out and combined with supervision trees, we would have super reliable distros and PaaS style one click infrastructure.

122 points by binaryphile 8y ago

Does anyone know the rationale for creating Nixlang? Guix's use of Scheme proves there isn't a novel feature unavailable elsewhere, so it seems like a lot of wasted effort to implement a language that will likely only ever be used for one suite of programs. (And tooling; though almost none exists now, making the choice even more expensive.) I've tried to find one, but "nix" is a difficult thing to google for given the couple decades people have used it as a catchall term for unix and unix-like operating systems.

clhodapp 8y ago
I believe the motivation is that it eschews a lot of convenient crutches that would defeat the purpose of using nix (e.g. impurity) and adds a lot of convenient crutches that don't (e.g. a lot of implicit coercion for derivations).
- rekado 8y ago
  Purity is a property of the functional methodology, not of the language. You don't need to use an effect-free language when operating in an environment that renders most effects void.
catern 8y ago
The Nix thesis may give some insight: https://nixos.org/~eelco/pubs/phd-thesis.pdf https://nixos.org/~eelco/pubs/phd-thesis.pdf Guix's use of Guile is fundamentally equivalent to the Nix package manager's use of the Nix language, but the approach Guix takes to organizing packages is very different. IMO, the Nix language approach is cleaner and more elegant when it comes to describing packages, but it's not clear whether that's worth the cost of using a domain-specific language. So, it still remains to be seen what's the best approach.
- rekado 8y ago
  In Guix there are first class package objects that have references to other first class packages. Together they form a lazily evaluated graph. At a lower level, package objects can be compiled into one or many derivations, which is where things start to look more similar to Nix again. In Nix the idea of "functional package management" is more visible as there are no packages but functions with arguments that would result in a package once evaluated.
  - catern 8y ago
    Yep, Guix package objects with references to other package objects, versus Nix functions with arguments, is what I was referring to.
cwp 8y ago
I think one of the keys features is lazy evaluation. Nixpkgs is one giant expression that evaluates to the complete set of packages that it provides. But since only some the attributes of that set are evaluated in any given invocation, it's still efficient. That can be vertical (eg, a package and it's dependencies) or horizontal (eg, the names of all the packages). Guix shows that this isn't the only way to do it, but it is a good way.
- rekado 8y ago
  Guix packages are also evaluated lazily. Package objects (values of the `<package>` type) can have an arbitrary number of declared inputs, which are package objects as well. These are not evaluated eagerly, of course. Lazy evaluation does not have to be a language feature. In the case of Guix's `<package>` record, only some of the fields are delayed or thunked.
  - zimbatm 8y ago
    The difference is that the lazy evaluation needs to be baked in. There are many scenarios in the nixpkgs code-base where lazy evaluation is used on a not-package level. For example the NixOS module system depends heavily on it. I haven't used Guix to be able to compare but it saved my life many times. I suspect the advantage for Guix is that it forces things to be better structured.
mbrock 8y ago
I like Scheme, but I think a lot of people actually wouldn’t want to write their packages and OS configuration files as S-expressions. Nix is an extremely simple language with quite familiar syntax, a kind of JSON with functions and string interpolation. Note also that Guix uses Scheme a lot more deeply than Nix uses the Nix language, in the sense that Nix uses e.g. shell scripts where Guix uses Scheme statements. Actual “coding” in the Nix language is relatively rare.
- akavel 8y ago
  For me personally, as a person who's tried Nix, it's helpful that the Nix language is simple, self contained and easy to learn & grok quickly. I never learnt Scheme, and its use in Guix, though extremely interesting and appealing in theory, also feels notably overwhelming to me. That I'd have to learn a whole (presumably huge) R7RS or something, just to be able to use Guix. And then still have to learn the Guix "API" or DSL over that to be able to actually use it. While the Nix language is small and fully described in the Nix manual, in surprisingly few words.
  - rekado 8y ago
    Disclaimer: as a Guix co-maintainer I'm totally biased. We don't use R7RS in Guix. You need to know about the Scheme syntax, obviously (including keyword arguments), and a couple of common procedures like `string-append`, but aside from that you don't really need to know much about Scheme at all. What comes in handy is the Guix DSL, which provides a convenient way to specify packages and download origins. Guix also has a bunch of utility procedures that are useful extensions to their Scheme counterparts, such as `mkdir-p` (which does what you think it does) or the `substitute*` form to substitute expressions in a file or list of files. One important difference between Nix and Guix is that Guix does not glue shell snippets together, but eventually compiles to Guile builder scripts, so it's Scheme all the way down.
    - mbrock 8y ago
      Does Guix let you write interactive programs in Scheme as part of the system configuration? For example, defining a systemd service that uses Guile libraries and so on, as a subexpression of your OS configuration file? While also referring to shared variables like the system's hostname, etc? I skimmed the paper on "Code Staging in Guix" and I think this should be very doable, but I haven't yet tried Guix for real. This ability seems like it would have huge implications for system development... I've dabbled with such experiments using Nix, but the lack of hygienic code staging makes it a bit icky.
otabdeveloper2 8y ago
Nobody wants an arbitrary complex program for describing some trivial build steps. A programming language here is an anti-feature. Ideal would be some DSL that isn't even Turing complete, but that's not practical at the moment. Maybe we'll get there some day.
- k_bx 8y ago
  Dhall and Dhall-nix are definitely a step towards that.
- mbrock 8y ago
  It's difficult to not be Turing complete. Makefiles are Turing complete: https://nullprogram.com/blog/2016/04/30/ https://nullprogram.com/blog/2016/04/30/
- X6S1x6Okd1st 8y ago
  Really what no one wants is a build step that takes forever. I'm more upset about how long nix-env -i takes than what the computational complexity of the programming language is
  - Nullabillity 8y ago
    nix-env -i is slow because it has to evaluate every package definition. nix-env -iA is way faster.
- joepie91_ 8y ago
  The ability to build abstractions is actually one of the key points of what makes Nix work, conceptually. So yes, a 'programming language' (how are you defining that anyway?) is absolutely a feature here.
solatic 8y ago
Do yourself a favor and watch Gabriel Gonzalez's talk "Nix: Under the Hood" ( https://youtu.be/GMQPzv3Sx58 https://youtu.be/GMQPzv3Sx58 ).
deleted 8y ago
[deleted]
karmakaze 8y ago
This is the first I've heard of Nix. A quick search reminded me that it has nothing to do with Nim, and turned up NixOS that I have heard of. Here's a good description of why I should care: https://yakking.branchable.com/posts/what-and-why-nix/ https://yakking.branchable.com/posts/what-and-why-nix/

r32a_ 8y ago

I've had the displeasure of being forced to use nix and to me it seems like an overengineered piece of garbage. I don't see why anyone should use this vs competitors

kjeetgill 8y ago
Huh. I think this is the first bad experience with NIX I've ever heard on HN. Having only ever heard about it in passing but never worked with it myself I'm curious, what were you building when you'd encountered it and what issues did you have. Was there a particular language you'd preferred to have worked in?
- josteink 8y ago
  > Huh. I think this is the first bad experience with NIX I've ever heard on HN. Strictly speaking "Nix" may refer to two things, and I'm not sure which thing either of you are referring to, nor if you two are referring to the same thing. Personally I found the nix package-manager a bit odd, but to me that was not a deal-breaker for trying out NixOS. The bigeset problem for me was lacking packages... I was however willing to help the community out by trying to package the things I needed which wasn't already provided for. But then I had to use the Nix language. And that was a great displeasure, and I ended giving up NixOS all together because of this one pain point. And to address your comment: I think I've seen similar comments on HN at regular intervals. It's definitely not a one-off comment. If they had used another more standard language (like LISP!) I would probably still be on NixOS as my main OS. In the long run, I really hope Guix manages to get something good going. My biggest concern for Guix is that they will restrict the available packages by being overly zealous about package-licensing (like everything the FSF does).
  - danieldk 8y ago
    But then I had to use the Nix language. And that was a great displeasure, and I ended giving up NixOS all together because of this one pain point. Could you articulate why? I have only been using Nix for ~3 months, but so far I do not really have qualms with the Nix language. Maybe it is because I have written a fair bit of Haskell before. (Of course, I understand the argument that it would have been more comfortable if an existing language like Scheme was used.) My biggest concern for Guix is that they will restrict the available packages by being overly zealous a bout package-licensing (like everything the FSF does). Same here. When I got interested in NixOS/GuixSD I seriously considered using GuixSD, because Scheme is more familiar. But there are just some proprietary packages that I need for work (Skype) and at home. I didn't really feel like having to rely on third-party repositories, etc.

malkia 8y ago

There is also https://jsonnet.org/ https://jsonnet.org/

arianvanp 8y ago
Yep they're pretty similar in goals, (and syntax?) except for that nix has built in support for derivations as a language feature. Nix is a language for derivations first a language for configurations second. Jsonnet is only a configuration language.

spacesuitman2 8y ago

Why the need for another new language? Why not just use Guile or Scheme instead?

k__ 8y ago
Probably the same reason why Guile or Scheme didn't get as wide-spread as other languages in general.
- adev_ 8y ago
  Because nobody want to program in (((( Guile )))), to the point that even GDB disable its support as extension language.
  - spacesuitman2 8y ago
    However undesirable it appears at first, it does prevent the creation of an infinite number of new, incompatible languages. Config languages tend toward turing completeness and just become turing tarpits. That's why I'm asking about using a Lisp which is extendable to its problem domain. Source on gdb disabling it? Just writing `gr` in gdb enters the guile repl.
    - adev_ 8y ago
      It has been disable by default and is not even compiled-in anymore on all major linux distributions ( RHEL, Debian, ubuntu ). >(gdb) gr Guile scripting is not supported in this copy of GDB.
      - spacesuitman2 8y ago
        What's your version? I'm on 8.2
- dorfsmay 8y ago
  And what would that be? Because it is lispy? Or because of the restrictive GNU licenses?

zimbatm 8y ago

A few correction on the otherwise good article: * Nix 2.0 includes the repl. Just run `nix repl` * The URL literal returns a string * It would be good to explain how the `<...>` notation resolves paths from the NIX_PATH. For example `<nixpkgs>` looks for the `nixpkgs` key in `NIX_PATH=nixpkgs=/path/to/channel`

binaryphile 8y ago
Thanks for the tips. We're still running on 1.11 because of problems with the recent glibc updates not matching the kernels in our deployment environment.
binaryphile 8y ago
Finally got around to updating the post with your suggestions. Thanks again.

deleted 8y ago

[deleted]

nwmcsween 8y ago

I don't understand why any languages exist for describing packages, its a layer above describing real dependencies like with make, ninja, etc. Just have an os written in something like premake but with a nicer DSL.

lallysingh 8y ago
Make & ninja don't support two key activities: - Versioning - Getting the package (the actual URL, deciding if you want the package, determining which variant -- sometimes two packages solve the same dependency, etc.)
zimbatm 8y ago
It allows to have some guarantees around the evaluation. The Nix expression language enforces things like not allowing arbitrary network access during evaluation. This is hard to do in a general-purpose language where network and file access are usually backed in.
- nwmcsween 8y ago
  This is easily done by chain-loading and system()

rufugee 8y ago

I switched all of my machines to NixOS for a few months. It was a wonderful experience being able to keep all my environments in sync. Nix the language is one reason I reverted to Ubuntu a few weeks ago. It just didn't mesh well with me....so when the inevitable problem occurred, it was a pain to deal with. Other pain points: - the NixOS learning curve is steep. There are two many tools with too many obscure options with sometimes overlapping features. - small group of maintainers with no real documented security process...just an assurance that they path as quickly as possible - many packages are old if they're not mainstream I would love to see the concept of Nix reimplemented in a non-niche language without the fsf zealotry... I'd switch my company to something like this in a heartbeat. I think there's a real business opportunity for someone willing to take that on.

zzzcpan 8y ago
Imagine if all the right ideas from nix were taken out and combined with supervision trees, we would have super reliable distros and PaaS style one click infrastructure.