4 ms·

Having used nix for two years now for both work and personal purposes, I agree. Though I have also come to think nix is doing things in much better ways than Ap

Having used nix for two years now for both work and personal purposes, I agree. Though I have also come to think nix is doing things in much better ways than AppImage, Docker, Snap, Flatpack, and others. I won't focus on the good parts, but focus on your comment. Nix has three things that I think are confusing and took me an embarrassing long time to grok, and I find other people confused by as well. 1. Where does the semicolon go? Reading code makes it feel arbitrary, but really the only time a semicolon is used is to terminate an assignment (=). If there are no equal signs, then you don't need any semicolons. 2. The heredoc strings ('' '') are in fact strings. Nixpkgs passes these strings to a shell quite frequently and their docs also make it feel like these statements are shell scripts and not strings. The problem here is that to pull a variable from nix you use ${VAR} and to use an environment variable you use $VAR. When you look at these heredocs it looks like a pure bash script, but it doesn't quickly jump out at you that ${VAR} are being templated in by nix. It also isn't obvious that $VAR is not being templated in by nix. 3. The nix language is very very tiny, and almost everything comes from nixpkgs which is basically stdlib as well as the package repository. Almost everything, including callPackage is in nixpkgs. The documentation is fairly decent, but when trying to figure out how do something, you are basically hunting in that code base. Bonus 4. In nix it is really easy to build single language applications. However if you need to mix two, finding good docs or examples is really hard. IE, if you want to build the javascript front end of your web app as well as the python backend. These combinations need at least two derivations, and how to make it happen is bespoke every time.

siraben 6y ago
Regular Nix user here, I agree with your points regarding the language's peculiarities, and adding on a few points: > 1. Where does the semicolon go? Reading code makes it feel arbitrary, but really the only time a semicolon is used is to terminate an assignment (=). If there are no equal signs, then you don't need any semicolons. The semicolon is also required when using with expressions (i.e. augments the environment of an expression like let, but confusingly doesn't shadow let bindings) nix-repl> with { a = 3; }; a 3 nix-repl> let a = 5; in with { a = 3; }; a 5 # ?????? > it it doesn't quickly jump out at you that ${VAR} are being templated in by nix One situation I've run into is how to quote ${ in a multiline string, and the solution is to write ''${ : nix-repl> '' ''${ '' "${ " > The documentation is fairly decent, but when trying to figure out how do something, you are basically hunting in that code base. This is very much true, essentially Nix and its ecosystem is a big instance of Hyrum's Law[0] whereby people go with what they actually observe rather than what should be according to the spec (if it exists). [0] https://www.hyrumslaw.com/ https://www.hyrumslaw.com/
- raxi 6y ago
  `with` should be considered harmful. Well, you recognized the gotcha, but there are people who don't, they inserted a bug into nixpkgs, then someone else fixed it not realizing its genesis. There is a lot of spaghetti code in nixpkgs caused by misunderstanding of `with`
chriswarbo 6y ago
> the only time a semicolon is used is to terminate an assignment (=) I find it easier to think of "attribute sets" like `{ foo = bar; }` more like a JSON objects (e.g. `{ "foo": bar }`) rather than "assignment". Nix does have a `let` syntax, which probably looks more like assignment, but I've never used it in the 7 years I've been using Nix. I find `with myAttrs; myExpr` to be far better (except for the WontFix issue that `with` doesn't shadow function arguments). Off the top of my head, the following expressions also require `;`: with myAttrs; myBody assert myCondition; myBody The way I remember this is that every Nix expression returns a value; yet `with myAttrs` doesn't have a value, and neither does `assert myCondition`. Hence they have a separate return value "stuck on" via the `;`
- georgyo 6y ago
  > I find `with myAttrs; myExpr` To make it clearer, he is saying that he does with { foo = "bar"; }; <some other block> instead of let foo = "bar"; in <some other block> The differences here so other people can follow along. 1. with will not shadow variables defined higher up. This can be either desirable or very confusing. 2. let is recursive by default, meaning you can reference other variables in that block. This behavior can be mimicked with `with rec {}`
  - chriswarbo 6y ago
    Yep. The advantage of `with` is that we're dealing with first-class values (attribute sets, like `{ foo = "bar"; }`), rather than an implicit environment/context. My rationale is at https://github.com/NixOS/nix/issues/1361#issuecomment-390420439 https://github.com/NixOS/nix/issues/1361#issuecomment-390420... The thing about shadowing is that `with` will not shadow anything that's in the implicit context, which includes things defined by `let` (hence why I avoid it) and function arguments (unavoidable). We can shadow other `with` bindings as much as we like, e.g. with { foo = 1; }; with { foo = 2; }; with { foo = 3; }; with { bar = with { foo = 4; }; foo; }; [ foo bar ] This evaluates to [ 3 4 ], since later bindings to `foo` will shadow/override earlier ones. This doesn't work for bindings that are in the implicit environment/context, for example: with { f = foo: with { foo = 42; }; foo; }; f 123 This evaluates to 123, since the binding `with { foo = 42; }; ...` doesn't shadow/override the argument `foo`. The rationale for this behaviour is to avoid changes made to an attrset from overriding explicit arguments. For example: foo: with bar; x + foo This is presumably meant to behave like `foo: bar.x + foo`, but a future change to `bar` might include a `bar.foo` attribute, in which case we don't want the above to change behaviour to `foo: bar.x + bar.foo`. I understand this rationale, especially when using huge, constantly-changing attrsets like `with nixpkgs; ...`, but it's still annoying ;)
chriswarbo 6y ago
> Bonus 4. In nix it is really easy to build single language applications. However if you need to mix two, finding good docs or examples is really hard. IE, if you want to build the javascript front end of your web app as well as the python backend. These combinations need at least two derivations, and how to make it happen is bespoke every time. For anything that doesn't fit a common nixpkgs function (like `buildPythonPackage`, etc.) then I always start with this: nixpkgs.runCommand "my-application" { dependencies go here } '' shell commands go here '' This is certainly "bespoke every time", but it's literally just a shell script (with explicit dependencies). This approach doesn't need much documentation either: we just copy-paste whatever commands are in the normal, non-Nix documentation for whatever we're doing. We'll probably need to add a few dependencies to the `buildInputs`, and possibly disable the sandbox to allow network access (which we can factor out later, if we like).
- georgyo 6y ago
  That is a gross oversimplification. An javascript front end can have thousands of dependencies, and the nix build environment does not have network access unless it's computing a fixed output. To do that manually would be a nightmare, so you must automated either with tools you write or tools someone else has written. Then you still have to figure out how to combine it all together. Also on allowing network access in the sandbox requires elevated privileges which means you cannot do that method on environments you don't have full control. And it would create builds that are likely not reproducible.
  - chriswarbo 6y ago
    > That is a gross oversimplification It's really not. I've used this successfully for Javascript projects (running npm and yarn), Python projects (running pip install), Scala projects (running sbt), Haskell projects (running cabal), Racket projects (running raco), Go projects (running go get), etc. It's pretty much like writing a Makefile, except we have to specify the buildInputs, and there are occasional annoyances like tools trying to write to $HOME (requiring an `export HOME="$PWD"` line at the top of the script). > the nix build environment does not have network access unless it's computing a fixed output Yes it does, if we avoid the sandbox. This used to be done by specifying a `__noChroot = true;` attribute; I don't know if that's changed in more recent Nix versions. > Also on allowing network access in the sandbox requires elevated privileges Sandboxing is a sensible default, but for practicality I usually set it to "relaxed" on NixOS and disable it entirely on macOS (since that often uses stuff from the host system). > which means you cannot do that method on environments you don't have full control. Could you give an example of where that would ever be an issue? The only thing I can think of is hydra.nixos.org, but there's no way in hell such a derivation would be acceptable for nixpkgs, so that's pretty irrelevant (similar to how it's really easy to make a .deb package from a directory and a control file; but there's no way in hell it complies with the Debian project's packaging guidelines) > And it would create builds that are likely not reproducible. Yes, that's why I mentioned we might want to factor out the input-fetching at some point. > Then you still have to figure out how to combine it all together. That has nothing to do with Nix. As long as a project has a build script, or a Makefile, or a list of commands in a README, or whatever, then we can plug it into that template and see if it works. Are you complaining that the Nix package manager doesn't have specific documentation for how to build every random application ever written?
chriswarbo 6y ago
Regarding point 2, yes it's definitely a pain to mentally switch back-and-forth when writing shell code in a Nix string. Three things I find immensely helpful: - Syntax highlighting (I use Emacs, which has a Nix mode) - Syntax checking (again I use Emacs for this) - Writing complicated scripts in a separate file. This makes it harder to splice in Nix values (there are ways to substitute in text, but I find them more trouble than they're worth). Setting some extra env vars in the build environment is usually the easiest approach.

siraben 6y ago

Regular Nix user here, I agree with your points regarding the language's peculiarities, and adding on a few points: > 1. Where does the semicolon go? Reading code makes it feel arbitrary, but really the only time a semicolon is used is to terminate an assignment (=). If there are no equal signs, then you don't need any semicolons. The semicolon is also required when using with expressions (i.e. augments the environment of an expression like let, but confusingly doesn't shadow let bindings) nix-repl> with { a = 3; }; a 3 nix-repl> let a = 5; in with { a = 3; }; a 5 # ?????? > it it doesn't quickly jump out at you that ${VAR} are being templated in by nix One situation I've run into is how to quote ${ in a multiline string, and the solution is to write ''${ : nix-repl> '' ''${ '' "${ " > The documentation is fairly decent, but when trying to figure out how do something, you are basically hunting in that code base. This is very much true, essentially Nix and its ecosystem is a big instance of Hyrum's Law[0] whereby people go with what they actually observe rather than what should be according to the spec (if it exists). [0] https://www.hyrumslaw.com/ https://www.hyrumslaw.com/

raxi 6y ago
`with` should be considered harmful. Well, you recognized the gotcha, but there are people who don't, they inserted a bug into nixpkgs, then someone else fixed it not realizing its genesis. There is a lot of spaghetti code in nixpkgs caused by misunderstanding of `with`

chriswarbo 6y ago

> the only time a semicolon is used is to terminate an assignment (=) I find it easier to think of "attribute sets" like `{ foo = bar; }` more like a JSON objects (e.g. `{ "foo": bar }`) rather than "assignment". Nix does have a `let` syntax, which probably looks more like assignment, but I've never used it in the 7 years I've been using Nix. I find `with myAttrs; myExpr` to be far better (except for the WontFix issue that `with` doesn't shadow function arguments). Off the top of my head, the following expressions also require `;`: with myAttrs; myBody assert myCondition; myBody The way I remember this is that every Nix expression returns a value; yet `with myAttrs` doesn't have a value, and neither does `assert myCondition`. Hence they have a separate return value "stuck on" via the `;`

georgyo 6y ago
> I find `with myAttrs; myExpr` To make it clearer, he is saying that he does with { foo = "bar"; }; <some other block> instead of let foo = "bar"; in <some other block> The differences here so other people can follow along. 1. with will not shadow variables defined higher up. This can be either desirable or very confusing. 2. let is recursive by default, meaning you can reference other variables in that block. This behavior can be mimicked with `with rec {}`
- chriswarbo 6y ago
  Yep. The advantage of `with` is that we're dealing with first-class values (attribute sets, like `{ foo = "bar"; }`), rather than an implicit environment/context. My rationale is at https://github.com/NixOS/nix/issues/1361#issuecomment-390420439 https://github.com/NixOS/nix/issues/1361#issuecomment-390420... The thing about shadowing is that `with` will not shadow anything that's in the implicit context, which includes things defined by `let` (hence why I avoid it) and function arguments (unavoidable). We can shadow other `with` bindings as much as we like, e.g. with { foo = 1; }; with { foo = 2; }; with { foo = 3; }; with { bar = with { foo = 4; }; foo; }; [ foo bar ] This evaluates to [ 3 4 ], since later bindings to `foo` will shadow/override earlier ones. This doesn't work for bindings that are in the implicit environment/context, for example: with { f = foo: with { foo = 42; }; foo; }; f 123 This evaluates to 123, since the binding `with { foo = 42; }; ...` doesn't shadow/override the argument `foo`. The rationale for this behaviour is to avoid changes made to an attrset from overriding explicit arguments. For example: foo: with bar; x + foo This is presumably meant to behave like `foo: bar.x + foo`, but a future change to `bar` might include a `bar.foo` attribute, in which case we don't want the above to change behaviour to `foo: bar.x + bar.foo`. I understand this rationale, especially when using huge, constantly-changing attrsets like `with nixpkgs; ...`, but it's still annoying ;)

chriswarbo 6y ago

> Bonus 4. In nix it is really easy to build single language applications. However if you need to mix two, finding good docs or examples is really hard. IE, if you want to build the javascript front end of your web app as well as the python backend. These combinations need at least two derivations, and how to make it happen is bespoke every time. For anything that doesn't fit a common nixpkgs function (like `buildPythonPackage`, etc.) then I always start with this: nixpkgs.runCommand "my-application" { dependencies go here } '' shell commands go here '' This is certainly "bespoke every time", but it's literally just a shell script (with explicit dependencies). This approach doesn't need much documentation either: we just copy-paste whatever commands are in the normal, non-Nix documentation for whatever we're doing. We'll probably need to add a few dependencies to the `buildInputs`, and possibly disable the sandbox to allow network access (which we can factor out later, if we like).

georgyo 6y ago
That is a gross oversimplification. An javascript front end can have thousands of dependencies, and the nix build environment does not have network access unless it's computing a fixed output. To do that manually would be a nightmare, so you must automated either with tools you write or tools someone else has written. Then you still have to figure out how to combine it all together. Also on allowing network access in the sandbox requires elevated privileges which means you cannot do that method on environments you don't have full control. And it would create builds that are likely not reproducible.
- chriswarbo 6y ago
  > That is a gross oversimplification It's really not. I've used this successfully for Javascript projects (running npm and yarn), Python projects (running pip install), Scala projects (running sbt), Haskell projects (running cabal), Racket projects (running raco), Go projects (running go get), etc. It's pretty much like writing a Makefile, except we have to specify the buildInputs, and there are occasional annoyances like tools trying to write to $HOME (requiring an `export HOME="$PWD"` line at the top of the script). > the nix build environment does not have network access unless it's computing a fixed output Yes it does, if we avoid the sandbox. This used to be done by specifying a `__noChroot = true;` attribute; I don't know if that's changed in more recent Nix versions. > Also on allowing network access in the sandbox requires elevated privileges Sandboxing is a sensible default, but for practicality I usually set it to "relaxed" on NixOS and disable it entirely on macOS (since that often uses stuff from the host system). > which means you cannot do that method on environments you don't have full control. Could you give an example of where that would ever be an issue? The only thing I can think of is hydra.nixos.org, but there's no way in hell such a derivation would be acceptable for nixpkgs, so that's pretty irrelevant (similar to how it's really easy to make a .deb package from a directory and a control file; but there's no way in hell it complies with the Debian project's packaging guidelines) > And it would create builds that are likely not reproducible. Yes, that's why I mentioned we might want to factor out the input-fetching at some point. > Then you still have to figure out how to combine it all together. That has nothing to do with Nix. As long as a project has a build script, or a Makefile, or a list of commands in a README, or whatever, then we can plug it into that template and see if it works. Are you complaining that the Nix package manager doesn't have specific documentation for how to build every random application ever written?

chriswarbo 6y ago

Regarding point 2, yes it's definitely a pain to mentally switch back-and-forth when writing shell code in a Nix string. Three things I find immensely helpful: - Syntax highlighting (I use Emacs, which has a Nix mode) - Syntax checking (again I use Emacs for this) - Writing complicated scripts in a separate file. This makes it harder to splice in Nix values (there are ways to substitute in text, but I find them more trouble than they're worth). Setting some extra env vars in the build environment is usually the easiest approach.