4 ms·

With Eelco as a cofounder of DetSys, I feel really excited about where we're going here. I think the world is in many ways primed and ready for Nix, as long as

by grhmc 4y ago

With Eelco as a cofounder of DetSys, I feel really excited about where we're going here. I think the world is in many ways primed and ready for Nix, as long as we can help Nix "meet them in the middle." We're working on making Nix more accessible and producing good and usable, production-ready workflows so teams can just pick it up and go. I'd love to hear what y'all think, to help make sure we're going in the right direction.

nbp 4y ago
My thinking is that this is long overdue! I am working on a very large project where multiple solutions have been invented to solve problem which are all handled by Nix: - Bake in the repository the clone of the dependencies. - Compile the tool chain to create reproducible builds. - Create "artifact" builds, to avoid large recompilation times for many developers. - Use various tools to cache and distribute builds. - Use a bleeding-edge version of some dependencies. - Use Node, Java (Android), C++, Rust, … If only Nix were to run natively on Windows, this would check all the boxes.
- bpye 4y ago
  There was some work being done to make Nix work on Windows [0] - but it might have fizzled out. I too would really love Nix to work on Windows. I already use it on both macOS and Linux and it’s great. [0] - https://github.com/nix-windows/nix https://github.com/nix-windows/nix
tadfisher 4y ago
I might suggest making a typed superset of Nix, basically adding record types and function signatures. That in itself would be a huge boon for tooling and interacting with nixpkgs. nixpkgs describes a huge and varied meta-API for derivations in various build ecosystems, which makes it hard to use because you ultimately need to dive into source code.
- yewenjie 4y ago
  https://github.com/tweag/nickel https://github.com/tweag/nickel
- grhmc 4y ago
  We've heard this feedback pretty frequently. We have a few ideas of ways to add some types to Nix in place. I think even simple naive approaches would add a lot of safety and improve the usability of Nix. However, one of the bigger problems here is the general API design of Nixpkgs. Most of the function interfaces use some named parameters and then use `...` to accept any additional parameters. All of those then get passed down to some other function, and on and on. This makes a mess! It is hard to tell what functions use what parameters, and none of them can restrict their interface. We could see some significant DX here by creating smaller, more specific interfaces that actually restrict their inputs.
  - mikepurvis 4y ago
    I think the biggest win with some progressive typing would be better error diagnostics. Right now the errors are horrifyingly bad in many cases, puking out massive recursive stack traces that have nothing to do with the real problem. Being able to pinpoint exactly where the square peg is going into a round hole would be amazing.
- tazjin 4y ago
  Some community members have looked into this quite a bit, and personally I have come to the conclusion that types don't really give that much benefit to Nix (and I say this as a Rust & Haskell developer). Nix expressions are almost always[2] short-lived snippets that evaluate to a data structure, not long-running programs where the distinction between "static analysis time" and runtime is extremely relevant. In fact, deriving the full set of potentially relevant type constraints is only possible at runtime due to how the import system works and the shape of most expressions. If you pick a single file from nixpkgs without context, you can't get much information from it at all (other than a lot of unknown types being passed to things that accept an unknown type) - it's only in the context of an evaluation of a graph node that uses that file that you can infer anything meaningful about it (and even then that information is only relevant in that context). I think what people actually want is a way to derive more sensible documentation information from the code so that the various builders etc. could have checked documentation pages, as well as tooling for auto-completing members of attribute sets, names of (formals) function arguments and so on. fwiw, there's an alternative implementation of the Nix language[0] that we (TVL[1]) are slowly open-sourcing at the moment, and we're aiming to design it in such a way that things like an LSP can be implemented on top of it, and to be able to dump out static information from a context that can help with documentation etc. [0]: https://cs.tvl.fyi/depot/-/tree/tvix/eval https://cs.tvl.fyi/depot/-/tree/tvix/eval [1]: https://tvl.fyi/ https://tvl.fyi/ [2]: Yes, I'm aware that Nix is turing-complete, but surely nobody would write a web framework[3] and HTML templater[4] in it. [3]: https://cs.tvl.fyi/depot/-/blob/web/bubblegum/README.md https://cs.tvl.fyi/depot/-/blob/web/bubblegum/README.md [4]: https://cl.tvl.fyi/c/depot/+/3410 https://cl.tvl.fyi/c/depot/+/3410
  - hamandcheese 4y ago
    > If you pick a single file from nixpkgs without context, you can't get much information from it at all (other than a lot of unknown types being passed to things that accept an unknown type) - it's only in the context of an evaluation of a graph node that uses that file that you can infer anything meaningful about it (and even then that information is only relevant in that context). This is exactly where types would be the most beneficial, and what makes nix so hard to read. If a nix function could specify the types of its arguments, then I might have a chance of better understanding it. Just because most nix functions currently don’t have many constraints on their inputs doesn’t mean that they couldn’t in the future. > it's only in the context of an evaluation of a graph node that uses that file that you can infer anything meaningful about it (and even then that information is only relevant in that context) That’s how nixpkgs works now. But we can make functions that work only in more limited contexts.
setheron 4y ago
I wish this blog post had more meat to it. It was pretty minimal on information. This is also due to the fact that I tend to value Graham's posts highly :)
- biggestlou 4y ago
  Consider this post more of a "hello world." Substance is on the way :)
mamcx 4y ago
My main gripe is documentation. Look at this: https://nixos.wiki/wiki/PostgreSQL https://nixos.wiki/wiki/PostgreSQL Now, the real "workflow" with nix is look at other setups, or just look at the code: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/databases/postgresql.nix https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/s... And look at the code is the only reliable way to see what exactly is supported. My second gripe is the fact is hard to see what nix do. Today I hit this trouble: https://www.reddit.com/r/NixOS/comments/x46w98/why_new_user_is_not_added/ https://www.reddit.com/r/NixOS/comments/x46w98/why_new_user_... The thing is: Nixos not tell what is doing. `verbose` is too much noise. What I wish now is something like: nix change nothing (if my changes somehow don't do anything, like in my issue) nix setup users... nix setup postgresql
- Bolkan 4y ago
  Yeah, dealing with user directory is a pain, and it is a pain because you can't make it completely functional (what about passwords). And it is the only place I found where nix doesn't work as you would expect it to in my 3 years of nixos use.
  - colordrops 4y ago
    You can use agenix for secrets management. You can store all secrets encrypted with your SSH key in public config. The only non functional part is installing your SSH key, which is probably not a bad thing.
  - kaba0 4y ago
    Are you using home-manager for that?
- colordrops 4y ago
  I'm looking at your second link, and it's not really that bad if you are familiar with nix. It's a standard pattern for a package with options. Mostly a long enumerated list, not any different from a long enumerated list of the postgres options themselves. The great thing about it is that it's a single DSL, instead of 10,000 DSLs, one for each package you install. I do think command line options to do what you ask would be nice, but in order to keep the declarative reprudicibility you get with nix, those commands would need to operate on nix config rather than the system itself. They could update the config then update the system based on the config, so it could look like one step even though it is two behind the scenes.
flurie 4y ago
Hi, Graham, I have a few questions. Does Eelco's departure from Tweag have any impact on Nickel as a possible replacement for nix? There are lots of fragmented attempts at making nix easier/more immediately valuable/lower barrier to entry. Some of these are personal projects, some are businesses. Some of these just make nix easier, and some attempt to put something in between nix and the user. I'm thinking about things like Cachix, Flox, and divnix (formerly devos). How do you see your work interacting with these? One of my barriers to adoption at work has been ensuring maintenance continuity. That's always going to be the case for tools that aren't ubiquitous, but I worry with nix that teams without my assistance will revert to other tools they know better, even if I've invested a good bit of time in teaching them enough to keep things going. Do you seek to address that?

by grhmc 4y ago

nbp 4y ago

My thinking is that this is long overdue! I am working on a very large project where multiple solutions have been invented to solve problem which are all handled by Nix: - Bake in the repository the clone of the dependencies. - Compile the tool chain to create reproducible builds. - Create "artifact" builds, to avoid large recompilation times for many developers. - Use various tools to cache and distribute builds. - Use a bleeding-edge version of some dependencies. - Use Node, Java (Android), C++, Rust, … If only Nix were to run natively on Windows, this would check all the boxes.

bpye 4y ago
There was some work being done to make Nix work on Windows [0] - but it might have fizzled out. I too would really love Nix to work on Windows. I already use it on both macOS and Linux and it’s great. [0] - https://github.com/nix-windows/nix https://github.com/nix-windows/nix

tadfisher 4y ago

I might suggest making a typed superset of Nix, basically adding record types and function signatures. That in itself would be a huge boon for tooling and interacting with nixpkgs. nixpkgs describes a huge and varied meta-API for derivations in various build ecosystems, which makes it hard to use because you ultimately need to dive into source code.

yewenjie 4y ago
https://github.com/tweag/nickel https://github.com/tweag/nickel
grhmc 4y ago
We've heard this feedback pretty frequently. We have a few ideas of ways to add some types to Nix in place. I think even simple naive approaches would add a lot of safety and improve the usability of Nix. However, one of the bigger problems here is the general API design of Nixpkgs. Most of the function interfaces use some named parameters and then use `...` to accept any additional parameters. All of those then get passed down to some other function, and on and on. This makes a mess! It is hard to tell what functions use what parameters, and none of them can restrict their interface. We could see some significant DX here by creating smaller, more specific interfaces that actually restrict their inputs.
- mikepurvis 4y ago
  I think the biggest win with some progressive typing would be better error diagnostics. Right now the errors are horrifyingly bad in many cases, puking out massive recursive stack traces that have nothing to do with the real problem. Being able to pinpoint exactly where the square peg is going into a round hole would be amazing.
tazjin 4y ago
Some community members have looked into this quite a bit, and personally I have come to the conclusion that types don't really give that much benefit to Nix (and I say this as a Rust & Haskell developer). Nix expressions are almost always[2] short-lived snippets that evaluate to a data structure, not long-running programs where the distinction between "static analysis time" and runtime is extremely relevant. In fact, deriving the full set of potentially relevant type constraints is only possible at runtime due to how the import system works and the shape of most expressions. If you pick a single file from nixpkgs without context, you can't get much information from it at all (other than a lot of unknown types being passed to things that accept an unknown type) - it's only in the context of an evaluation of a graph node that uses that file that you can infer anything meaningful about it (and even then that information is only relevant in that context). I think what people actually want is a way to derive more sensible documentation information from the code so that the various builders etc. could have checked documentation pages, as well as tooling for auto-completing members of attribute sets, names of (formals) function arguments and so on. fwiw, there's an alternative implementation of the Nix language[0] that we (TVL[1]) are slowly open-sourcing at the moment, and we're aiming to design it in such a way that things like an LSP can be implemented on top of it, and to be able to dump out static information from a context that can help with documentation etc. [0]: https://cs.tvl.fyi/depot/-/tree/tvix/eval https://cs.tvl.fyi/depot/-/tree/tvix/eval [1]: https://tvl.fyi/ https://tvl.fyi/ [2]: Yes, I'm aware that Nix is turing-complete, but surely nobody would write a web framework[3] and HTML templater[4] in it. [3]: https://cs.tvl.fyi/depot/-/blob/web/bubblegum/README.md https://cs.tvl.fyi/depot/-/blob/web/bubblegum/README.md [4]: https://cl.tvl.fyi/c/depot/+/3410 https://cl.tvl.fyi/c/depot/+/3410
- hamandcheese 4y ago
  > If you pick a single file from nixpkgs without context, you can't get much information from it at all (other than a lot of unknown types being passed to things that accept an unknown type) - it's only in the context of an evaluation of a graph node that uses that file that you can infer anything meaningful about it (and even then that information is only relevant in that context). This is exactly where types would be the most beneficial, and what makes nix so hard to read. If a nix function could specify the types of its arguments, then I might have a chance of better understanding it. Just because most nix functions currently don’t have many constraints on their inputs doesn’t mean that they couldn’t in the future. > it's only in the context of an evaluation of a graph node that uses that file that you can infer anything meaningful about it (and even then that information is only relevant in that context) That’s how nixpkgs works now. But we can make functions that work only in more limited contexts.

setheron 4y ago

I wish this blog post had more meat to it. It was pretty minimal on information. This is also due to the fact that I tend to value Graham's posts highly :)

biggestlou 4y ago
Consider this post more of a "hello world." Substance is on the way :)

mamcx 4y ago

My main gripe is documentation. Look at this: https://nixos.wiki/wiki/PostgreSQL https://nixos.wiki/wiki/PostgreSQL Now, the real "workflow" with nix is look at other setups, or just look at the code: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/databases/postgresql.nix https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/s... And look at the code is the only reliable way to see what exactly is supported. My second gripe is the fact is hard to see what nix do. Today I hit this trouble: https://www.reddit.com/r/NixOS/comments/x46w98/why_new_user_is_not_added/ https://www.reddit.com/r/NixOS/comments/x46w98/why_new_user_... The thing is: Nixos not tell what is doing. `verbose` is too much noise. What I wish now is something like: nix change nothing (if my changes somehow don't do anything, like in my issue) nix setup users... nix setup postgresql

Bolkan 4y ago
Yeah, dealing with user directory is a pain, and it is a pain because you can't make it completely functional (what about passwords). And it is the only place I found where nix doesn't work as you would expect it to in my 3 years of nixos use.
- colordrops 4y ago
  You can use agenix for secrets management. You can store all secrets encrypted with your SSH key in public config. The only non functional part is installing your SSH key, which is probably not a bad thing.
- kaba0 4y ago
  Are you using home-manager for that?
colordrops 4y ago
I'm looking at your second link, and it's not really that bad if you are familiar with nix. It's a standard pattern for a package with options. Mostly a long enumerated list, not any different from a long enumerated list of the postgres options themselves. The great thing about it is that it's a single DSL, instead of 10,000 DSLs, one for each package you install. I do think command line options to do what you ask would be nice, but in order to keep the declarative reprudicibility you get with nix, those commands would need to operate on nix config rather than the system itself. They could update the config then update the system based on the config, so it could look like one step even though it is two behind the scenes.

flurie 4y ago

Hi, Graham, I have a few questions. Does Eelco's departure from Tweag have any impact on Nickel as a possible replacement for nix? There are lots of fragmented attempts at making nix easier/more immediately valuable/lower barrier to entry. Some of these are personal projects, some are businesses. Some of these just make nix easier, and some attempt to put something in between nix and the user. I'm thinking about things like Cachix, Flox, and divnix (formerly devos). How do you see your work interacting with these? One of my barriers to adoption at work has been ensuring maintenance continuity. That's always going to be the case for tools that aren't ubiquitous, but I worry with nix that teams without my assistance will revert to other tools they know better, even if I've invested a good bit of time in teaching them enough to keep things going. Do you seek to address that?