6 ms·

Writing the packaging expressions should not be harder than writing the program, for the average developer. The average developer doesn't know FP, at this point

by Jeaye 4y ago

Writing the packaging expressions should not be harder than writing the program, for the average developer. The average developer doesn't know FP, at this point. Nix needs: 1. Approachability for those not indoctrinated in lazy, declarative, functional programming (i.e. Haskell); nope, Nix Pills are not sufficient for the average dev 2. Editor tooling to help guide the writing of expressions (just as anyone learning C# or Rust can use LSP); a better type system for Nix would help here 3. Better documentation for practical things like "Using Nix to manage the dependencies and package a C++ program using Meson", rather than having people piece this together from a bunch of disparate docs 4. A much better CLI UX than `nix-env -qa` and the like (this is ongoing and experimental, but even that broke recently, causing lots of confusion; now it requires `nix --extra-experimental-features nix-command --extra-experimental-features flakes search nixpkgs`) 5. To seriously answer the question: is the Nix language required for the Nix packaging system to exist? Laziness is required, to some degree, but can the next iteration provide an on-ramp which doesn't involve learning a new lang and paradigm? Guix folks sure think so. I feel like Nix folks have been focused so long on solving the tough problems of declarative, deterministic packaging that they haven't been able to focus on the UX. I also feel like folks for whom Haskell is comfortable may not realize just how absurd it feels to everyone else. Perhaps like the early days of Git. I do really hope Nix succeeds in this, though; I've been using it, or it's been using me, for several years. [1] and [2] for more info. [1]: https://blog.jeaye.com/2015/11/24/nixos/ https://blog.jeaye.com/2015/11/24/nixos/ [2]: https://blog.jeaye.com/2017/07/30/nixos-revisited/ https://blog.jeaye.com/2017/07/30/nixos-revisited/

Bolkan 4y ago
What nix needs more than anything is a centralized repository containing recipes for running all the different kinds of programs, as nic files. ie a dockerhub alternative.
- xyzzy_plugh 4y ago
  nixpkgs?
  - grumbel 4y ago
    Something a little more compact would be welcome. There is: nix flake show templates But the examples in there are a bit out of date and don't necessarily reflect real world usage and are lacking in comments (e.g. still using defaultPackage, not using flake-utils, etc.).
rgoulter 4y ago
> they haven't been able to focus on the UX. I also feel like folks for whom Haskell is comfortable may not realize just how absurd it feels to everyone else. The majority of communication I've seen from Nix maintainers over the last few months has been about exactly this. The submitted post is about exactly that while Nix powerful, they want more people to be able to use it without having to pay a steep penalty to learn it.
- tinco 4y ago
  It also doesn't have anything to do with Haskell or it looking somewhat like Haskell. I'm very comfortable in Haskell, and Nix looks awful to me. It's not that it's functional or it's declarative, it's that it's badly designed and not intuitive, making it hard to remember and therefore awkward to code in.
  - kaba0 4y ago
    I think the only hard part is not Nix, but Nixpkgs’s not well-documented/not-stable APIs. It would be a problem in any language.
techdragon 4y ago
Point 4 is what lost me when I tried using NixOS for a machine I explicitly wanted as reproducible as possible. I needed to get a bunch of stuff working, then freeze/bundle/lock the useland as a documented working state I could rebuild later if anything went wrong. When documented examples began steering me towards the still experimental nix flakes and using the sorts of with ever more flags required commands, I bailed and built it using salt. I want to like nix but the current state of things was a massive put off.
- ParetoOptimal 4y ago
  > When documented examples began steering me towards the still experimental nix flakes There's a push to make them not be experimental anymore since so much of the community including myself is using them.
- dial9-1 4y ago
  flakes are anything but experimental at this point, they've been out for many years and I pretty much use them for everything. you don't have to type those flags if you add them to the configuration file
  - mook 4y ago
    A newcomer is going to treat anything documented as experimental as that, regardless of what the community thinks; if it's no longer experimental, it needs to be presented as such at all the normal entry points. (None of this is unique to Nix; imagine you needed to start packaging for Windows for some reason.)
    - techdragon 4y ago
      This is exactly what I mean. It’s a big put off to see several year old experimental features and other signs that no one is making the decision to move things forward. If I’m learning something as big as nix with all the facets it has, I don’t also have time to learn it’s entire community history. It’s not a good thing that I need to do that in order to make a decision about how much to trust experimental features of a tool when I’ve not even been using it more than a few hours from the moment of downloading the iso and installing it. It’s a put off to someone new coming to use nix for the first time.
rssoconnor 4y ago
> To seriously answer the question: is the Nix language required for the Nix packaging system to exist? Laziness is required, to some degree, but can the next iteration provide an on-ramp which doesn't involve learning a new lang and paradigm? Guix folks sure think so. I'd love to hear from someone deeply familiar with Nix and Guix about laziness. I'm deeply familiar with Nix and I've concluded that lazy semantics is absolutely critical for a configuration language. It lets me refer to other attributes of my configuration from anywhere. For example, I can refer to port number from my whatever service in my firewall. Nix's system of overlays depends on laziness too to provide efficient late-binding familiar from OOP. I don't need to topologically sort the evaluation of the various inter-dependencies of my configuration. So long as there exists an evaluation order, laziness finds it. Laziness is compelling enough that I managed to convince the author of Jsonnet <https://jsonnet.org/ https://jsonnet.org/> of it when he was designing it, and in turn he helped me design what is now known as overlays in Nix. I don't even understand how Guix manages to work without laziness, though clearly it does somehow. I'm curious as to how that is possible, though I fear I will only ever truly understand by diving into Guix.
- infogulch 4y ago
  I'm not convinced that any of the things you mentioned as benefits of lazy evaluation are a necessary condition to getting those benefits.
  - WastingMyTime89 4y ago
    It’s not. Guix has exactly the same properties while using an eager language.
- mikepurvis 4y ago
  As an 18 month Nix packager/user, I continue to be blown away by how powerful laziness is and how expressive the interactions between multiple overlays can be.
- jackac 4y ago
  Recursion and merging semantics are also necessary. I use Nix and Jsonnet a lot, but Nix is much more expressive for complex structures, but the tooling being tightly coupled with the package manage make it impossible to adopt for common use cases.
tikhonj 4y ago
Having used Nix a bunch, the system has been hard to use and learn in exactly the same ways that messy legacy codebases I've worked with have been hard—which makes sense since Nixpkgs is an absolutely massive codebase that's evolved over the years with very distributed development. People latch onto the Nix language and the paradigm because that's the most tangible difference but, everything else held equal, Nix + Nixpkgs would be no easier to work with if everything were in Python. (And, if my limited experience is any guide, would be worse with an ad-hoc DSL pretending to be Python à la Starlark!) This is a pattern I've seen over and over: if a project tries to do something different, the novel aspect gets blamed for issues that would not be surprising in any other context. When faced with a massive awful codebase in Java, well, sometimes legacy code can be a pain, right? It is what it is. People rarely jump to "OOP is a mistake" or "Java is completely unusable". But that's exactly what happens all the time with, say, functional programming. Pointing to an uncommon choice as the underlying cause of a problem makes for a nice, pat story; the pervasive, diffuse problems inherent to messy legacy code do not. After all, here's a single thing that, presumably we can actually change! Nix being functional is an objective fact while people can—reasonably or not—disagree on how well-designed and well-written Nixpkgs-the-codebase is. Besides, even if we could agree, what could we do about it? We all know how much momentum legacy code has and how hard it is to make anything beyond local improvements. Being a socially convenient story doesn't mean it's a particularly strong explanation, but it does mean that people will believe it even if it isn't.
- qbasic_forever 4y ago
  Yeah IMHO nix just needs someone like Brian Kernighan to sit down and write the definitive book about it and its internals. Something that clearly explains the practical and theoretical uses of nix and nixpkgs. Nix pills, the official manual, etc. aren't really there yet--they talk about all kinds of paradigms that might (?) be going away like nix-env and its ilk. It's just extremely painful to track down the current best practice and way to do things in nix--you're bouncing off all kinds of docs and finding multiple conflicting or out of date ways to do things. Someone needs to give folks a slim book about nix flakes and the flakes way of doing things, the nix language, and a tour of nixpkgs, and IMHO there will be a lot more people using it.
seqizz 4y ago
> I also feel like folks for whom Haskell is comfortable may not realize just how absurd it feels to everyone else. Yup. I can't even imagine what could I accomplish if I'd have something like a "python library" for nixpkgs. I still use nixos but gave up fiddling with nix for a while, after trying to wrap some stuff I am using/developing.* * (Yes, I know it's easy. I just need flakes or correct amount of "lib: prev: { self"s. Sadly not for me.)
theptip 4y ago
Having just done some prototyping with Nix, this list is spot on in enumerating the pain points I hit. I think most of the docs problems could be solved with the meta-strategy of “deeply understand all the things the Rust ecosystem got right with docs, discovery, and general pedagogical structure, and replicate that”. Actually have a canonical set of docs. Link liberally between packafes and stdlib. Figure out typing so you can get those hints and links in your IDE. Fix the package docs so they actually tell you what functions the packages export, instead of relying on the wiki for instructions. Etc. Basically, pay a couple excellent docs people for a few years. I found the language a bit weird but not terrible to learn. But if it’s possible to build some porcelain for common usecases that skips there need to grok the language, that could be quite transformative. I do wonder if the “dev env/packaging” use-case might be simpler than the “run and configure a whole OS” usecase.
- WastingMyTime89 4y ago
  > I think most of the docs problems could be solved with the meta-strategy of “deeply understand all the things the Rust ecosystem got right with docs, discovery, and general pedagogical structure, and replicate that”. I think every project would love having the money to hire someone full time to work on the documentation and evangelism.
  - theptip 4y ago
    True, but the Nix project is actually investing in this area and did just hire a docs team.
amelius 4y ago
Also: it should be able to deal with NVidia cards.
- ParetoOptimal 4y ago
  NixOS handles nvidia cards better than any linux distro i've ever uses including Debian, Ubuntu, and Archlinux. I've only had to add a few lines to my configuration over 10 PCs and laptops.
  - amelius 4y ago
    Ok, that is good to know. But how well does it run on NVidia's Jetson line of systems? https://developer.nvidia.com/embedded-computing https://developer.nvidia.com/embedded-computing
    - WastingMyTime89 4y ago
      The Jetson environment provided by Nvidia runs Ubuntu and are setup for you to use docker. I have trouble understanding your question. Why do you want to use Nix if you use a system which comes pre-configured to do something else?
      - amelius 4y ago
        Same reason why I removed Windows from my desktop computer and installed Linux :)
avgcorrection 4y ago
> Writing the packaging expressions should not be harder than writing the program, for the average developer. Nah. The logical conclusion of this is that Nix should be some Java-like, Python-like, or whatever else qualifies as something that the average developer knows. Nix needs to be declarative which rules out the average PL paradigm. > […] , but can the next iteration provide an on-ramp which doesn't involve learning a new lang and paradigm? Guix folks sure think so. Does the average developer know Scheme? I think you just played yourself.
soraminazuki 4y ago
There's lots of criticism about the Nix expression language, but it's way overblown IMO. The core language is small. It's essentially just JSON with functions, very similar to Jsonnet. But with Nix, people make too big of a deal out of implementation details like it being lazy, functional, etc. I personally like Nix as a configuration language because it's suitable for complex configurations despite it being small and simple. There's a case to be made about improving the state of the Nixpkgs APIs though. Stabilizing the APIs and documenting it more thoroughly would dramatically change how people perceive the complexity of Nix. Experienced Nix users grep the Nixpkgs repo to learn about its APIs, but that's far from ideal for newcomers who won't even know where they should look for information. Still, despite its reputation, Nix lowers the bar for software packaging. Package definitions tend to be more concise, customizable, and reusable than the competition. Nixpkgs contains many useful utilities for packaging and there's an abundance of real world examples contained in a single repo. I also frequently work with RPM, but Nix is much easier in comparison.
Bolkan 4y ago
> Everything should be made as simple as possible, but not simpler
WastingMyTime89 4y ago
I think you are missing the heart of the issue. Learning to use a lazy functional programming language remains relatively easy - we use to teach the basis in Ocaml in three hours when I was a student - and Git shows people can tolerate awful CLI tool UX. I think your point 3 is where most of people issues with Nix lie except I wouldn’t limit myself at a lack of practical enough documentation. Nix documentation pretty much just doesn’t exist period. If you want to use Nix, you will have to go through the packages or the implementation to understand what’s happening. That’s not very welcoming especially when you consider that’s the other Linux distributions which tried to do things differently in the past and could be a source of users for Nix OS, first of them Gentoo, have absolutely stellar documentation.
otabdeveloper4 4y ago
Nix (the language) is just JSON with variables and arrow functions. It's the easiest programming language in the world. (Which is a problem when people start building huge projects like nixpkgs with it. It's not a language fit for programming-in-the-large, just like Javascript isn't.)

by Jeaye 4y ago

Bolkan 4y ago

What nix needs more than anything is a centralized repository containing recipes for running all the different kinds of programs, as nic files. ie a dockerhub alternative.

xyzzy_plugh 4y ago
nixpkgs?
- grumbel 4y ago
  Something a little more compact would be welcome. There is: nix flake show templates But the examples in there are a bit out of date and don't necessarily reflect real world usage and are lacking in comments (e.g. still using defaultPackage, not using flake-utils, etc.).

rgoulter 4y ago

> they haven't been able to focus on the UX. I also feel like folks for whom Haskell is comfortable may not realize just how absurd it feels to everyone else. The majority of communication I've seen from Nix maintainers over the last few months has been about exactly this. The submitted post is about exactly that while Nix powerful, they want more people to be able to use it without having to pay a steep penalty to learn it.

tinco 4y ago
It also doesn't have anything to do with Haskell or it looking somewhat like Haskell. I'm very comfortable in Haskell, and Nix looks awful to me. It's not that it's functional or it's declarative, it's that it's badly designed and not intuitive, making it hard to remember and therefore awkward to code in.
- kaba0 4y ago
  I think the only hard part is not Nix, but Nixpkgs’s not well-documented/not-stable APIs. It would be a problem in any language.

techdragon 4y ago

Point 4 is what lost me when I tried using NixOS for a machine I explicitly wanted as reproducible as possible. I needed to get a bunch of stuff working, then freeze/bundle/lock the useland as a documented working state I could rebuild later if anything went wrong. When documented examples began steering me towards the still experimental nix flakes and using the sorts of with ever more flags required commands, I bailed and built it using salt. I want to like nix but the current state of things was a massive put off.

ParetoOptimal 4y ago
> When documented examples began steering me towards the still experimental nix flakes There's a push to make them not be experimental anymore since so much of the community including myself is using them.
dial9-1 4y ago
flakes are anything but experimental at this point, they've been out for many years and I pretty much use them for everything. you don't have to type those flags if you add them to the configuration file
- mook 4y ago
  A newcomer is going to treat anything documented as experimental as that, regardless of what the community thinks; if it's no longer experimental, it needs to be presented as such at all the normal entry points. (None of this is unique to Nix; imagine you needed to start packaging for Windows for some reason.)
  - techdragon 4y ago
    This is exactly what I mean. It’s a big put off to see several year old experimental features and other signs that no one is making the decision to move things forward. If I’m learning something as big as nix with all the facets it has, I don’t also have time to learn it’s entire community history. It’s not a good thing that I need to do that in order to make a decision about how much to trust experimental features of a tool when I’ve not even been using it more than a few hours from the moment of downloading the iso and installing it. It’s a put off to someone new coming to use nix for the first time.

rssoconnor 4y ago

> To seriously answer the question: is the Nix language required for the Nix packaging system to exist? Laziness is required, to some degree, but can the next iteration provide an on-ramp which doesn't involve learning a new lang and paradigm? Guix folks sure think so. I'd love to hear from someone deeply familiar with Nix and Guix about laziness. I'm deeply familiar with Nix and I've concluded that lazy semantics is absolutely critical for a configuration language. It lets me refer to other attributes of my configuration from anywhere. For example, I can refer to port number from my whatever service in my firewall. Nix's system of overlays depends on laziness too to provide efficient late-binding familiar from OOP. I don't need to topologically sort the evaluation of the various inter-dependencies of my configuration. So long as there exists an evaluation order, laziness finds it. Laziness is compelling enough that I managed to convince the author of Jsonnet <https://jsonnet.org/ https://jsonnet.org/> of it when he was designing it, and in turn he helped me design what is now known as overlays in Nix. I don't even understand how Guix manages to work without laziness, though clearly it does somehow. I'm curious as to how that is possible, though I fear I will only ever truly understand by diving into Guix.

infogulch 4y ago
I'm not convinced that any of the things you mentioned as benefits of lazy evaluation are a necessary condition to getting those benefits.
- WastingMyTime89 4y ago
  It’s not. Guix has exactly the same properties while using an eager language.
mikepurvis 4y ago
As an 18 month Nix packager/user, I continue to be blown away by how powerful laziness is and how expressive the interactions between multiple overlays can be.
jackac 4y ago
Recursion and merging semantics are also necessary. I use Nix and Jsonnet a lot, but Nix is much more expressive for complex structures, but the tooling being tightly coupled with the package manage make it impossible to adopt for common use cases.

tikhonj 4y ago

Having used Nix a bunch, the system has been hard to use and learn in exactly the same ways that messy legacy codebases I've worked with have been hard—which makes sense since Nixpkgs is an absolutely massive codebase that's evolved over the years with very distributed development. People latch onto the Nix language and the paradigm because that's the most tangible difference but, everything else held equal, Nix + Nixpkgs would be no easier to work with if everything were in Python. (And, if my limited experience is any guide, would be worse with an ad-hoc DSL pretending to be Python à la Starlark!) This is a pattern I've seen over and over: if a project tries to do something different, the novel aspect gets blamed for issues that would not be surprising in any other context. When faced with a massive awful codebase in Java, well, sometimes legacy code can be a pain, right? It is what it is. People rarely jump to "OOP is a mistake" or "Java is completely unusable". But that's exactly what happens all the time with, say, functional programming. Pointing to an uncommon choice as the underlying cause of a problem makes for a nice, pat story; the pervasive, diffuse problems inherent to messy legacy code do not. After all, here's a single thing that, presumably we can actually change! Nix being functional is an objective fact while people can—reasonably or not—disagree on how well-designed and well-written Nixpkgs-the-codebase is. Besides, even if we could agree, what could we do about it? We all know how much momentum legacy code has and how hard it is to make anything beyond local improvements. Being a socially convenient story doesn't mean it's a particularly strong explanation, but it does mean that people will believe it even if it isn't.

qbasic_forever 4y ago
Yeah IMHO nix just needs someone like Brian Kernighan to sit down and write the definitive book about it and its internals. Something that clearly explains the practical and theoretical uses of nix and nixpkgs. Nix pills, the official manual, etc. aren't really there yet--they talk about all kinds of paradigms that might (?) be going away like nix-env and its ilk. It's just extremely painful to track down the current best practice and way to do things in nix--you're bouncing off all kinds of docs and finding multiple conflicting or out of date ways to do things. Someone needs to give folks a slim book about nix flakes and the flakes way of doing things, the nix language, and a tour of nixpkgs, and IMHO there will be a lot more people using it.

seqizz 4y ago

> I also feel like folks for whom Haskell is comfortable may not realize just how absurd it feels to everyone else. Yup. I can't even imagine what could I accomplish if I'd have something like a "python library" for nixpkgs. I still use nixos but gave up fiddling with nix for a while, after trying to wrap some stuff I am using/developing.* * (Yes, I know it's easy. I just need flakes or correct amount of "lib: prev: { self"s. Sadly not for me.)

theptip 4y ago

Having just done some prototyping with Nix, this list is spot on in enumerating the pain points I hit. I think most of the docs problems could be solved with the meta-strategy of “deeply understand all the things the Rust ecosystem got right with docs, discovery, and general pedagogical structure, and replicate that”. Actually have a canonical set of docs. Link liberally between packafes and stdlib. Figure out typing so you can get those hints and links in your IDE. Fix the package docs so they actually tell you what functions the packages export, instead of relying on the wiki for instructions. Etc. Basically, pay a couple excellent docs people for a few years. I found the language a bit weird but not terrible to learn. But if it’s possible to build some porcelain for common usecases that skips there need to grok the language, that could be quite transformative. I do wonder if the “dev env/packaging” use-case might be simpler than the “run and configure a whole OS” usecase.

WastingMyTime89 4y ago
> I think most of the docs problems could be solved with the meta-strategy of “deeply understand all the things the Rust ecosystem got right with docs, discovery, and general pedagogical structure, and replicate that”. I think every project would love having the money to hire someone full time to work on the documentation and evangelism.
- theptip 4y ago
  True, but the Nix project is actually investing in this area and did just hire a docs team.

amelius 4y ago

Also: it should be able to deal with NVidia cards.

ParetoOptimal 4y ago
NixOS handles nvidia cards better than any linux distro i've ever uses including Debian, Ubuntu, and Archlinux. I've only had to add a few lines to my configuration over 10 PCs and laptops.
- amelius 4y ago
  Ok, that is good to know. But how well does it run on NVidia's Jetson line of systems? https://developer.nvidia.com/embedded-computing https://developer.nvidia.com/embedded-computing
  - WastingMyTime89 4y ago
    The Jetson environment provided by Nvidia runs Ubuntu and are setup for you to use docker. I have trouble understanding your question. Why do you want to use Nix if you use a system which comes pre-configured to do something else?
    - amelius 4y ago
      Same reason why I removed Windows from my desktop computer and installed Linux :)

avgcorrection 4y ago

> Writing the packaging expressions should not be harder than writing the program, for the average developer. Nah. The logical conclusion of this is that Nix should be some Java-like, Python-like, or whatever else qualifies as something that the average developer knows. Nix needs to be declarative which rules out the average PL paradigm. > […] , but can the next iteration provide an on-ramp which doesn't involve learning a new lang and paradigm? Guix folks sure think so. Does the average developer know Scheme? I think you just played yourself.

soraminazuki 4y ago

There's lots of criticism about the Nix expression language, but it's way overblown IMO. The core language is small. It's essentially just JSON with functions, very similar to Jsonnet. But with Nix, people make too big of a deal out of implementation details like it being lazy, functional, etc. I personally like Nix as a configuration language because it's suitable for complex configurations despite it being small and simple. There's a case to be made about improving the state of the Nixpkgs APIs though. Stabilizing the APIs and documenting it more thoroughly would dramatically change how people perceive the complexity of Nix. Experienced Nix users grep the Nixpkgs repo to learn about its APIs, but that's far from ideal for newcomers who won't even know where they should look for information. Still, despite its reputation, Nix lowers the bar for software packaging. Package definitions tend to be more concise, customizable, and reusable than the competition. Nixpkgs contains many useful utilities for packaging and there's an abundance of real world examples contained in a single repo. I also frequently work with RPM, but Nix is much easier in comparison.

Bolkan 4y ago

> Everything should be made as simple as possible, but not simpler

WastingMyTime89 4y ago

I think you are missing the heart of the issue. Learning to use a lazy functional programming language remains relatively easy - we use to teach the basis in Ocaml in three hours when I was a student - and Git shows people can tolerate awful CLI tool UX. I think your point 3 is where most of people issues with Nix lie except I wouldn’t limit myself at a lack of practical enough documentation. Nix documentation pretty much just doesn’t exist period. If you want to use Nix, you will have to go through the packages or the implementation to understand what’s happening. That’s not very welcoming especially when you consider that’s the other Linux distributions which tried to do things differently in the past and could be a source of users for Nix OS, first of them Gentoo, have absolutely stellar documentation.

otabdeveloper4 4y ago

Nix (the language) is just JSON with variables and arrow functions. It's the easiest programming language in the world. (Which is a problem when people start building huge projects like nixpkgs with it. It's not a language fit for programming-in-the-large, just like Javascript isn't.)