8 ms·

Nuenv: An experimental Nushell builder for Nix packages

87 points by biggestlou 4y ago

ParetoOptimal 4y ago
I'm very happy alternative builders such as nushell and Nickel are being worked on. Even happier that concurrently the other side of Nix DX that is documentation is very actively being worked on.
- biggestlou 4y ago
  Apologies for my "well, actually" pedantry but Nickel is a type-safe alternative to Nix for creating expressions and, as far as I know, not a builder. _But_, that being said, there's indeed a lot of fun areas to explore here that, with enough work and care, could be pretty transformative.
  - pxc 4y ago
    Maybe with the effects system Nickel itself could actually be used as a builder as well as the expression language, although I don't think that's really the intent atm.
eviks 4y ago
Nice to see an alternative to dealing with bash, though was wondering why a shell based on a proper language like python wasn't considered
- biggestlou 4y ago
  I know that Python builders do exist out there. Personally, I've found Nushell much more concise than Python et al and quite adequate for the task at hand. But who knows, maybe I should do a whole series :)
- 0x69420 4y ago
  i do not want my shell to depend on whatever idea of package management python has this week, nor do i necessarily want to warm up the jit of a ~proper~ general-purpose scripting language in the process of execing a one-line script
- pxc 4y ago
  Because the main business of most Nix builders is shelling out to external programs, and for that there isn't anything that's a better fit than a shell language. At the same time, the whole mission of these next-gen shell languages (Nushell, Elvish, Oil, Crush, Murex, NGS) is to provide a language that is at once a shell language suitable for daily interactive usage and a capable, general purpose programming language in its own right. Truthfully, I don't think a better fit exists for this use case than one of these shell languages.
  - eviks 4y ago
    But Python is also a shell language suitable for daily interactive usage, e.g., within a python-based shell called xonsh, so the naming alone doesn't explain much re. which of the next-gen shell languages are better suitable
    - pxc 4y ago
      Imo if you spend a little while using a shell like Xonsh or Ammonite, it becomes clear why the internal DSL approach is not as suitable for this as a new language that is designed with interactive shell usage in mind from the start. (Lisp-y languages might be a bit of an exception since command invocations are very close to the same thing syntactically anyway. But idrk since I've never tried using an internal DSL in a Lisp as a Unix shell.) But sure, some people do use and love Xonsh, and you could make the case for using a Xonsh builder in this way as well.
    - pasc1878 4y ago
      For an better shell language that also allows extension in python see xonsh
pxc 4y ago
This kind of thing has been on my fantasy wishlist for Nix for a long time. I am so glad to see experiments like this being done at a company that is dedicating resources to the Nix ecosystem with the potential to rapidly follow through if they see something valuable there. With so many fruitful experiments in the Nix ecosystem being carried out by people who can dedicate more time to Nix than ever, I increasingly feel like I'm wasting my life by not seriously working on or with the Nix ecosystem as my main daily occupation. I know there are still unresolved difficulties and growing pains, but later seems like such a stupid time to really dive in, for people who already know Nix and are already excited about it.
- SkyMarshal 4y ago
  I use NixOS as my daily driver on my two workstations, and package manager on my Mabook Pro now. It's super helpful if you have an extra computer you can reformat and spend a couple months experimenting with NixOS on, learning it, getting it set up just right, and verifying it can do everything you need your main OS to. Nix is great for this kind of rapid experimental iteration with system config, since if something doesn't work you either quickly fix it and make a new build, or just roll back to the prior working build. The beauty of Nix is, if that succeeds, the declarative config can instantly apply it to your other computer/s too, and fully transition with minimal risk. Then you can use multiple Nix computers in a sort of A-B testing setup, where you test new config options on one till they work, and copy them over to the others only after you're certain they're ready. Repeat. And if the initial experiment with it doesn't work out for any reason, you still have your non-Nix computer chugging along doing all the things you need it to do.
  - amelius 4y ago
    Yes, but if it doesn't work there's an indirection between you and the actual problem. Which feels like having to use chopsticks to build a Lego project.
    - SkyMarshal 4y ago
      How do you mean? I ran into a number of problems back when learning NixOS, but the error messages were usually sufficient to track them down.
dlahoda 4y ago
default links bash with well know linter and one may make bash or cp other bash into nix and tune. also with nix it is easy to add alternative cli tools like sad, rg, etc. if i want to be heavy and code real stuff, in nix it is easy to build and run any language. many things people do in bash to glue, i glue using nix declaratively. so imho until other alt shell wins without nix, will use ugly weird non scalable bash.
- scns 4y ago
  Thanks for mentionind sad, didn't know it yet. To save others a search: https://github.com/ms-jpq/sad https://github.com/ms-jpq/sad Space Age seD
Arcuru 4y ago
I'm kind of surprised this didn't link to the Nix RFC that considers changing the default builder from Bash to Oil: https://github.com/NixOS/rfcs/pull/99 https://github.com/NixOS/rfcs/pull/99 Of course it does have a different goal (of replacing Bash with OSH instead of only offering an alternative), but it has a lot of good discussion. Personally I think having alternative builders available is a great idea to improve the developer experience here.
- ofchnofc 4y ago
  [dead]
- chubot 4y ago
  (Oil author here) Thanks for mentioning it -- Oil is intended for exactly that kind of thing, in theory. That is, a clean shell language for gluing together distros, which are inherently heterogeneous and messy. In practice, I think there is limited time / resources / appetite for cleaning up a bunch of 15 year old bash, with the potential to break many things. Shell, and especially bash, can be very fragile. (I'm not speaking for the Nix team, I'd be interested their viewpoint) OSH runs many shell scripts unmodified, but not Nix's, even though we implemented many bash features that Nix uses. I mentioned that in the latest release: https://www.oilshell.org/blog/2023/03/release-0.14.2.html https://www.oilshell.org/blog/2023/03/release-0.14.2.html I'd still be interested in bug reports -- non-interactive language features tend to be prioritized, and are realistic --- So to "concede to reality", I think the approach of having new builders makes sense -- it's ADDING rather than REPLACING. It's sort of like what Rich Hickey says about functional programming -- you don't refactor your code, you just keep adding stuff. Having only used Nix a tiny bit, I didn't even realize new builders are possible. Is this the only other one? Is it possible to create some sort of "interface" for them? If there was a documented interface, maybe it will help people understand how to build packages, in addition to plugging in new builders. --- BTW for people wondering about the project, I just wrote a FAQ: Oils 2023 FAQ - https://www.oilshell.org/blog/2023/03/faq.html https://www.oilshell.org/blog/2023/03/faq.html tl;dr We're stabilizing OSH, the "good parts" that runs your existing scripts. In theory Nix could adopt this, with the harder "replacing" strategy. And we're renaming Oil to YSH to delineate the stable and unstable parts. YSH could be used with the "adding" strategy. Though the idea is that you can seamlessly upgrade from OSH and YSH, and not have two different things forever, but the costs/benefits are different for each project. And of course as I say, YSH isn't yet stable. (nushell is also pre - 1.0 from what I understand, but I guess version pinning helps with that.) Much more detail in the post. We can still use help :)

87 points by biggestlou 4y ago

I'm very happy alternative builders such as nushell and Nickel are being worked on. Even happier that concurrently the other side of Nix DX that is documentation is very actively being worked on.

biggestlou 4y ago
Apologies for my "well, actually" pedantry but Nickel is a type-safe alternative to Nix for creating expressions and, as far as I know, not a builder. _But_, that being said, there's indeed a lot of fun areas to explore here that, with enough work and care, could be pretty transformative.
- pxc 4y ago
  Maybe with the effects system Nickel itself could actually be used as a builder as well as the expression language, although I don't think that's really the intent atm.

eviks 4y ago

Nice to see an alternative to dealing with bash, though was wondering why a shell based on a proper language like python wasn't considered

biggestlou 4y ago
I know that Python builders do exist out there. Personally, I've found Nushell much more concise than Python et al and quite adequate for the task at hand. But who knows, maybe I should do a whole series :)
0x69420 4y ago
i do not want my shell to depend on whatever idea of package management python has this week, nor do i necessarily want to warm up the jit of a ~proper~ general-purpose scripting language in the process of execing a one-line script
pxc 4y ago
Because the main business of most Nix builders is shelling out to external programs, and for that there isn't anything that's a better fit than a shell language. At the same time, the whole mission of these next-gen shell languages (Nushell, Elvish, Oil, Crush, Murex, NGS) is to provide a language that is at once a shell language suitable for daily interactive usage and a capable, general purpose programming language in its own right. Truthfully, I don't think a better fit exists for this use case than one of these shell languages.
- eviks 4y ago
  But Python is also a shell language suitable for daily interactive usage, e.g., within a python-based shell called xonsh, so the naming alone doesn't explain much re. which of the next-gen shell languages are better suitable
  - pxc 4y ago
    Imo if you spend a little while using a shell like Xonsh or Ammonite, it becomes clear why the internal DSL approach is not as suitable for this as a new language that is designed with interactive shell usage in mind from the start. (Lisp-y languages might be a bit of an exception since command invocations are very close to the same thing syntactically anyway. But idrk since I've never tried using an internal DSL in a Lisp as a Unix shell.) But sure, some people do use and love Xonsh, and you could make the case for using a Xonsh builder in this way as well.
  - pasc1878 4y ago
    For an better shell language that also allows extension in python see xonsh

pxc 4y ago

This kind of thing has been on my fantasy wishlist for Nix for a long time. I am so glad to see experiments like this being done at a company that is dedicating resources to the Nix ecosystem with the potential to rapidly follow through if they see something valuable there. With so many fruitful experiments in the Nix ecosystem being carried out by people who can dedicate more time to Nix than ever, I increasingly feel like I'm wasting my life by not seriously working on or with the Nix ecosystem as my main daily occupation. I know there are still unresolved difficulties and growing pains, but later seems like such a stupid time to really dive in, for people who already know Nix and are already excited about it.

SkyMarshal 4y ago
I use NixOS as my daily driver on my two workstations, and package manager on my Mabook Pro now. It's super helpful if you have an extra computer you can reformat and spend a couple months experimenting with NixOS on, learning it, getting it set up just right, and verifying it can do everything you need your main OS to. Nix is great for this kind of rapid experimental iteration with system config, since if something doesn't work you either quickly fix it and make a new build, or just roll back to the prior working build. The beauty of Nix is, if that succeeds, the declarative config can instantly apply it to your other computer/s too, and fully transition with minimal risk. Then you can use multiple Nix computers in a sort of A-B testing setup, where you test new config options on one till they work, and copy them over to the others only after you're certain they're ready. Repeat. And if the initial experiment with it doesn't work out for any reason, you still have your non-Nix computer chugging along doing all the things you need it to do.
- amelius 4y ago
  Yes, but if it doesn't work there's an indirection between you and the actual problem. Which feels like having to use chopsticks to build a Lego project.
  - SkyMarshal 4y ago
    How do you mean? I ran into a number of problems back when learning NixOS, but the error messages were usually sufficient to track them down.

dlahoda 4y ago

default links bash with well know linter and one may make bash or cp other bash into nix and tune. also with nix it is easy to add alternative cli tools like sad, rg, etc. if i want to be heavy and code real stuff, in nix it is easy to build and run any language. many things people do in bash to glue, i glue using nix declaratively. so imho until other alt shell wins without nix, will use ugly weird non scalable bash.

scns 4y ago
Thanks for mentionind sad, didn't know it yet. To save others a search: https://github.com/ms-jpq/sad https://github.com/ms-jpq/sad Space Age seD

Arcuru 4y ago

I'm kind of surprised this didn't link to the Nix RFC that considers changing the default builder from Bash to Oil: https://github.com/NixOS/rfcs/pull/99 https://github.com/NixOS/rfcs/pull/99 Of course it does have a different goal (of replacing Bash with OSH instead of only offering an alternative), but it has a lot of good discussion. Personally I think having alternative builders available is a great idea to improve the developer experience here.

ofchnofc 4y ago
[dead]
chubot 4y ago
(Oil author here) Thanks for mentioning it -- Oil is intended for exactly that kind of thing, in theory. That is, a clean shell language for gluing together distros, which are inherently heterogeneous and messy. In practice, I think there is limited time / resources / appetite for cleaning up a bunch of 15 year old bash, with the potential to break many things. Shell, and especially bash, can be very fragile. (I'm not speaking for the Nix team, I'd be interested their viewpoint) OSH runs many shell scripts unmodified, but not Nix's, even though we implemented many bash features that Nix uses. I mentioned that in the latest release: https://www.oilshell.org/blog/2023/03/release-0.14.2.html https://www.oilshell.org/blog/2023/03/release-0.14.2.html I'd still be interested in bug reports -- non-interactive language features tend to be prioritized, and are realistic --- So to "concede to reality", I think the approach of having new builders makes sense -- it's ADDING rather than REPLACING. It's sort of like what Rich Hickey says about functional programming -- you don't refactor your code, you just keep adding stuff. Having only used Nix a tiny bit, I didn't even realize new builders are possible. Is this the only other one? Is it possible to create some sort of "interface" for them? If there was a documented interface, maybe it will help people understand how to build packages, in addition to plugging in new builders. --- BTW for people wondering about the project, I just wrote a FAQ: Oils 2023 FAQ - https://www.oilshell.org/blog/2023/03/faq.html https://www.oilshell.org/blog/2023/03/faq.html tl;dr We're stabilizing OSH, the "good parts" that runs your existing scripts. In theory Nix could adopt this, with the harder "replacing" strategy. And we're renaming Oil to YSH to delineate the stable and unstable parts. YSH could be used with the "adding" strategy. Though the idea is that you can seamlessly upgrade from OSH and YSH, and not have two different things forever, but the costs/benefits are different for each project. And of course as I say, YSH isn't yet stable. (nushell is also pre - 1.0 from what I understand, but I guess version pinning helps with that.) Much more detail in the post. We can still use help :)