23 ms·

Happy to see someone inspired by Nix, but wanting to carve their own path. Nix popularized some powerful ideas in the Linux world, but it has a steep learning c

by imiric 2y ago

Happy to see someone inspired by Nix, but wanting to carve their own path. Nix popularized some powerful ideas in the Linux world, but it has a steep learning curve and a very unfriendly UI, so there is plenty of room for improvement there. I'm not sure if Lua is the right choice, though. A declarative language seems like a better fit for reproducibility. The goal of supporting non-deterministic builds also seems to go against this. But I'm interested to know how this would work in practice. Good luck!

hinkley 2y ago
If you design it like SCons, it'll look imperative but behave more declaratively. If I understand the architecture correctly, the imperative calls in the config file don't actually run the build process. They run a Builder Pattern that sets up the state machine necessary for the builds to happen. So it's a bit like LINQ in C# (but older). I have no idea how that plays out single-step debugging build problems though. That depends on how it's implemented and a lot of abstractions (especially frameworks) seem to forget that breakpoints are things other people want to use as well.
- zombiezen 2y ago
  That's accurate (unless the config file attempts to read something from the build process, that will trigger a build). It's a good point about debugging build problems. This is an issue I've experienced in Nix and Bazel as well. I'm not convinced that I have a great solution yet, but at least for my own debugging while using the system, I've included a `zb derivation env` command which spits out a .env file that matches the environment the builder runs under. I'd like to extend that to pop open a shell.
  - skybrian 2y ago
    One thing I like to see is a 'dry run' like 'make -n'. Although, maybe that's not possible in all cases. Another possibility might be to output a something like a shell script that would do a rebuild the same way, so you can see what it did and hack it when debugging.
    - photonthug 2y ago
      Yes. Dry runs at least, and better yet terraform-style planning that produces an artifact that can be applied. These should really be more common with all kinds of software
      - hinkley 2y ago
        I would like to see more tools iterate on trying to do terraform-like output because while terraform diffs are interesting, practically most of my teammates couldn’t tell what the fuck they said and a couple times I missed important lines that caused us prod issues. I think we can do a better job than showing a wall of text.
        photonthug 2y ago
        Presentation is a separate matter though, just like with git diffs ideally you could choose a wall of text or a side by side ui, see things at a high level or drill down to line by line. A tag layer plus custom validation between plan/apply gives you an automatic way to short circuit things. But none of it can work without a plan as a first class object. Thing is the plan/apply split isn’t even primarily for users necessarily, it’s just good design. It makes testing easier, and leaves open the possibility for plugging in totally different resolution strategies without rewriting the whole core. The benefits are so big there that I’d strongly prefer that more software is using it more often, even if I’m going to shut my eyes and auto apply every time without glancing at the plan.
  - pdimitar 2y ago
    Surface-level feedback: get rid of the word "derivation". Surely there must be a better way to describe the underlying thing...
    - School-Cotton 2y ago
      What’s wrong with it? It’s a term of art that means a specific thing in both nix and guix; it’d just be confusing if zb renamed it to something else.
      - kstenerud 2y ago
        I'm 80% finished moving all of my servers from NixOS to Debian. I used NixOS for 3 years (even wrote some custom flakes) before finally giving up (for the final year I was just too scared to touch it, and then said "I shouldn't be scared of my OS"). I should know what "derivation" means, but I can't for the life of me remember...
        smilliken 2y ago
        "Derivation" refers to the nix intermediate build artifact, a .drv file, which contains the instructions to perform the build itself. Basically a nix program compiles to a derivation file which gets run to produce the build outputs. The hash in the /nix/store for a dependency is the hash of the derivation. Conveniently if the hash is already in a build cache, you can download the cached build outputs instead of building it yourself.
        kstenerud 2y ago
        Ah OK, then I'd actually never actually understood what a derivation is. But then again, the name "derivation" doesn't at all lead to guessing at such a definition, either.
        School-Cotton 2y ago
        “Build plan” would maybe be a more obvious name, but it’d still be confusing to deviate from what Nox uses, IMO.
kortex 2y ago
> The goal of supporting non-deterministic builds also seems to go against this. I think this is actually a great escape hatch. Supporting non-deterministic builds means more folks will be able to migrate their existing build to zb. Postel's law and all that.
- imiric 2y ago
  Right, could be. One of the insane things with Nix is that the suggested workflow is to manage _everything_ with it. This means that it wants to replace every package manager in existence, so you see Python, Emacs and other dependency trees entirely replicated in Nix. As well as every possible configuration format. It's craziness... Now I don't need to depend on just the upstream package, I also have to wait for these changes to propagate to Nix packages. And sometimes I just want to do things manually as a quick fix, instead of spending hours figuring out why the Nix implementation doesn't work. So, yeah, having an escape hatch that allows easier integration with other ecosystems or doing things manually in some cases, would be nice to have.
sweeter 2y ago
I thought of creating something similar and I was going to use a personal fork of the Go compiler with some mods, anko (which is a really cool go binding language) or righting my own DSL. It's quite the undertaking. I like Nix and NixOS a lot, its really cool, but it has some really odd management issues and the language IMO is horrendous. I used NixOS for around a year and I was changing my Nixpkgs version and I got that same generic nonsense error that doesn't have any semantic meaning and I was just over it. I'm not too fond of commenting out random parts of code to figure out where something minor and obscure failed. Sometimes it tells you the module it had a problem with, or will point out an out of place comma, and other times its just like "idk bruh ¯\_(ツ)_/¯ "failed at builtin 'seq'" is the best I can do" the paradigm is a million dollar idea though. I have no doubt its the future of a large portion of the future, both for programming and generic systems. I just wish it wasn't a pain to write and it had some sensible error handling.
- danmur 2y ago
  The language has grown on me a bit. I initially hated it but a lot of my pain was not actually the language but the lack of good docs for the standard library. Still struggle with the tracebacks though. It's painful when things go wrong.
deleted 2y ago
[deleted]

by imiric 2y ago

hinkley 2y ago

If you design it like SCons, it'll look imperative but behave more declaratively. If I understand the architecture correctly, the imperative calls in the config file don't actually run the build process. They run a Builder Pattern that sets up the state machine necessary for the builds to happen. So it's a bit like LINQ in C# (but older). I have no idea how that plays out single-step debugging build problems though. That depends on how it's implemented and a lot of abstractions (especially frameworks) seem to forget that breakpoints are things other people want to use as well.

zombiezen 2y ago
That's accurate (unless the config file attempts to read something from the build process, that will trigger a build). It's a good point about debugging build problems. This is an issue I've experienced in Nix and Bazel as well. I'm not convinced that I have a great solution yet, but at least for my own debugging while using the system, I've included a `zb derivation env` command which spits out a .env file that matches the environment the builder runs under. I'd like to extend that to pop open a shell.
- skybrian 2y ago
  One thing I like to see is a 'dry run' like 'make -n'. Although, maybe that's not possible in all cases. Another possibility might be to output a something like a shell script that would do a rebuild the same way, so you can see what it did and hack it when debugging.
  - photonthug 2y ago
    Yes. Dry runs at least, and better yet terraform-style planning that produces an artifact that can be applied. These should really be more common with all kinds of software
    - hinkley 2y ago
      I would like to see more tools iterate on trying to do terraform-like output because while terraform diffs are interesting, practically most of my teammates couldn’t tell what the fuck they said and a couple times I missed important lines that caused us prod issues. I think we can do a better job than showing a wall of text.
      - photonthug 2y ago
        Presentation is a separate matter though, just like with git diffs ideally you could choose a wall of text or a side by side ui, see things at a high level or drill down to line by line. A tag layer plus custom validation between plan/apply gives you an automatic way to short circuit things. But none of it can work without a plan as a first class object. Thing is the plan/apply split isn’t even primarily for users necessarily, it’s just good design. It makes testing easier, and leaves open the possibility for plugging in totally different resolution strategies without rewriting the whole core. The benefits are so big there that I’d strongly prefer that more software is using it more often, even if I’m going to shut my eyes and auto apply every time without glancing at the plan.
- pdimitar 2y ago
  Surface-level feedback: get rid of the word "derivation". Surely there must be a better way to describe the underlying thing...
  - School-Cotton 2y ago
    What’s wrong with it? It’s a term of art that means a specific thing in both nix and guix; it’d just be confusing if zb renamed it to something else.
    - kstenerud 2y ago
      I'm 80% finished moving all of my servers from NixOS to Debian. I used NixOS for 3 years (even wrote some custom flakes) before finally giving up (for the final year I was just too scared to touch it, and then said "I shouldn't be scared of my OS"). I should know what "derivation" means, but I can't for the life of me remember...
      - smilliken 2y ago
        "Derivation" refers to the nix intermediate build artifact, a .drv file, which contains the instructions to perform the build itself. Basically a nix program compiles to a derivation file which gets run to produce the build outputs. The hash in the /nix/store for a dependency is the hash of the derivation. Conveniently if the hash is already in a build cache, you can download the cached build outputs instead of building it yourself.
        kstenerud 2y ago
        Ah OK, then I'd actually never actually understood what a derivation is. But then again, the name "derivation" doesn't at all lead to guessing at such a definition, either.
        School-Cotton 2y ago
        “Build plan” would maybe be a more obvious name, but it’d still be confusing to deviate from what Nox uses, IMO.

kortex 2y ago

> The goal of supporting non-deterministic builds also seems to go against this. I think this is actually a great escape hatch. Supporting non-deterministic builds means more folks will be able to migrate their existing build to zb. Postel's law and all that.

imiric 2y ago
Right, could be. One of the insane things with Nix is that the suggested workflow is to manage _everything_ with it. This means that it wants to replace every package manager in existence, so you see Python, Emacs and other dependency trees entirely replicated in Nix. As well as every possible configuration format. It's craziness... Now I don't need to depend on just the upstream package, I also have to wait for these changes to propagate to Nix packages. And sometimes I just want to do things manually as a quick fix, instead of spending hours figuring out why the Nix implementation doesn't work. So, yeah, having an escape hatch that allows easier integration with other ecosystems or doing things manually in some cases, would be nice to have.

sweeter 2y ago

I thought of creating something similar and I was going to use a personal fork of the Go compiler with some mods, anko (which is a really cool go binding language) or righting my own DSL. It's quite the undertaking. I like Nix and NixOS a lot, its really cool, but it has some really odd management issues and the language IMO is horrendous. I used NixOS for around a year and I was changing my Nixpkgs version and I got that same generic nonsense error that doesn't have any semantic meaning and I was just over it. I'm not too fond of commenting out random parts of code to figure out where something minor and obscure failed. Sometimes it tells you the module it had a problem with, or will point out an out of place comma, and other times its just like "idk bruh ¯\_(ツ)_/¯ "failed at builtin 'seq'" is the best I can do" the paradigm is a million dollar idea though. I have no doubt its the future of a large portion of the future, both for programming and generic systems. I just wish it wasn't a pain to write and it had some sensible error handling.

danmur 2y ago
The language has grown on me a bit. I initially hated it but a lot of my pain was not actually the language but the lack of good docs for the standard library. Still struggle with the tracebacks though. It's painful when things go wrong.

deleted 2y ago

[deleted]