4 ms·

Show HN: Flake schemas – teaching Nix about your flake outputs

32 points by biggestlou 3y ago

mongol 3y ago
I thought I started to understand flakes, but now I feel clueless again.
- zamalek 3y ago
  What are you struggling with? I've become okay at Nix and flakes - so maybe I can help understand.
  - mongol 3y ago
    What does this enable that will be useful to me as a user?
    - rq1 3y ago
      Better tooling, simpler interfaces, better error reporting, validation… etc.
  - res0nat0r 3y ago
    I'm using NixOS again after installing it years ago and enjoying it, but I still have a real hard time understanding just how to somehow implement or use a flake, or create a useful new package with it that isn't in the nixpkgs repo already. I still find much of the syntax incredibly confusing and hard to just get simple concrete self contained examples, or without worrying if this is intermingled with some non flake code somewhere that isn't compatible. I've bumped a few nix packages via github PR's and can hack around a bit with existing things I see upstream, but knowing how to do everything is still confusing. I guess I need to sit down and read all of https://zero-to-nix.com/ https://zero-to-nix.com/ and see if there is anything I'm missing.
- grhmc 3y ago
  This is a deeper cut of pretty foundational stuff. Almost no user will need to actually interact with schemas, and instead will use schemas provided by other folks. In other words, if this doesn't make sense: don't worry, you don't need to know about it :). The tl;dr, though: * nix flake show and other tools only knew about very specific types of things you might put in a flake. * with flake-schemas, that limitation is lifted and people can do more creative things.
- jljljl 3y ago
  Flakes have a set of default outputs they can generate, like packages, devShells, templates, and NixOS configs. Nix knows how these outputs work and can validate them using `nix flake check`, or list them with `nix flake show` But what if you want your flake to output a library? A Dockerfile? A Terraform or Kubernetes config? Nix doesn't know about those custom outputs, and doesn't have a way to validate that you're using them correctly. Now with Flake Schemas, you can provide a schema so that built in tools like `nix flake show` and `nix flake check` can properly validate the new output. This makes it easier for Flake developers to communicate the interface of their custom output to Flake consumers
  - alexeldeib 3y ago
    I’m modestly familiar with nix. I’ve written flakes and some other derivations. I’ve dabbled with overlays and such but I’m not super familiar. I liked flakes enough I created a template repo for packaging dev envs. Until this post/your comment I didn’t know flakes could even have custom outputs. I guess they’re all just conventions for attribute names, and everything is a derivation, so it makes sense. But realizing that point was key, even with passing nix familiarity.
grhmc 3y ago
Hey folks, this is Graham -- CEO of DetSys. Happy to help answer any questions!
- moojd 3y ago
  Thanks Graham! Glad to see some attempts at improving the dx of sharing expressions with flakes instead of just derivations. I've seen libraries and tools handle this in a ton of different ways. Do you think there will ever be a standardization of the 'lib' output?
  - grhmc 3y ago
    I hope there will be, but I'm not really sure. I think the future of Nix is in smaller, focused flakes being shared and exchanged. People dunk on projects like Rust and Node for having infinity dependencies, but it is that way because those ecosystems have rich library ecosystems that are worth using. I see the same thing happening in Nix.
    - clhodapp 3y ago
      Regularization of more Flake outputs into schemas sounds compelling. E.g. thanks to this great work on flake schemas, do the definitions of outputs related to NixOS modules, NixOS configs, and nixpkgs overlays even need to be pre-baked into Nix? In terms of this decomposition effort, one additional thing that would be really nice to see would be turning flake URL schemes & updaters extensible. One could imagine some kind of a dictionary that maps from schemes to derivations that contain the code needed to get the primitive URL for the "latest" version of the flake when an update is called for.
      - grhmc 3y ago
        I totally agree. This is one of the reasons we made FlakeHub, to make a better update mechanism that was flexible and supported things that aren't just github as sources. I think that'll happen, it'll just take a bit of time, and likely experimenting outside of nix before patches land upstream.
VTimofeenko 3y ago
Ooh, shiny! No more trying to remember how I usually nest home-manager modules for custom stuff
pshirshov 3y ago
> Flake schemas are not a type system for Nix, since that would be a huge project. The lack of a reasonable type system is one of the biggest Nix problems.
- rq1 3y ago
  There’s this https://github.com/tweag/nickel https://github.com/tweag/nickel Coupled with https://github.com/nickel-lang/organist https://github.com/nickel-lang/organist I’m not sure how immature it is though.

32 points by biggestlou 3y ago

mongol 3y ago

I thought I started to understand flakes, but now I feel clueless again.

zamalek 3y ago
What are you struggling with? I've become okay at Nix and flakes - so maybe I can help understand.
- mongol 3y ago
  What does this enable that will be useful to me as a user?
  - rq1 3y ago
    Better tooling, simpler interfaces, better error reporting, validation… etc.
- res0nat0r 3y ago
  I'm using NixOS again after installing it years ago and enjoying it, but I still have a real hard time understanding just how to somehow implement or use a flake, or create a useful new package with it that isn't in the nixpkgs repo already. I still find much of the syntax incredibly confusing and hard to just get simple concrete self contained examples, or without worrying if this is intermingled with some non flake code somewhere that isn't compatible. I've bumped a few nix packages via github PR's and can hack around a bit with existing things I see upstream, but knowing how to do everything is still confusing. I guess I need to sit down and read all of https://zero-to-nix.com/ https://zero-to-nix.com/ and see if there is anything I'm missing.
grhmc 3y ago
This is a deeper cut of pretty foundational stuff. Almost no user will need to actually interact with schemas, and instead will use schemas provided by other folks. In other words, if this doesn't make sense: don't worry, you don't need to know about it :). The tl;dr, though: * nix flake show and other tools only knew about very specific types of things you might put in a flake. * with flake-schemas, that limitation is lifted and people can do more creative things.
jljljl 3y ago
Flakes have a set of default outputs they can generate, like packages, devShells, templates, and NixOS configs. Nix knows how these outputs work and can validate them using `nix flake check`, or list them with `nix flake show` But what if you want your flake to output a library? A Dockerfile? A Terraform or Kubernetes config? Nix doesn't know about those custom outputs, and doesn't have a way to validate that you're using them correctly. Now with Flake Schemas, you can provide a schema so that built in tools like `nix flake show` and `nix flake check` can properly validate the new output. This makes it easier for Flake developers to communicate the interface of their custom output to Flake consumers
- alexeldeib 3y ago
  I’m modestly familiar with nix. I’ve written flakes and some other derivations. I’ve dabbled with overlays and such but I’m not super familiar. I liked flakes enough I created a template repo for packaging dev envs. Until this post/your comment I didn’t know flakes could even have custom outputs. I guess they’re all just conventions for attribute names, and everything is a derivation, so it makes sense. But realizing that point was key, even with passing nix familiarity.

grhmc 3y ago

Hey folks, this is Graham -- CEO of DetSys. Happy to help answer any questions!

moojd 3y ago
Thanks Graham! Glad to see some attempts at improving the dx of sharing expressions with flakes instead of just derivations. I've seen libraries and tools handle this in a ton of different ways. Do you think there will ever be a standardization of the 'lib' output?
- grhmc 3y ago
  I hope there will be, but I'm not really sure. I think the future of Nix is in smaller, focused flakes being shared and exchanged. People dunk on projects like Rust and Node for having infinity dependencies, but it is that way because those ecosystems have rich library ecosystems that are worth using. I see the same thing happening in Nix.
  - clhodapp 3y ago
    Regularization of more Flake outputs into schemas sounds compelling. E.g. thanks to this great work on flake schemas, do the definitions of outputs related to NixOS modules, NixOS configs, and nixpkgs overlays even need to be pre-baked into Nix? In terms of this decomposition effort, one additional thing that would be really nice to see would be turning flake URL schemes & updaters extensible. One could imagine some kind of a dictionary that maps from schemes to derivations that contain the code needed to get the primitive URL for the "latest" version of the flake when an update is called for.
    - grhmc 3y ago
      I totally agree. This is one of the reasons we made FlakeHub, to make a better update mechanism that was flexible and supported things that aren't just github as sources. I think that'll happen, it'll just take a bit of time, and likely experimenting outside of nix before patches land upstream.

VTimofeenko 3y ago

Ooh, shiny! No more trying to remember how I usually nest home-manager modules for custom stuff

pshirshov 3y ago

> Flake schemas are not a type system for Nix, since that would be a huge project. The lack of a reasonable type system is one of the biggest Nix problems.

rq1 3y ago
There’s this https://github.com/tweag/nickel https://github.com/tweag/nickel Coupled with https://github.com/nickel-lang/organist https://github.com/nickel-lang/organist I’m not sure how immature it is though.