4 ms·

I use Nix, so I get incremental builds and your problem goes away.

by Jhsto 3y ago

schemescape 3y ago
You’re using Nix to drive your static site generation? If so, please share more details because that sounds intriguing!
- Jhsto 3y ago
  Here you go: https://juuso.dev/blogPosts/nix-as-a-static-site-generator/nix-as-a-static-site-generator.html https://juuso.dev/blogPosts/nix-as-a-static-site-generator/n...
  - schemescape 3y ago
    Wow, I ask and a blog post just appears! My initial reaction is: I should probably get around to learning about Flakes. I’m not sure I’d want each blog post to pin its templates, but it’s nice to have that choice.
    - __MatrixMan__ 3y ago
      A flake is a function, right? Invoking `nix build` resolves its inputs and computes its outputs. So if the static site is the output, wouldn't the content have to be one of the inputs? Personally, I think I'd use two flakes, one that builds the content into something that's ready to hand-off to code, and a second one that turns it into a usable site. That way you wouldn't end up with a new input for each post, but instead would have a versioned something which represents your content all bundled together, and then the site builder consumes it as a single dependency--but conceptually it's the same. I guess I'm just saying that it's not conventional, but it's a pretty logical conclusion to reach.
__MatrixMan__ 3y ago
I was just reading this and thinking that `nix build` would do the same trick even more nicely.
- Jhsto 3y ago
  I posted a link to the other comment, but here it is for you as well: https://juuso.dev/blogPosts/nix-as-a-static-site-generator/nix-as-a-static-site-generator.html https://juuso.dev/blogPosts/nix-as-a-static-site-generator/n...
  - __MatrixMan__ 3y ago
    I admire how nimble you are. I aspire to write blog posts at the drop of a hat like this, but I rarely do. I also like the use of flake inputs for content. It reminds me of a world that I've been imagining where the conclusions in scientific papers are generated as a flake outputs (an adjacent output would be the human readable thing, a PDF or whatever). In this world, you can just run `nix flake update && nix build`, and if a paper that you cite published an update which invalidates your conclusion, you know right away because their output is your input, so your build fails. We think about repeatable builds being for executable binaries, but they could equally well be for conclusions and assumptions. Perhaps nix is too big of a hammer for the job, but it seems like the best shot we have at achieving this without also constraining the scientist re: tooling. I realize that you don't want to be storing mountains of data in the nix store, but it would work just as well if the output in question is an IPFS CID, to be resolved during the build instead. The publisher can then be in charge of keeping that CID pinned and of notifying scientists when they're "build" starts failing.
    - Jhsto 3y ago
      > I admire how nimble you are. I aspire to write blog posts at the drop of a hat like this, but I rarely do. Thanks! I took up blogging more often as of recent, and for me, having a manageable system is a large part of that. The last thing I want to happen on a Sunday evening is breaking some page of my website. That being said, I hope to one day make the workflow easier. > It reminds me of a world that I've been imagining where the conclusions in scientific papers are generated as a flake outputs (and adjacent output would be the human readable thing, a PDF or whatever). I happen to be a reviewer for software artifacts in a scientific journal, and I often use Nix here. Not that many projects do use it, but if I'm able to reproduce it with Nix, then I know the author has not missed any implicit dependencies. I like to imagine it's also useful for the authors as a feedback, whether they use Nix or not. > I realize that you don't want to be storing mountains of data in the nix store, but it would work just as well if the output in question is an IPFS CID, to be resolved during the build instead. I maintain separate build serves of my own using Nix integrations and the Nix cache is quite large already (so called remote builders) sitting at around 500GB. I host these at Hetzner. I have also thought adding IPFS integration for my website, but haven't got around to it.

I use Nix, so I get incremental builds and your problem goes away.

by Jhsto 3y ago

I use Nix, so I get incremental builds and your problem goes away.

schemescape 3y ago

You’re using Nix to drive your static site generation? If so, please share more details because that sounds intriguing!

Jhsto 3y ago
Here you go: https://juuso.dev/blogPosts/nix-as-a-static-site-generator/nix-as-a-static-site-generator.html https://juuso.dev/blogPosts/nix-as-a-static-site-generator/n...
- schemescape 3y ago
  Wow, I ask and a blog post just appears! My initial reaction is: I should probably get around to learning about Flakes. I’m not sure I’d want each blog post to pin its templates, but it’s nice to have that choice.
  - __MatrixMan__ 3y ago
    A flake is a function, right? Invoking `nix build` resolves its inputs and computes its outputs. So if the static site is the output, wouldn't the content have to be one of the inputs? Personally, I think I'd use two flakes, one that builds the content into something that's ready to hand-off to code, and a second one that turns it into a usable site. That way you wouldn't end up with a new input for each post, but instead would have a versioned something which represents your content all bundled together, and then the site builder consumes it as a single dependency--but conceptually it's the same. I guess I'm just saying that it's not conventional, but it's a pretty logical conclusion to reach.

__MatrixMan__ 3y ago

I was just reading this and thinking that `nix build` would do the same trick even more nicely.

Jhsto 3y ago
I posted a link to the other comment, but here it is for you as well: https://juuso.dev/blogPosts/nix-as-a-static-site-generator/nix-as-a-static-site-generator.html https://juuso.dev/blogPosts/nix-as-a-static-site-generator/n...
- __MatrixMan__ 3y ago
  I admire how nimble you are. I aspire to write blog posts at the drop of a hat like this, but I rarely do. I also like the use of flake inputs for content. It reminds me of a world that I've been imagining where the conclusions in scientific papers are generated as a flake outputs (an adjacent output would be the human readable thing, a PDF or whatever). In this world, you can just run `nix flake update && nix build`, and if a paper that you cite published an update which invalidates your conclusion, you know right away because their output is your input, so your build fails. We think about repeatable builds being for executable binaries, but they could equally well be for conclusions and assumptions. Perhaps nix is too big of a hammer for the job, but it seems like the best shot we have at achieving this without also constraining the scientist re: tooling. I realize that you don't want to be storing mountains of data in the nix store, but it would work just as well if the output in question is an IPFS CID, to be resolved during the build instead. The publisher can then be in charge of keeping that CID pinned and of notifying scientists when they're "build" starts failing.
  - Jhsto 3y ago
    > I admire how nimble you are. I aspire to write blog posts at the drop of a hat like this, but I rarely do. Thanks! I took up blogging more often as of recent, and for me, having a manageable system is a large part of that. The last thing I want to happen on a Sunday evening is breaking some page of my website. That being said, I hope to one day make the workflow easier. > It reminds me of a world that I've been imagining where the conclusions in scientific papers are generated as a flake outputs (and adjacent output would be the human readable thing, a PDF or whatever). I happen to be a reviewer for software artifacts in a scientific journal, and I often use Nix here. Not that many projects do use it, but if I'm able to reproduce it with Nix, then I know the author has not missed any implicit dependencies. I like to imagine it's also useful for the authors as a feedback, whether they use Nix or not. > I realize that you don't want to be storing mountains of data in the nix store, but it would work just as well if the output in question is an IPFS CID, to be resolved during the build instead. I maintain separate build serves of my own using Nix integrations and the Nix cache is quite large already (so called remote builders) sitting at around 500GB. I host these at Hetzner. I have also thought adding IPFS integration for my website, but haven't got around to it.