4 ms·

I plan on writing a static site generator soon myself. And I briefly wondered the same thing and realized that I could very easily answer that question: ruthles

I plan on writing a static site generator soon myself. And I briefly wondered the same thing and realized that I could very easily answer that question: ruthlessly build whatever I personally need and not what others need. I don't usually follow that line of thinking, but the static site generator space is so full of so many great tools that I don't think there's really anything for me to contribute to it. Instead, I think the killer feature of my static site generator is going to be exactly what makes it difficult for others to use: its tight coupling with my specific use case. I'll be able to hard-code configuration knobs. Avoid dealing with edge cases. Bake strange non-generic and single-blog-post-specific logic right into the tool. (The reason why I want to build my own is because I've tried using someone else's. And I just keep getting bitten: someone else's tool gets so big and moves so rapidly, that by the time I get around to updating my site, something about the new version of the tool has broken the way my web site works. And then I usually have to spend hours figuring out how to fix it, which is hugely deflating because I had just worked up the energy to write a new post. My naive belief is that if I have my own tool that evolves at precisely the pace I want it to, I won't run into that specific problem. I'll run into other problems of course, but I'm thinking that I'll prefer those problems to the ones I currently face.)

impoppy 4y ago
Sounds cool, I hope one day I stumble upon your article that utilizes those very custom hard-coded knobs. Yet I wouldn't make a tool like that since tools should be KISS.
- burntsushi 4y ago
  KISS is exactly my point. :)
epage 4y ago
You should check out what fasterthanlime does for their blog. Theirs uses sqlite for storing all state and the write plugins into the Rust Liquid template implementation to directly query sqlite so they could do whatever they wanted without recompiling. It was pretty neat. I have been looking to break out parts of static-site-generators to make it easier to bring one up. So far I only have two primitive building blocks - https://docs.rs/engarde/latest/engarde/ https://docs.rs/engarde/latest/engarde/ - https://docs.rs/file-serve/latest/file_serve/ https://docs.rs/file-serve/latest/file_serve/
- burntsushi 4y ago
  My needs are far simpler than that. I don't need state. I don't need dynamic content. I don't need ads. I don't need tracking. I don't need search. I just need a way to publish content and maybe an RSS feed. It doesn't even need to serve files. Everything is static. (I'm not saying fasterthanlime has all those things. Just saying what I don't need.) I can't even imagine I need a template system beyond something I can write in a few dozen lines with some regexes. I did the SQLite thing more than a decade ago. I ain't going back to it unless I have a really really good reason for it. I really just need a purely static web site where I can write Markdown posts. There's some other stuff I want in terms of producing the static content, but in terms of web site functionality itself, it should just be HTML, CSS and some images. I also don't mind re-compiling. For my needs, "compiling" my web site is likely to take less than 1 second. Definitely don't need plugins. :)
  - pdimitar 4y ago
    I plan on trying https://luapress.org/ https://luapress.org/ one of the next weeks. Looks good at a first glance -- give it a quick check if you get a free hour, would be interested what you think of it.
    - burntsushi 4y ago
      I don't have a free hour unfortunately. And I know I'm not going to use that because I don't want to setup a Lua environment. Indeed, I have enough Lua experience to know that they break the language in big ways for what seems like every release. That's a liability. What happens when luapress stops getting regular maintenance and the version of Lua it uses is no longer packaged in my Linux distro? Oops. Ticking time bomb. I meant it. I'm done with static site generators built by others. They have caused me too much pain. EDIT: That "oops" is already reality: https://github.com/Fizzadar/Luapress#luapress-v4 https://github.com/Fizzadar/Luapress#luapress-v4 Static site generators are just too easy to build and too easy to let die. And the ones that don't die grow big and bloated. That's my experience. The only remedy I see is to build my own and simplify ruthlessly through tight coupling. "You either live long enough to see yourself become the villain, or you die a hero."
gamegoblin 4y ago
I've done exactly what you describe and I highly recommend it. My personal website is served by a couple hundred lines of Rust. There is no database or ORM -- I just defined structs like `BlogPost` or whatever and stuck a `#[derive(Serialize, Deserialize)]` on it and use the `serde` library to JSONify it, store it flat in a hard-coded folder. I even wrote my own dead-simple Markdown-ish flavor that is tailored exactly to my writing style. I use a lot of em-dashes, for example, so it translates `--` into `—` (rather, &mdash). Or, since a lot of my posts link to other posts, I can do `[post title_of_other_post]` and it'll generate the appropriate link. Whenever I want a new feature while writing a particular blog post, e.g. pretty inline SVG, I just go add it and then keep writing.
a1369209993 4y ago
> And I just keep getting bitten: someone else's tool gets so big and moves so rapidly, that by the time I get around to updating my site, []something about the new version of the tool has broken the way my web site works. And then I usually have to spend hours figuring out how to fix it[], which is hugely deflating because I had just worked up the energy to write a new post. My naive belief is that if I have my own tool that evolves at precisely the pace I want it to, I won't run into that specific problem. I'll run into other problems of course, but I'm thinking that I'll prefer those problems to the ones I currently face. This is true, as a general principle, not just of static site generators, but of all software, and is one of the major Reasons Why We Can't Have Nice Things, and why automatic updates are evil and poisonous, and manual updates dangerous and often-avoided. See eg https://jacquesmattheij.com/why-johnny-wont-upgrade/ https://jacquesmattheij.com/why-johnny-wont-upgrade/.
- burntsushi 4y ago
  > but of all software I don't think so. There's lots of software that I've used and upgraded for decades and rarely if never run into any issues. I don't think it's a general principle. I don't like generalizing to "all" unless I have a really good reason to do so. Instead, I stuck to my specific experience with static site generators.
rumblings 4y ago
This. All the big static generators in the market are kinda of generic enough that they demand upfront investment from you. Whether its Jekyll, Hugo or any big shot you have to fit a lot of things according to them. My personal website is built by Jekyll, but I finding a lot harder to make changes because I am not comfortable with a Ruby tooling( Once, I had to debug an obscure Ruby related bug in M1 Mac). I am comfortable with Python though( I have considerable proficiency and good documentation) which then I will use it for my own static generator. I guess writing your own non-generic own-static generator should be relatively easy and worth the effort as you wouldn't have to read some obscure documentation (like Jekyll).
yakubin 4y ago
I went this route. At first I tried writing a bunch of Python scripts reading a YAML manifest. However, after ~2 weeks I already got warnings and errors about things which got deprecated/removed/changed by a new Python version and the YAML library I was using. So I dumped it and rewrote the thing in XML and XSLT. It's been working for a couple years with no breakages. Essentially 0 maintenance after the initial work. This design was about generating a tree of directories with Markdown files turned into HTML, PDF files turned into pages linking to them, and some directory renames, GUIDs and dates (for Atom feeds, which I generate for each directory) specified in a manifest file. I've since taken up photography, so now I need to automatically read metadata from jpegs, add some tags and organise it on the site in albums. It's a bit outside the reach of XSLT, so first I've tried using Python with Jinja2 and PIL. Same problem as previously. So I've rewritten it in Go with the help of encoding/json (for the manifest), html/template (for the template), image/jpeg, golang.org/x/image/draw (for generating thumbnails of different sizes for different screen sizes) packages and exiftool (reading and removing EXIF tags). Pandoc has been present throughout all the rewrites, because I like its flavour of Markdown and options to shift heading levels by a specified value, so that they fit in with the surrounding template. I'm putting some finishing touches and am going to upload it in the coming days. I know, you stopped reading at "XSLT". :D
qudat 4y ago
We think alike. I started out with Hugo because it's very popular and relatively easy to use. It has a bunch of features that I had to read about but didn't have a use for -- overhead. One of the most annoying aspects of working with hugo was the friction from idea to publishing it. I decided to go a completely different route and built https://prose.sh https://prose.sh With prose all I have to do is to create a markdown file and then `scp` it to prose.sh. The platform takes care of the rest and I can still enjoy my own editor and terminal tooling. Even better, authentication happens via SSH so the friction is extremely low. It's a nice balance between writing a blog on a platform that does everything for you and being able to use my own tooling to write content. Another big issue with Hugo is when I want to make edits to the content. My brain is weird and I only scrutinize my content after it has been published. So I end up making a lot of edits after an article has been live for a few minutes. It's a terrible habit and Hugo required deployment for every little change. I've thought about converting prose to be SSG -- we could generate the html when a user uploads the file -- but honestly the site is fast as it is so I don't see the value in that added complexity.
- majou 4y ago
  Tangentially related: the delay field in your hn profile will hide your comment for n minutes for you to make edits. I always set mine to 2.
- daretorant 4y ago
  Nice! I have a Hugo site I’ve tweaked over the years, and my workflow is similar to yours. I create/edit an MD file, then run a shell command which pushes changes to Github, which triggers a redeploy. How do you handle images? That’s my main annoyance / biggest hurdle when publishing. I have to take each image, resize it for web, put it in static folder, and reference that pathname in the MD file. Not a big deal, but would love an easier way. I suppose I could write a simple shell script that does all of the resizing and prints out a path back for me to use.
  - qudat 4y ago
    We are working on https://imgs.sh https://imgs.sh which will try to make optimizing for web automatic as well as making it easy to reference images from your blog.
kaashif 4y ago
I just wrote like a Perl script that generates my site. It's just printing stuff and converting markdown to HTML, right? Never breaks, never changes, does exactly what I need it to. Presumably there are reasons people want to use more specialised tools, but I don't have any of those.
jeffparsons 4y ago
I've just started to take that approach (exactly what I need, no more, no less) with personal process automation, and it's really working for me. I made a few attempts in the past at building more general-purpose tools that would cover my use cases, but I've never followed through on them. Too much of the work was in the bits that don't actually benifit me right now. My new tool is simply called "Other Jeff". I write onto it what I need right now. Do I have a series of pull requests that need babysitting to get them over the line? Other Jeff learns to poll their status and notify me of important events along the way. If I want to store some state about what I'm working on right now, I need a database of some kind, right? Nope. Not any more. I hard-code it into `db.rs`, because it turns out there's almost nothing I need to persist that's too burdensome to update by hand. So what I have in the end is a single Rust program that grows and shrinks depending on my needs at the time, accumulates reusable helpers if and when they actually make sense for _me_, and slowly learns to automate more of my daily distractions.

impoppy 4y ago

Sounds cool, I hope one day I stumble upon your article that utilizes those very custom hard-coded knobs. Yet I wouldn't make a tool like that since tools should be KISS.

burntsushi 4y ago
KISS is exactly my point. :)

epage 4y ago

You should check out what fasterthanlime does for their blog. Theirs uses sqlite for storing all state and the write plugins into the Rust Liquid template implementation to directly query sqlite so they could do whatever they wanted without recompiling. It was pretty neat. I have been looking to break out parts of static-site-generators to make it easier to bring one up. So far I only have two primitive building blocks - https://docs.rs/engarde/latest/engarde/ https://docs.rs/engarde/latest/engarde/ - https://docs.rs/file-serve/latest/file_serve/ https://docs.rs/file-serve/latest/file_serve/

burntsushi 4y ago
My needs are far simpler than that. I don't need state. I don't need dynamic content. I don't need ads. I don't need tracking. I don't need search. I just need a way to publish content and maybe an RSS feed. It doesn't even need to serve files. Everything is static. (I'm not saying fasterthanlime has all those things. Just saying what I don't need.) I can't even imagine I need a template system beyond something I can write in a few dozen lines with some regexes. I did the SQLite thing more than a decade ago. I ain't going back to it unless I have a really really good reason for it. I really just need a purely static web site where I can write Markdown posts. There's some other stuff I want in terms of producing the static content, but in terms of web site functionality itself, it should just be HTML, CSS and some images. I also don't mind re-compiling. For my needs, "compiling" my web site is likely to take less than 1 second. Definitely don't need plugins. :)
- pdimitar 4y ago
  I plan on trying https://luapress.org/ https://luapress.org/ one of the next weeks. Looks good at a first glance -- give it a quick check if you get a free hour, would be interested what you think of it.
  - burntsushi 4y ago
    I don't have a free hour unfortunately. And I know I'm not going to use that because I don't want to setup a Lua environment. Indeed, I have enough Lua experience to know that they break the language in big ways for what seems like every release. That's a liability. What happens when luapress stops getting regular maintenance and the version of Lua it uses is no longer packaged in my Linux distro? Oops. Ticking time bomb. I meant it. I'm done with static site generators built by others. They have caused me too much pain. EDIT: That "oops" is already reality: https://github.com/Fizzadar/Luapress#luapress-v4 https://github.com/Fizzadar/Luapress#luapress-v4 Static site generators are just too easy to build and too easy to let die. And the ones that don't die grow big and bloated. That's my experience. The only remedy I see is to build my own and simplify ruthlessly through tight coupling. "You either live long enough to see yourself become the villain, or you die a hero."

gamegoblin 4y ago

I've done exactly what you describe and I highly recommend it. My personal website is served by a couple hundred lines of Rust. There is no database or ORM -- I just defined structs like `BlogPost` or whatever and stuck a `#[derive(Serialize, Deserialize)]` on it and use the `serde` library to JSONify it, store it flat in a hard-coded folder. I even wrote my own dead-simple Markdown-ish flavor that is tailored exactly to my writing style. I use a lot of em-dashes, for example, so it translates `--` into `—` (rather, &mdash). Or, since a lot of my posts link to other posts, I can do `[post title_of_other_post]` and it'll generate the appropriate link. Whenever I want a new feature while writing a particular blog post, e.g. pretty inline SVG, I just go add it and then keep writing.

a1369209993 4y ago

> And I just keep getting bitten: someone else's tool gets so big and moves so rapidly, that by the time I get around to updating my site, []something about the new version of the tool has broken the way my web site works. And then I usually have to spend hours figuring out how to fix it[], which is hugely deflating because I had just worked up the energy to write a new post. My naive belief is that if I have my own tool that evolves at precisely the pace I want it to, I won't run into that specific problem. I'll run into other problems of course, but I'm thinking that I'll prefer those problems to the ones I currently face. This is true, as a general principle, not just of static site generators, but of all software, and is one of the major Reasons Why We Can't Have Nice Things, and why automatic updates are evil and poisonous, and manual updates dangerous and often-avoided. See eg https://jacquesmattheij.com/why-johnny-wont-upgrade/ https://jacquesmattheij.com/why-johnny-wont-upgrade/.

burntsushi 4y ago
> but of all software I don't think so. There's lots of software that I've used and upgraded for decades and rarely if never run into any issues. I don't think it's a general principle. I don't like generalizing to "all" unless I have a really good reason to do so. Instead, I stuck to my specific experience with static site generators.

rumblings 4y ago

This. All the big static generators in the market are kinda of generic enough that they demand upfront investment from you. Whether its Jekyll, Hugo or any big shot you have to fit a lot of things according to them. My personal website is built by Jekyll, but I finding a lot harder to make changes because I am not comfortable with a Ruby tooling( Once, I had to debug an obscure Ruby related bug in M1 Mac). I am comfortable with Python though( I have considerable proficiency and good documentation) which then I will use it for my own static generator. I guess writing your own non-generic own-static generator should be relatively easy and worth the effort as you wouldn't have to read some obscure documentation (like Jekyll).

yakubin 4y ago

I went this route. At first I tried writing a bunch of Python scripts reading a YAML manifest. However, after ~2 weeks I already got warnings and errors about things which got deprecated/removed/changed by a new Python version and the YAML library I was using. So I dumped it and rewrote the thing in XML and XSLT. It's been working for a couple years with no breakages. Essentially 0 maintenance after the initial work. This design was about generating a tree of directories with Markdown files turned into HTML, PDF files turned into pages linking to them, and some directory renames, GUIDs and dates (for Atom feeds, which I generate for each directory) specified in a manifest file. I've since taken up photography, so now I need to automatically read metadata from jpegs, add some tags and organise it on the site in albums. It's a bit outside the reach of XSLT, so first I've tried using Python with Jinja2 and PIL. Same problem as previously. So I've rewritten it in Go with the help of encoding/json (for the manifest), html/template (for the template), image/jpeg, golang.org/x/image/draw (for generating thumbnails of different sizes for different screen sizes) packages and exiftool (reading and removing EXIF tags). Pandoc has been present throughout all the rewrites, because I like its flavour of Markdown and options to shift heading levels by a specified value, so that they fit in with the surrounding template. I'm putting some finishing touches and am going to upload it in the coming days. I know, you stopped reading at "XSLT". :D

qudat 4y ago

We think alike. I started out with Hugo because it's very popular and relatively easy to use. It has a bunch of features that I had to read about but didn't have a use for -- overhead. One of the most annoying aspects of working with hugo was the friction from idea to publishing it. I decided to go a completely different route and built https://prose.sh https://prose.sh With prose all I have to do is to create a markdown file and then `scp` it to prose.sh. The platform takes care of the rest and I can still enjoy my own editor and terminal tooling. Even better, authentication happens via SSH so the friction is extremely low. It's a nice balance between writing a blog on a platform that does everything for you and being able to use my own tooling to write content. Another big issue with Hugo is when I want to make edits to the content. My brain is weird and I only scrutinize my content after it has been published. So I end up making a lot of edits after an article has been live for a few minutes. It's a terrible habit and Hugo required deployment for every little change. I've thought about converting prose to be SSG -- we could generate the html when a user uploads the file -- but honestly the site is fast as it is so I don't see the value in that added complexity.

majou 4y ago
Tangentially related: the delay field in your hn profile will hide your comment for n minutes for you to make edits. I always set mine to 2.
daretorant 4y ago
Nice! I have a Hugo site I’ve tweaked over the years, and my workflow is similar to yours. I create/edit an MD file, then run a shell command which pushes changes to Github, which triggers a redeploy. How do you handle images? That’s my main annoyance / biggest hurdle when publishing. I have to take each image, resize it for web, put it in static folder, and reference that pathname in the MD file. Not a big deal, but would love an easier way. I suppose I could write a simple shell script that does all of the resizing and prints out a path back for me to use.
- qudat 4y ago
  We are working on https://imgs.sh https://imgs.sh which will try to make optimizing for web automatic as well as making it easy to reference images from your blog.

kaashif 4y ago

I just wrote like a Perl script that generates my site. It's just printing stuff and converting markdown to HTML, right? Never breaks, never changes, does exactly what I need it to. Presumably there are reasons people want to use more specialised tools, but I don't have any of those.

jeffparsons 4y ago

I've just started to take that approach (exactly what I need, no more, no less) with personal process automation, and it's really working for me. I made a few attempts in the past at building more general-purpose tools that would cover my use cases, but I've never followed through on them. Too much of the work was in the bits that don't actually benifit me right now. My new tool is simply called "Other Jeff". I write onto it what I need right now. Do I have a series of pull requests that need babysitting to get them over the line? Other Jeff learns to poll their status and notify me of important events along the way. If I want to store some state about what I'm working on right now, I need a database of some kind, right? Nope. Not any more. I hard-code it into `db.rs`, because it turns out there's almost nothing I need to persist that's too burdensome to update by hand. So what I have in the end is a single Rust program that grows and shrinks depending on my needs at the time, accumulates reusable helpers if and when they actually make sense for _me_, and slowly learns to automate more of my daily distractions.