10 ms·

Rewriting my blog in Rust for fun and profit

226 points by lawn 4y ago

Tesl 4y ago
This is a cool idea, I've wanted to find a Rust project for a while and regenerating my own site (which uses jekyll) might not be a bad idea too. Every time I come back to mine it never works because it always complains about the wrong ruby library versions being installed.
- bbkane 4y ago
  I switched from Jekyll to Zola because of my difficulty running Jekyll locally too. I chose Zola because it's a single binary and its template language (Tera) is similar to Jekyll's so my blog was fairly easy to port.
jasonjmcghee 4y ago
Great post- great project. Always fun to see someone building something (from scratch!) that filled a need they personally had.
hamaluik 4y ago
I did the same thing a couple years ago [0]. It's not the most elegant and my blog is miniscule, but it was a fun project and also born out of frustration with the existing site generators. I structured it as the source sitting next to the site contents, so from a fresh computer (with Rust installed), it's as straightforward as `cargo build`. I think the best thing about this approach (writing your own generator) is that it really lets you build your site / content exactly the way you want to. Don't need categories and tags? Don't add them to begin with. Want to add an RSS feed? No problem, just add it. Want to do some fancy image compression and processing in the pipeline? No problem; you're the boss. It sounds simple but it really frees you up to write / generate your site however you want. 0: https://github.com/hamaluik/blog.hamaluik.ca https://github.com/hamaluik/blog.hamaluik.ca
- mxuribe 4y ago
  As enjoyable as this post was, i recalled my history with static site generators as follows: hand-code html pages --> hand-code html but w/ server-side includes (yes, back in the apache days) --> use php w/own custom templates and mysql db --> move to wordpress --> love the intent of wordpress, but regret the needed maintenance, so move back to custom php and custom templates --> move manage content via Lektor (python SSG) --> learn that i don't need a UI, and am content crafting my content in markdown --> start using pelican (python SSG) --> get bored, so move to Hugo as new SSG flavor of the month --> get annoyed with templating language --> move back to pelican --> realized i spent too much time bouncing around, just did away with any SSG tool set, and hand craft html pages whenever i want to publish something...and now only craft journal entries for my own private diary (with an occasional post on the fediverse). :-)
- annowiki 4y ago
  I just linked your blog post in a separate comment because I was reading it over the weekend. I was looking for articles about writing static site generators and didn't realize most people don't roll their own templating engines. I was disappointed to see not much on templating in your post but it definitely gave me the confidence to keep going. Managed to get aspects I'd been struggling to work for months working yesterday.
dceddia 4y ago
Fun project! I really resonate with this part > But I haven’t used Haskell in years, and the overhead for me to add slightly more complex things to the site is quite large I’m starting to come around to the idea that it might be better to avoid shiny fun things in favor of having most of my active projects use the same stack, or at least the same few languages. I’ve got a blog in Jekyll (with some Ruby-based customizations), a bunch of stuff in JavaScript, some TypeScript, an app in Elixir & Phoenix, a (desktop) app in Rust, and some Rails to maintain. For me personally, Elixir is the odd one out: I learned enough to build that app, and I really enjoyed Elixir, but that knowledge was fleeting because I haven’t needed to touch the code very much. I have to relearn how Phoenix works in order to make changes. As much as it pains me to say this, I kind of wish it was in TypeScript or something, just to reduce the mental overhead. (but then, that’s a whole other can of worms because of how piecemeal things are in the Node ecosystem, so I’d end up having to learn a web framework and an ORM and some other libraries, which feels like the sort of knowledge that might be even more fleeting!)
- zelphirkalt 4y ago
  > As much as it pains me to say this, I kind of wish it was in TypeScript or something, just to reduce the mental overhead. If you once knew how Elixir works (syntactically), then you will be able to quickly bounce back. If comparing TypeScript with Phoenix, then you would have to relearn your in TS written functionality, in contrast to relearning Phoenix. I don't see how it being TS would help much tbh. I think what matters much more is, whether your past self left good comments and wrote readable code.
- benreesman 4y ago
  There’s a lot of wisdom in that observation. Knowing a lot of languages well is a super-power in some ways, but heavily polyglot environments create real pragmatic challenges. There’s a reason so many marquee shops carefully curate a (short) menu of production languages. It’s been my experience that if (and it’s a big if) you have either the need or inclination to operate a big orgy of polyglot language interop, the hellacious learning curves on NixOS and Bazel are “The Way”. It’s a big investment and only worth it if you’re pretty committed/obligated, but there is a path for those whose needs justify the cost.
munificent 4y ago
I also rewrote my blog's static site generator to a bespoke one (in Dart in my case), largely for performance reasons. Prior, I was using Jekyll and Pygments. Jekyll and Pygments are really slow. My new site generator is literally 100x faster, and I did, like, zero real optimization work. In fact, it can rebuild my entire blog and every post from scratch faster than Jekyll's watch mode (with caching set up properly) could rebuild a single page.
- Keats 4y ago
  I wrote Zola because at that time I was using Hugo + Pygments and while Hugo was fast, Pygments was the opposite. I'm pretty sure Zola can build a 1k page website with syntax highlighting faster than Pygments can highlight 20 pages.
- radicalriddler 4y ago
  Is it open sourced? Would love to have a read of some dart not written by a Googler or Xoogler. Unless... Edit: and... just read his bio
impoppy 4y ago
I've been trying to make a cool static site generator numerous times. Well, yes, it actually is a quite fun thing to tinker with, but I've found myself struggling to decide on what do I want to see in the final app and what I do not; what features might be needed by other users and what features might be an overkill. So I ultimately went with Zola[1] and encourage you to try it too! Aaaand it is written in Rust, too. It's available in Alpine Linux's repos so it's trivial to set up a CI/CD to compile your blog. [1] https://www.getzola.org/ https://www.getzola.org/
- asicsp 4y ago
  I moved from Jekyll to Zola and it's been great until I started tinkering with templates and stuff recently. Single binary and simple themes were main reasons to choose this. Wanted a feature that wasn't present in the version I started with (about 2 years ago). So, I got the newest version and a few things broke subtly (ex: syntax highlighting). Which led me to tinkering (I don't know much about HTML/CSS/etc). It was actually good to try a few things almost blindly and then slowly narrowing with a bit more thought process. Been a long time since I had such fun investigating unknown codebase. This even helped me move my post processing scripts to templates. And then I went a step further by reading about meta tags (things like image, description, etc) and adding them via an extra variable in the post frontmatter.
  - impoppy 4y ago
    That's exactly why I never used Jekyll and don't really understand people that use it nowadays. It is AWFUL. Not only it uses greatly outdated Ruby, you also HAVE to install outdated Ruby on your machine to play with themes, serve, compile and are forced to use RVM. I know GitHub Pages can compile the page for you, but when you want to customize a theme you have to go through all that hell. It's just a terrible experience. And installing plugins is another hell.
    - BrandonS113 4y ago
      I use Jekyll for a few sites and it works perfectly. Design is bootstrap and so nothing to do with Jekyll and it always compiles the site, takes a few seconds. Works fine on my mac and linux. Would I have picked Jekyll today? no. Is it worth the cost of switching? no.
benreesman 4y ago
It’s refreshing to see a front-page Rust piece that just nails it on promoting the language. It cites concrete examples of where the rubber really meets the road: e.g. Cargo as a one-stop-shop for platform insensitive, “just works” build and dep management. It acknowledges that it’s ok for fun and novelty to be features, which it is! It steers well clear of A >> B nonsense. No preachy TED talk about memory safety. It’s respectful of the value that other languages provide in their respective niches. This is how to sell Rust. It’s novel and fun but also pragmatic and industrial grade! And if you’re into X, that’s cool too!
- taftster 4y ago
  Your comment helped me want to click through and read the post. I'm very tired of all the recent Rust fanboy posts, but level-headed and earnest comparisons against real implementations make for much better understanding of a new concept or language. Here are some select quotes from the article: > Please note that this is not to say that Rust this [sic] much faster than Haskell, instead see it as a comparison of two implementations. I’m sure someone could make it super fast in Haskell. > I don’t think it means that Rust is simpler or easier than Haskell, it just means that I personally had an easier time to grok Rust than Haskell.
  - benreesman 4y ago
    The author is clearly an experienced and mature hacker who knows many languages well. It’s pretty difficult to maintain the shrill “X is better than everything” tone you’re referring to with a diverse portfolio of marketable skills. I don’t know how Rust, which is quite a cool language, attracted this sizable bloc of aggressively intermediate jihadis, but it’s really worrying because Rust is in fact innovative and important, but epic languages have crashed and burned over less.
    - nindalf 4y ago
      You’re really going to call people jihadis for talking about the technical strengths of a programming language? And then you’re going to act like you’re the self appointed arbiter of discourse on HN, saying that this kind of article is ok but that kind isn’t? Maybe a person who calls those they disagree with jihadis isn’t qualified to police what others say. Let’s talk about the way you talk. Here’s you from 5 days ago accusing a good tool of being a “dumpster fire” and saying “it crashes process IDs more often than Justin Bieber crashes Maseratis”. (https://news.ycombinator.com/item?id=32581005 https://news.ycombinator.com/item?id=32581005). But it gets worse, because later you start crying about being downvoted. At this point it looks like a speed run of how many HN rules you can break. And then you follow that up with tone policing others.
zelphirkalt 4y ago
Ah yes, one of my favorite projects in a newly learned language: Make a blog, based on plain text files, maybe markdown, maybe something else. But since I have done that, I feel that many existing site generators are bloated, probably, because they cover scenarios, which I would not want on my blog anyway.
mynameisash 4y ago
> Several external dependencies. > It’s annoying to install all these and to keep them up-to-date, it would be great to just have a single thing to worry about. I feel the author's pain. There's a Python2 project for ebook management that I used pretty extensively years ago, but it depended on a number of external dependencies, both as compiled shared objects and other Python libraries. It felt extremely fragile and was always a pain to get it working properly, so I eventually rewrote the core parts of the entire project for my own use (in Rust, incidentally). Now I have a single app that I can be sure will pretty much always work.
- Macha 4y ago
  I also have a personal Rust ebook manager because of frustrations with calibre :) Though Calibre has at least made it to Python 3 these days.
deleted 4y ago
[deleted]
cletus 4y ago
I enjoyed this post. The premise is a good one, as in for fun. A lot of these "I wrote X in Y" go on some spree of questionable assertions but "because I wanted to" is intellectually honest and understandable. I agree with a lot of the points about Rust too. Every language is a selection of tradeoffs and for me Rust's chosen tradeoffs I think are both important (eg memory safety) and reasonable. Take something like Cargo. The author says "It just works". This is my experience as well. But more importantly, dependency management and packaging were built into the language and ecosystem from the start. It still boggles my mind that any language that came after Java didn't do this (yes, I'm looking at you, Go). My heart did sink a little bit at "Using Regex of course!" though.
- brightball 4y ago
  I honestly highly recommend the approach. Anytime I get excited about a new technology, I rewrite my blog in it. Gives me a simple live project with real traffic and all the right moving parts. Last time I did it was with Elixir + Phoenix + PostgreSQL, without any standard or reasonable approach. It was a lot of fun and still using it today. https://www.brightball.com/articles/insanity-with-elixir-phoenix-postgresql https://www.brightball.com/articles/insanity-with-elixir-pho...
heurisko 4y ago
> On my crappy laptop a full site rebuild took 75 seconds. (Not compile, just to generate the site.) I only have 240 posts so I don’t think it should be that slow. Wow, what on earth was Hakyll doing all that time?
brabel 4y ago
Don't forget to throw your static site generator on the pile: https://jamstack.org/generators/ https://jamstack.org/generators/ 347 options and counting!
skrtskrt 4y ago
I went from Jekyll to to Hugo to avoid being bothered with futzing with ruby gems. Eventually I really appreciated the Golang approach to templating much more than Jinja/Liquid style templates, Go has kind of a composition-over-inheritance approach to templates. If I ever have time to get proficient enough with Rust I'd love to port that Go templating approach to Rust. Maybe not so much the data interpolation syntax, but rather the approach to composing template partials. But ultimately I'm familiar enough with Jinja style, Zola would probably be a breeze to adopt as well.
nlh 4y ago
Nice work and great write up! I’m a huge fan of treating one’s blog as a “homelab” for exploring new (or old) software stacks for the web, so this idea resonates a lot. One thing I’d like to try (has anyone tried this?): eschew the idea of a static site generator and go the “old school” route of dynamically generating each page on the fly, but take advantage of the modern CDN layer of the web (CloudFlare, etc.) to effectively cache and give you the speed equivalent of a static site. Yes this approach has more dependencies but I think simplifies the work done on the blog side since you can build it using whichever simple web framework appeals to you most (or none at all…this is a homelab after all). My plan (don’t we all have this plan) is to try this out in Crystal when time allows…
thrwawy74 4y ago
I really liked the reactive pattern, so one of the best static site generators I found was docpad. It used (backbone?) collections, so you could subscribe to changes on things like "new files under this path" and then rerender the part of the page that needed to change. I haven't seen this done in others. Then docpad was rewritten in ES6, and I lost all love for it. Things were so expressively concise in Coffeescript.
- impoppy 4y ago
  It's kind of ironic and uncool that DocPad's website is made using GitBook.
  - thrwawy74 4y ago
    The author is a pretty interesting guy, but I just didn't want to follow him into ES6 land. Coffeescript is still the ranking king on concision, and so many features didn't make it in. I'm going to be sad when I can't support the transpile in my projects anymore :(
annowiki 4y ago
This is mindblowing synchronicity. I thought this article was one I had read this weekend for a second but checked the post date and saw it wasn't available yesterday. I'm working on trying to write a static site generator for my own personal site/blog in C++, been tinkering for a couple months. It started as a passion project in tribute to my dog that passed. GatsbyJS wasn't working (again) which is what my current site is built with, so I just said screw it, I'll write my own. Chose C to begin with and quickly gave up. Decided to switch to C++ because its what I'm supposed to be learning for work. I named it bluesky, after my dog that passed, Sky Blue. https://github.com/mas-4/bluesky https://github.com/mas-4/bluesky Templating is a lot harder than I had initially thought. I wanted the templating system to be as bare bones as possible, similar to another static site generator I rather like, sergey[0]. Most of these articles I found ([1], [2]) about writing your own SSG don't go into templating much, they just use an off the shelf library. Inja[3] is available for C++ but, like I said, I want something really bare bones, like if you were designing html now, you'd obviously include html-includes and html-templates. I finally got it working for includes and templates, now I have to add the markdown support, and then I plan on migrating my personal site to using it. [0]: https://sergey.cool/ https://sergey.cool/ [1]: https://www.smashingmagazine.com/2020/09/stack-custom-made-static-site-generator/ https://www.smashingmagazine.com/2020/09/stack-custom-made-s... [2]: https://blog.hamaluik.ca/posts/build-your-own-static-site-generator/ https://blog.hamaluik.ca/posts/build-your-own-static-site-ge... [3]: https://github.com/pantor/inja https://github.com/pantor/inja edit: a link
- brundolf 4y ago
  I've found you don't actually need a templating system, you just need format-strings. Write your rendering logic with normal code that just spits out a string Templating languages probably help with performance a bit when you're rendering at request-time, but for static stuff I don't really see much point I've got a hand-rolled static site generator in JavaScript that just uses template strings. I only have around 30 pages, to be fair, but when I make a change it re-generates the whole site before I can even alt-tab and reload the page. I assume a C++ version could be much faster
  - tomjakubowski 4y ago
bArray 4y ago
I have some 381 posts (including landing pages, etc). It takes maybe 45 minutes to build (there is no cache process). I have a faster version that skips audio processing, but it still takes minutes. The problem is that it does several pandoc parses for each page: 1. Plain text - Input for audio parsing (which takes a significant amount of time, I have a version of the build that skips this). 2. Simple HTML - Input for RSS feed. 3. Full HTML - Static web page. One change I would make to pandoc is to have the option for several outputs (as was discussed years ago [1]). I think generating multiple outputs is highly common usage. One feature I will get around to writing (eventually) is automatic image compression. I had some success using jpegoptim and optipng on another project. This takes a while, so I want to build a command line tool that caches the output of a command if both the input file(s) and command do not change, and invalidate based on time. That way I could blindly do something like: image="cat-pic.png" cache-run -i $image -c "optipng -dir out/ -o 4 -strip all -fix $image" (You might want to experiment with different optimization values based on resolution, etc). [1] https://groups.google.com/g/pandoc-discuss/c/lex900rSpOM https://groups.google.com/g/pandoc-discuss/c/lex900rSpOM
marosgrego 4y ago
If you're interested in this, a while ago, for my personal photo site with image galleries[1], I wrote a fast static site generator in Rust[0], which also supports plugins now. [0] https://www.getblades.org https://www.getblades.org [1] https://grego.site/photo https://grego.site/photo
AceJohnny2 4y ago
I'm seeing a few too many comments about static site generators rebuilding everything on a change. Have the tenets of Make (and the like) from over 40 years ago been forgotten? I guess maintaining a dependency graph just isn't a high priority feature for site generators.
- pitaj 4y ago
  For something run relatively infrequently, I can see why it wouldn't be a huge priority. On that topic, I wish there was a generic build system like SCons written in something faster. Maybe there is (does Bazel fit my bill?) and I just don't know of it yet. What I love best about SCons is that you can just write custom builders right there. No need to drop in a shell script - you have the much better Python standard library at your fingertips. This also means it's way easier to make cross-platform.
- vkou 4y ago
  Computers are fast, sites get updated infrequently, and compiling a static site is a lot less computational work than dealing with the utter *********** that is C++ templates. Unless you're 'web-scale', it's complexity that doesn't solve any particular business need.
manholio 4y ago
Why even bother with a static generator when you can run rust code at essentially wire speed and dynamically generate all the required formatting on the fly? Because Rust binaries are so secure, you can expose them over the internet safely. With a little bit of glue, you can automatically recompile the CGI binary every time the rs source is touched, for a dev experience similar to interpreted web scripts.
- Winsaucerer 4y ago
  I prefer to take your approach, but one advantage of a static generator is that it opens up hosting location options such as github pages, dropping in an S3 type public storage, many traditional web hosts, netlify, etc.
- chrismorgan 4y ago
  > Because Rust binaries are so secure, you can expose them over the internet safely. That doesn’t stand to reason. Safely in that you can confidently avoid things like remote code execution, sure. But that doesn’t necessarily protect you from denial of service attacks; for example, I expect most services built on Hyper to be vulnerable to slowloris attacks <https://en.wikipedia.org/wiki/Slowloris_(computer_security) https://en.wikipedia.org/wiki/Slowloris_(computer_security)>, because I believe it still defaults to basically no sane kind of timeout, and doesn’t provide the tools to handle it properly anyway <https://github.com/hyperium/hyper/issues/1628 https://github.com/hyperium/hyper/issues/1628>. If that’s part of your threat model, you should avoid exposing it directly and use a reverse proxy. Good somewhat related reading, on a technologically simple DoS attack on Rust-based code that was largely fixed by speeding things up, fixing caching and proper use of reverse proxying (via Cloudflare because it was genuinely heavy load): https://fasterthanli.me/articles/i-won-free-load-testing https://fasterthanli.me/articles/i-won-free-load-testing
  - manholio 4y ago
    Sorry for the lack of clarity, by "exposing the binary over the internet" I certainly wasn't implying re-implementing a HTTP server, HTTPS and certificate handling etc. You would of course have an nginx frontend that would cache the request and subtleties; the Rust FastCGI would see a well formed request, and rush to execute it just like any other dynamic language, PHP, Python. You can of course still do stupid things inside the rust application that explode your number of concurrent requests, like waiting for slow external resources, databases etc. For a simple static formatter, it should not be the case, it would just repeatedly hit the disk cache and some minimal CPU.
deleted 4y ago
[deleted]
eatbitseveryday 4y ago
Not to be a negative Nancy, but the phrase “for fun and profit” is quite overused that any originality is lost.
saimiam 4y ago
For my own blogging needs, I chose an even more pragmatic alternative to markdown - email! Well, email is a actually a single file CMS which contains html, plaintext, attachments/images, and metadata like when the email was sent and the subject line, With all this, it turns out you can create pretty nice looking webpages without having to futz around with md->html conversions. The huge advantage is that since I know gmail, whenever I have an idea for a blog post, I can go from idea to published in minutes. The biggest time suck tends to be finding stock photos for the post. As far as tech stack goes, it’s all serverless. I use SES to handle inbound emails and the excellent nodemailer package to parse .eml files.
rtpg 4y ago
My recommendation for people using anything like Jekyll: just bite the bullet and write a custom build script. A 200 line script that takes can handle Jinja templates or markdown or something will mean you actually understand all that is happening and aren't trying to fight the system or just figure out how to do anything. And of course don't write a plugin system or something you would share with others! Just write the thing that you need, and change it as you need to! Static site generators are great for people less comfortable doing this, but having to deal with all the _stuff_ is a major waste of time for what it's offering for people who don't need it.
- hombre_fatal 4y ago
  I have the opposite advice: optimize for writing, minimize janitorial stuff like tweaking your own tooling. It just distracts you from writing and offers only masturbatory value. The best path depends on your personality, but building my own blog tooling killed my writing. I'd start writing something only to realize I needed to tweak something. Then I wouldn't even write the post or I wouldn't have the energy to fix the tooling and I'd do neither. And then I stopped writing. I didn't start writing again until I used Wordpress which let me focus on writing, though after proving myself for a year, I switched to Jekyll + Github Pages so that everything was on Github.
- csande17 4y ago
  100% agreed. The things a static site generator does are typically very simple: load some Markdown into memory, run it through some HTML templates, and maybe scale some pictures with ImageMagick if you're feeling especially fancy. Writing a script to do those things is way easier than trying to learn and wrangle a Static Site Generator(tm). The one time static stie generators are helpful is if you really, really like the out-of-the-box look and behavior of some existing theme.
  - rtpg 4y ago
    To this point: Markdown is a nice format for ingest because you can just paste in HTML. I am a bit more courageous and have some custom directives for my Markdown stuff, but it's all really 20 lines of code. I think it's tempting to add a bunch of "time saving" features for writing, but lots of time just going with the HTML is fine. Maybe if you're writing out a post every couple of days then you can focus on time saves. Until then MD will get you most of the way, and otherwise you can always hack in some special case for your unique post

226 points by lawn 4y ago

Tesl 4y ago

This is a cool idea, I've wanted to find a Rust project for a while and regenerating my own site (which uses jekyll) might not be a bad idea too. Every time I come back to mine it never works because it always complains about the wrong ruby library versions being installed.

bbkane 4y ago
I switched from Jekyll to Zola because of my difficulty running Jekyll locally too. I chose Zola because it's a single binary and its template language (Tera) is similar to Jekyll's so my blog was fairly easy to port.

jasonjmcghee 4y ago

Great post- great project. Always fun to see someone building something (from scratch!) that filled a need they personally had.

hamaluik 4y ago

I did the same thing a couple years ago [0]. It's not the most elegant and my blog is miniscule, but it was a fun project and also born out of frustration with the existing site generators. I structured it as the source sitting next to the site contents, so from a fresh computer (with Rust installed), it's as straightforward as `cargo build`. I think the best thing about this approach (writing your own generator) is that it really lets you build your site / content exactly the way you want to. Don't need categories and tags? Don't add them to begin with. Want to add an RSS feed? No problem, just add it. Want to do some fancy image compression and processing in the pipeline? No problem; you're the boss. It sounds simple but it really frees you up to write / generate your site however you want. 0: https://github.com/hamaluik/blog.hamaluik.ca https://github.com/hamaluik/blog.hamaluik.ca

mxuribe 4y ago
As enjoyable as this post was, i recalled my history with static site generators as follows: hand-code html pages --> hand-code html but w/ server-side includes (yes, back in the apache days) --> use php w/own custom templates and mysql db --> move to wordpress --> love the intent of wordpress, but regret the needed maintenance, so move back to custom php and custom templates --> move manage content via Lektor (python SSG) --> learn that i don't need a UI, and am content crafting my content in markdown --> start using pelican (python SSG) --> get bored, so move to Hugo as new SSG flavor of the month --> get annoyed with templating language --> move back to pelican --> realized i spent too much time bouncing around, just did away with any SSG tool set, and hand craft html pages whenever i want to publish something...and now only craft journal entries for my own private diary (with an occasional post on the fediverse). :-)
annowiki 4y ago
I just linked your blog post in a separate comment because I was reading it over the weekend. I was looking for articles about writing static site generators and didn't realize most people don't roll their own templating engines. I was disappointed to see not much on templating in your post but it definitely gave me the confidence to keep going. Managed to get aspects I'd been struggling to work for months working yesterday.

dceddia 4y ago

Fun project! I really resonate with this part > But I haven’t used Haskell in years, and the overhead for me to add slightly more complex things to the site is quite large I’m starting to come around to the idea that it might be better to avoid shiny fun things in favor of having most of my active projects use the same stack, or at least the same few languages. I’ve got a blog in Jekyll (with some Ruby-based customizations), a bunch of stuff in JavaScript, some TypeScript, an app in Elixir & Phoenix, a (desktop) app in Rust, and some Rails to maintain. For me personally, Elixir is the odd one out: I learned enough to build that app, and I really enjoyed Elixir, but that knowledge was fleeting because I haven’t needed to touch the code very much. I have to relearn how Phoenix works in order to make changes. As much as it pains me to say this, I kind of wish it was in TypeScript or something, just to reduce the mental overhead. (but then, that’s a whole other can of worms because of how piecemeal things are in the Node ecosystem, so I’d end up having to learn a web framework and an ORM and some other libraries, which feels like the sort of knowledge that might be even more fleeting!)

zelphirkalt 4y ago
> As much as it pains me to say this, I kind of wish it was in TypeScript or something, just to reduce the mental overhead. If you once knew how Elixir works (syntactically), then you will be able to quickly bounce back. If comparing TypeScript with Phoenix, then you would have to relearn your in TS written functionality, in contrast to relearning Phoenix. I don't see how it being TS would help much tbh. I think what matters much more is, whether your past self left good comments and wrote readable code.
benreesman 4y ago
There’s a lot of wisdom in that observation. Knowing a lot of languages well is a super-power in some ways, but heavily polyglot environments create real pragmatic challenges. There’s a reason so many marquee shops carefully curate a (short) menu of production languages. It’s been my experience that if (and it’s a big if) you have either the need or inclination to operate a big orgy of polyglot language interop, the hellacious learning curves on NixOS and Bazel are “The Way”. It’s a big investment and only worth it if you’re pretty committed/obligated, but there is a path for those whose needs justify the cost.

munificent 4y ago

I also rewrote my blog's static site generator to a bespoke one (in Dart in my case), largely for performance reasons. Prior, I was using Jekyll and Pygments. Jekyll and Pygments are really slow. My new site generator is literally 100x faster, and I did, like, zero real optimization work. In fact, it can rebuild my entire blog and every post from scratch faster than Jekyll's watch mode (with caching set up properly) could rebuild a single page.

Keats 4y ago
I wrote Zola because at that time I was using Hugo + Pygments and while Hugo was fast, Pygments was the opposite. I'm pretty sure Zola can build a 1k page website with syntax highlighting faster than Pygments can highlight 20 pages.
radicalriddler 4y ago
Is it open sourced? Would love to have a read of some dart not written by a Googler or Xoogler. Unless... Edit: and... just read his bio

impoppy 4y ago

I've been trying to make a cool static site generator numerous times. Well, yes, it actually is a quite fun thing to tinker with, but I've found myself struggling to decide on what do I want to see in the final app and what I do not; what features might be needed by other users and what features might be an overkill. So I ultimately went with Zola[1] and encourage you to try it too! Aaaand it is written in Rust, too. It's available in Alpine Linux's repos so it's trivial to set up a CI/CD to compile your blog. [1] https://www.getzola.org/ https://www.getzola.org/

asicsp 4y ago
I moved from Jekyll to Zola and it's been great until I started tinkering with templates and stuff recently. Single binary and simple themes were main reasons to choose this. Wanted a feature that wasn't present in the version I started with (about 2 years ago). So, I got the newest version and a few things broke subtly (ex: syntax highlighting). Which led me to tinkering (I don't know much about HTML/CSS/etc). It was actually good to try a few things almost blindly and then slowly narrowing with a bit more thought process. Been a long time since I had such fun investigating unknown codebase. This even helped me move my post processing scripts to templates. And then I went a step further by reading about meta tags (things like image, description, etc) and adding them via an extra variable in the post frontmatter.
- impoppy 4y ago
  That's exactly why I never used Jekyll and don't really understand people that use it nowadays. It is AWFUL. Not only it uses greatly outdated Ruby, you also HAVE to install outdated Ruby on your machine to play with themes, serve, compile and are forced to use RVM. I know GitHub Pages can compile the page for you, but when you want to customize a theme you have to go through all that hell. It's just a terrible experience. And installing plugins is another hell.
  - BrandonS113 4y ago
    I use Jekyll for a few sites and it works perfectly. Design is bootstrap and so nothing to do with Jekyll and it always compiles the site, takes a few seconds. Works fine on my mac and linux. Would I have picked Jekyll today? no. Is it worth the cost of switching? no.

benreesman 4y ago

It’s refreshing to see a front-page Rust piece that just nails it on promoting the language. It cites concrete examples of where the rubber really meets the road: e.g. Cargo as a one-stop-shop for platform insensitive, “just works” build and dep management. It acknowledges that it’s ok for fun and novelty to be features, which it is! It steers well clear of A >> B nonsense. No preachy TED talk about memory safety. It’s respectful of the value that other languages provide in their respective niches. This is how to sell Rust. It’s novel and fun but also pragmatic and industrial grade! And if you’re into X, that’s cool too!

taftster 4y ago
Your comment helped me want to click through and read the post. I'm very tired of all the recent Rust fanboy posts, but level-headed and earnest comparisons against real implementations make for much better understanding of a new concept or language. Here are some select quotes from the article: > Please note that this is not to say that Rust this [sic] much faster than Haskell, instead see it as a comparison of two implementations. I’m sure someone could make it super fast in Haskell. > I don’t think it means that Rust is simpler or easier than Haskell, it just means that I personally had an easier time to grok Rust than Haskell.
- benreesman 4y ago
  The author is clearly an experienced and mature hacker who knows many languages well. It’s pretty difficult to maintain the shrill “X is better than everything” tone you’re referring to with a diverse portfolio of marketable skills. I don’t know how Rust, which is quite a cool language, attracted this sizable bloc of aggressively intermediate jihadis, but it’s really worrying because Rust is in fact innovative and important, but epic languages have crashed and burned over less.
  - nindalf 4y ago
    You’re really going to call people jihadis for talking about the technical strengths of a programming language? And then you’re going to act like you’re the self appointed arbiter of discourse on HN, saying that this kind of article is ok but that kind isn’t? Maybe a person who calls those they disagree with jihadis isn’t qualified to police what others say. Let’s talk about the way you talk. Here’s you from 5 days ago accusing a good tool of being a “dumpster fire” and saying “it crashes process IDs more often than Justin Bieber crashes Maseratis”. (https://news.ycombinator.com/item?id=32581005 https://news.ycombinator.com/item?id=32581005). But it gets worse, because later you start crying about being downvoted. At this point it looks like a speed run of how many HN rules you can break. And then you follow that up with tone policing others.

zelphirkalt 4y ago

Ah yes, one of my favorite projects in a newly learned language: Make a blog, based on plain text files, maybe markdown, maybe something else. But since I have done that, I feel that many existing site generators are bloated, probably, because they cover scenarios, which I would not want on my blog anyway.

mynameisash 4y ago

> Several external dependencies. > It’s annoying to install all these and to keep them up-to-date, it would be great to just have a single thing to worry about. I feel the author's pain. There's a Python2 project for ebook management that I used pretty extensively years ago, but it depended on a number of external dependencies, both as compiled shared objects and other Python libraries. It felt extremely fragile and was always a pain to get it working properly, so I eventually rewrote the core parts of the entire project for my own use (in Rust, incidentally). Now I have a single app that I can be sure will pretty much always work.

Macha 4y ago
I also have a personal Rust ebook manager because of frustrations with calibre :) Though Calibre has at least made it to Python 3 these days.

deleted 4y ago

[deleted]

cletus 4y ago

I enjoyed this post. The premise is a good one, as in for fun. A lot of these "I wrote X in Y" go on some spree of questionable assertions but "because I wanted to" is intellectually honest and understandable. I agree with a lot of the points about Rust too. Every language is a selection of tradeoffs and for me Rust's chosen tradeoffs I think are both important (eg memory safety) and reasonable. Take something like Cargo. The author says "It just works". This is my experience as well. But more importantly, dependency management and packaging were built into the language and ecosystem from the start. It still boggles my mind that any language that came after Java didn't do this (yes, I'm looking at you, Go). My heart did sink a little bit at "Using Regex of course!" though.

brightball 4y ago
I honestly highly recommend the approach. Anytime I get excited about a new technology, I rewrite my blog in it. Gives me a simple live project with real traffic and all the right moving parts. Last time I did it was with Elixir + Phoenix + PostgreSQL, without any standard or reasonable approach. It was a lot of fun and still using it today. https://www.brightball.com/articles/insanity-with-elixir-phoenix-postgresql https://www.brightball.com/articles/insanity-with-elixir-pho...

heurisko 4y ago

> On my crappy laptop a full site rebuild took 75 seconds. (Not compile, just to generate the site.) I only have 240 posts so I don’t think it should be that slow. Wow, what on earth was Hakyll doing all that time?

brabel 4y ago

Don't forget to throw your static site generator on the pile: https://jamstack.org/generators/ https://jamstack.org/generators/ 347 options and counting!

skrtskrt 4y ago

I went from Jekyll to to Hugo to avoid being bothered with futzing with ruby gems. Eventually I really appreciated the Golang approach to templating much more than Jinja/Liquid style templates, Go has kind of a composition-over-inheritance approach to templates. If I ever have time to get proficient enough with Rust I'd love to port that Go templating approach to Rust. Maybe not so much the data interpolation syntax, but rather the approach to composing template partials. But ultimately I'm familiar enough with Jinja style, Zola would probably be a breeze to adopt as well.

nlh 4y ago

Nice work and great write up! I’m a huge fan of treating one’s blog as a “homelab” for exploring new (or old) software stacks for the web, so this idea resonates a lot. One thing I’d like to try (has anyone tried this?): eschew the idea of a static site generator and go the “old school” route of dynamically generating each page on the fly, but take advantage of the modern CDN layer of the web (CloudFlare, etc.) to effectively cache and give you the speed equivalent of a static site. Yes this approach has more dependencies but I think simplifies the work done on the blog side since you can build it using whichever simple web framework appeals to you most (or none at all…this is a homelab after all). My plan (don’t we all have this plan) is to try this out in Crystal when time allows…

thrwawy74 4y ago

I really liked the reactive pattern, so one of the best static site generators I found was docpad. It used (backbone?) collections, so you could subscribe to changes on things like "new files under this path" and then rerender the part of the page that needed to change. I haven't seen this done in others. Then docpad was rewritten in ES6, and I lost all love for it. Things were so expressively concise in Coffeescript.

impoppy 4y ago
It's kind of ironic and uncool that DocPad's website is made using GitBook.
- thrwawy74 4y ago
  The author is a pretty interesting guy, but I just didn't want to follow him into ES6 land. Coffeescript is still the ranking king on concision, and so many features didn't make it in. I'm going to be sad when I can't support the transpile in my projects anymore :(

annowiki 4y ago

This is mindblowing synchronicity. I thought this article was one I had read this weekend for a second but checked the post date and saw it wasn't available yesterday. I'm working on trying to write a static site generator for my own personal site/blog in C++, been tinkering for a couple months. It started as a passion project in tribute to my dog that passed. GatsbyJS wasn't working (again) which is what my current site is built with, so I just said screw it, I'll write my own. Chose C to begin with and quickly gave up. Decided to switch to C++ because its what I'm supposed to be learning for work. I named it bluesky, after my dog that passed, Sky Blue. https://github.com/mas-4/bluesky https://github.com/mas-4/bluesky Templating is a lot harder than I had initially thought. I wanted the templating system to be as bare bones as possible, similar to another static site generator I rather like, sergey[0]. Most of these articles I found ([1], [2]) about writing your own SSG don't go into templating much, they just use an off the shelf library. Inja[3] is available for C++ but, like I said, I want something really bare bones, like if you were designing html now, you'd obviously include html-includes and html-templates. I finally got it working for includes and templates, now I have to add the markdown support, and then I plan on migrating my personal site to using it. [0]: https://sergey.cool/ https://sergey.cool/ [1]: https://www.smashingmagazine.com/2020/09/stack-custom-made-static-site-generator/ https://www.smashingmagazine.com/2020/09/stack-custom-made-s... [2]: https://blog.hamaluik.ca/posts/build-your-own-static-site-generator/ https://blog.hamaluik.ca/posts/build-your-own-static-site-ge... [3]: https://github.com/pantor/inja https://github.com/pantor/inja edit: a link

brundolf 4y ago
I've found you don't actually need a templating system, you just need format-strings. Write your rendering logic with normal code that just spits out a string Templating languages probably help with performance a bit when you're rendering at request-time, but for static stuff I don't really see much point I've got a hand-rolled static site generator in JavaScript that just uses template strings. I only have around 30 pages, to be fair, but when I make a change it re-generates the whole site before I can even alt-tab and reload the page. I assume a C++ version could be much faster
- tomjakubowski 4y ago

bArray 4y ago

I have some 381 posts (including landing pages, etc). It takes maybe 45 minutes to build (there is no cache process). I have a faster version that skips audio processing, but it still takes minutes. The problem is that it does several pandoc parses for each page: 1. Plain text - Input for audio parsing (which takes a significant amount of time, I have a version of the build that skips this). 2. Simple HTML - Input for RSS feed. 3. Full HTML - Static web page. One change I would make to pandoc is to have the option for several outputs (as was discussed years ago [1]). I think generating multiple outputs is highly common usage. One feature I will get around to writing (eventually) is automatic image compression. I had some success using jpegoptim and optipng on another project. This takes a while, so I want to build a command line tool that caches the output of a command if both the input file(s) and command do not change, and invalidate based on time. That way I could blindly do something like: image="cat-pic.png" cache-run -i $image -c "optipng -dir out/ -o 4 -strip all -fix $image" (You might want to experiment with different optimization values based on resolution, etc). [1] https://groups.google.com/g/pandoc-discuss/c/lex900rSpOM https://groups.google.com/g/pandoc-discuss/c/lex900rSpOM

marosgrego 4y ago

If you're interested in this, a while ago, for my personal photo site with image galleries[1], I wrote a fast static site generator in Rust[0], which also supports plugins now. [0] https://www.getblades.org https://www.getblades.org [1] https://grego.site/photo https://grego.site/photo

AceJohnny2 4y ago

I'm seeing a few too many comments about static site generators rebuilding everything on a change. Have the tenets of Make (and the like) from over 40 years ago been forgotten? I guess maintaining a dependency graph just isn't a high priority feature for site generators.

pitaj 4y ago
For something run relatively infrequently, I can see why it wouldn't be a huge priority. On that topic, I wish there was a generic build system like SCons written in something faster. Maybe there is (does Bazel fit my bill?) and I just don't know of it yet. What I love best about SCons is that you can just write custom builders right there. No need to drop in a shell script - you have the much better Python standard library at your fingertips. This also means it's way easier to make cross-platform.
vkou 4y ago
Computers are fast, sites get updated infrequently, and compiling a static site is a lot less computational work than dealing with the utter *********** that is C++ templates. Unless you're 'web-scale', it's complexity that doesn't solve any particular business need.

manholio 4y ago

Why even bother with a static generator when you can run rust code at essentially wire speed and dynamically generate all the required formatting on the fly? Because Rust binaries are so secure, you can expose them over the internet safely. With a little bit of glue, you can automatically recompile the CGI binary every time the rs source is touched, for a dev experience similar to interpreted web scripts.

Winsaucerer 4y ago
I prefer to take your approach, but one advantage of a static generator is that it opens up hosting location options such as github pages, dropping in an S3 type public storage, many traditional web hosts, netlify, etc.
chrismorgan 4y ago
> Because Rust binaries are so secure, you can expose them over the internet safely. That doesn’t stand to reason. Safely in that you can confidently avoid things like remote code execution, sure. But that doesn’t necessarily protect you from denial of service attacks; for example, I expect most services built on Hyper to be vulnerable to slowloris attacks <https://en.wikipedia.org/wiki/Slowloris_(computer_security) https://en.wikipedia.org/wiki/Slowloris_(computer_security)>, because I believe it still defaults to basically no sane kind of timeout, and doesn’t provide the tools to handle it properly anyway <https://github.com/hyperium/hyper/issues/1628 https://github.com/hyperium/hyper/issues/1628>. If that’s part of your threat model, you should avoid exposing it directly and use a reverse proxy. Good somewhat related reading, on a technologically simple DoS attack on Rust-based code that was largely fixed by speeding things up, fixing caching and proper use of reverse proxying (via Cloudflare because it was genuinely heavy load): https://fasterthanli.me/articles/i-won-free-load-testing https://fasterthanli.me/articles/i-won-free-load-testing
- manholio 4y ago
  Sorry for the lack of clarity, by "exposing the binary over the internet" I certainly wasn't implying re-implementing a HTTP server, HTTPS and certificate handling etc. You would of course have an nginx frontend that would cache the request and subtleties; the Rust FastCGI would see a well formed request, and rush to execute it just like any other dynamic language, PHP, Python. You can of course still do stupid things inside the rust application that explode your number of concurrent requests, like waiting for slow external resources, databases etc. For a simple static formatter, it should not be the case, it would just repeatedly hit the disk cache and some minimal CPU.

deleted 4y ago

[deleted]

eatbitseveryday 4y ago

Not to be a negative Nancy, but the phrase “for fun and profit” is quite overused that any originality is lost.

saimiam 4y ago

For my own blogging needs, I chose an even more pragmatic alternative to markdown - email! Well, email is a actually a single file CMS which contains html, plaintext, attachments/images, and metadata like when the email was sent and the subject line, With all this, it turns out you can create pretty nice looking webpages without having to futz around with md->html conversions. The huge advantage is that since I know gmail, whenever I have an idea for a blog post, I can go from idea to published in minutes. The biggest time suck tends to be finding stock photos for the post. As far as tech stack goes, it’s all serverless. I use SES to handle inbound emails and the excellent nodemailer package to parse .eml files.

rtpg 4y ago

My recommendation for people using anything like Jekyll: just bite the bullet and write a custom build script. A 200 line script that takes can handle Jinja templates or markdown or something will mean you actually understand all that is happening and aren't trying to fight the system or just figure out how to do anything. And of course don't write a plugin system or something you would share with others! Just write the thing that you need, and change it as you need to! Static site generators are great for people less comfortable doing this, but having to deal with all the _stuff_ is a major waste of time for what it's offering for people who don't need it.

hombre_fatal 4y ago
I have the opposite advice: optimize for writing, minimize janitorial stuff like tweaking your own tooling. It just distracts you from writing and offers only masturbatory value. The best path depends on your personality, but building my own blog tooling killed my writing. I'd start writing something only to realize I needed to tweak something. Then I wouldn't even write the post or I wouldn't have the energy to fix the tooling and I'd do neither. And then I stopped writing. I didn't start writing again until I used Wordpress which let me focus on writing, though after proving myself for a year, I switched to Jekyll + Github Pages so that everything was on Github.
csande17 4y ago
100% agreed. The things a static site generator does are typically very simple: load some Markdown into memory, run it through some HTML templates, and maybe scale some pictures with ImageMagick if you're feeling especially fancy. Writing a script to do those things is way easier than trying to learn and wrangle a Static Site Generator(tm). The one time static stie generators are helpful is if you really, really like the out-of-the-box look and behavior of some existing theme.
- rtpg 4y ago
  To this point: Markdown is a nice format for ingest because you can just paste in HTML. I am a bit more courageous and have some custom directives for my Markdown stuff, but it's all really 20 lines of code. I think it's tempting to add a bunch of "time saving" features for writing, but lots of time just going with the HTML is fine. Maybe if you're writing out a post every couple of days then you can focus on time saves. Until then MD will get you most of the way, and otherwise you can always hack in some special case for your unique post