59 ms·
I've been exploring an architectural pattern for a few years now which I think gives you the best of both worlds: it lets you run dynamic server-side code, but
by simonw 3y ago
I've been exploring an architectural pattern for a few years now which I think gives you the best of both worlds: it lets you run dynamic server-side code, but in a way that's both extremely inexpensive to scale up and that is self-healing if anything breaks.
I call it the Baked Data pattern: https://simonwillison.net/2021/Jul/28/baked-data/ https://simonwillison.net/2021/Jul/28/baked-data/
The key idea is that you deploy a full read-only copy of your site's data as an asset bundled into your application.
This only works for sites that don't constantly have updates, since you need to re-deploy the entire site when anything changes (just like a fully static site).
Benefits are you can deploy to inexpensive dynamic scale-to-zero hosting (like Vercel), you can handle any amount of traffic by running multiple copies of your app and if the app crashes your host can just restart the application automatically.
- snowwrestler 3y agoBack in the day, Drupal had a module called “Boost” that sort of did this. Once you activated it, it baked out every page of the site to HTML in a directory, and altered .htaccess to send all traffic there. When you updated content, it rebaked everything. https://www.drupal.org/project/boost https://www.drupal.org/project/boost
- mattlondon 3y agoI am not sure how this differs from a static site generator that also has some backend functionality? E.g. I've seen static sites with server-side search etc, comments systems etc which submit each post/comment as a separate flat file and then regenerate static pages automatically from those etc. Is it more that your approach stores stuff in sqlite and builds from that, rather than markdown files? That seems to be the only meaningful difference I can see compared to normal static sites with backend/server-side functionality?
- simonw 3y agoIt's a variant on that pattern. The key idea here is that your server-side stuff is completely read-only - so comment systems aren't supported - because the server-side data is treated as a deployment asset. Have you used platforms like Vercel? Their one big limitation is that you can't use persistent storage with them - if you want to talk to a database of some sort you need to add an extra database vendor. The baked data pattern works around that by bundling read-only data (usually a SQLite file) with your deployment.
- mattlondon 3y agoOh ok so you still have a "dynamic" site that is database driven, but that database is read-only and stored in a pre-generated sqlite file that is deployed with the rest of the site assets? I don't want to piss on your parade here - perhaps I just don't get this pattern - but isn't this all of the downsides of a dynamic site but without any of the benefits? You are still running slow server side stuff that relies on doing per-request database stuff and all the complexity and security headaches that go along with it (upgrading PHP or Wordpress or whatever), but with none of the benefits of server side functionality and databases (i.e. it is all read-only).
- simonw 3y agoI talk about the benefits of the approach in quite some detail here: https://simonwillison.net/2021/Jul/28/baked-data/#benefits-of-baked-data https://simonwillison.net/2021/Jul/28/baked-data/#benefits-o... - does that help clarify?
- sixdimensional 3y agoTangentially, I've been exploring something kind of fascinating... So, in compiled binary executables it was not uncommon to include encoded binary resources, like files or images. Not tons of them, because they would explode the size of the executable. For example, for C or C++ https://github.com/graphitemaster/incbin https://github.com/graphitemaster/incbin So here's the interesting part - we decided to build executables in a way that data in those executables could not be changed (makes sense since it's compiled and the compiled code runs on the machine, and for security reasons). That said, think about what a container (e.g. docker) is. A running container is kind of like a packaged executable, except it also has a filesystem. So if you pack data (which can be modified and runtime!) into a container, it's a similar concept to an embedded resource in an executable, except it can change. Now, in a container, any changes made to data at runtime inside the container won't persist unless the container is given persistent storage. What I've been wondering lately is why we didn't invent some kind of single "executable plus volatile data space embedded within the executable", so that programs and data (say, a database) could couple together into a single file. Just a musing but tangentially related to your "baked data" - basically, embedded resources in executables just embeds the encoded data right into an executable. For scripting languages, of course, we can just make a script file that contains a variable with the encoded data as base64 directly. For the latter 2, it's only good for relatively small static data of course, but it would be interesting to build tech that somehow lifted that constraint of executables.
- zX41ZdbW 3y agoThanks for the link. We also make static websites by direct inclusion their HTML code in the C++ binary of ClickHouse. I've tried to use this library but found a bug: https://github.com/graphitemaster/incbin/issues/61 https://github.com/graphitemaster/incbin/issues/61
- simonw 3y agoHave you seen redbean? It's exploring ideas that are in that kind of space: https://redbean.dev/ https://redbean.dev/
- 3y ago
- muhammadusman 3y agoI’ve been following this pattern and now it has a name! I host my projects page like this (https://usmanity.com/projects https://usmanity.com/projects). I didn’t want to edit an html file each time I wanted to add new projects to the list or change the details on an existing one so I used Notion and I bake the data before committing to GitHub.