5 ms·

I've tried so many static generators, but they just all fail miserably past a certain point. For a simple blog, just text, etc, they're great. But once you get

by alan_n 8y ago

I've tried so many static generators, but they just all fail miserably past a certain point. For a simple blog, just text, etc, they're great. But once you get into adding images, rss, or any sort of complex layout and routing, none of them are really designed to handle that much complexity. Metalsmith came the closest because of it's plugin system, but only because I ended up writing my own plugins and just basically writing a static generator on top of it. But it can't do live previewing at all which is really annoying. Recently I tried Nuxt because I really like Vue, but although these new types of generators are great for handling layout, and the live reloading rocks, they're not really designed for a truly static site, the pre-rendering includes a lot of unnecessary bloat. There's basically a copy of every post both in the html and the javascript, it's also super hard to control it so it's only the posts for that one page. You can remove the scripts manually but then you lose all js functionality. I gave up trying to find a sane workaround. They're not really meant for content that's truly static that you don't need to update. I've given up on this point and have just started to write my own, focusing on fast live reloading and easy config (like Vue's cli) because that's one of the other huge problems with all generators.

ianwalter 8y ago
Have you tried Gatsby? I have tried many SSGs and really like Vue as well, but I tried Nuxt out a while ago and it wasn't flexible enough for my liking. I'm not sure I 100% understand your issue, but I haven't found anything that I can't do in Gatsby so far and I'm blown away by the sources/graphql feature.
- alan_n 8y ago
  No, I don't really like react, and it's probably bound to have the same problem as Nuxt. The issue is that the site, at least with Nuxt, is pre-rendered to HTML, right, but it still needs to have Vue (because usually you would want to then use it to then refresh the content). The thing is this, apart from having to have Vue and a bunch of JS, it also adds your posts inline to the script (because Vue can't just extract the content from the DOM (well, it technically can, but this would be a mess for varying reasons). This is a bunch of bloat since I want the site to be static to be fast. You can delete the script tags after the rendering (using something like cheerio), but this gets rid of all your scripts (which I still need some small ones to for example, lazy load images). I could keep them separate and then insert them back in but then I lose the pleasure of using Vue. It's kind of impossible to solve as far as I see.
  - ianwalter 8y ago
    Ok I see what you're saying but I don't see why it's a big deal. The JavaScript is non-blocking right? So the static page is still displayed just as quickly. Gatsby does code splitting so the bundles are pretty small and keep in mind it takes advantage of preload/prefetch. Using JS to render a new page is going to be faster than loading the entire static page. Maybe you should try testing out real world performance vs something like Jekyll.
    - alan_n 8y ago
      Technically, yes it's non-blocking, that's not the problem. For something like, for example, a store front, I agree, these framework renderers are great because/when: - You have little data that needs to be pre-loaded (e.g. item name, price, description, thumbnail, etc). - If you do have a lot of data, then your website is of the type that users revisit a lot (e.g. a documentation site). - The data is modular (it's not one chunk of data) and can benefit from using a framework to organize it. - Very little content changes from page to page (in a store front, the majority of the layout is re-used). - There's a lot of DOM manipulation you want to do (e.g. checkout, cart, etc) This is not true of content heavy blogs: - It's a lot of data which must at a minimum be duplicated twice. - The data is not modular. In one markdown file you have to process the text, images, videos, embeds, etc., all at once. So, for example, you can't take advantage of your framework to generate image tags. - There's almost zero DOM manipulation needed (at most I might have a portfolio page which needs some js, a lazy-load script, and some share buttons). - A lot of content changes from page to page (it overshadows any content they might have in common). - The content is not exactly of the type users would revisit often. With Nuxt (again don't know with Gatsby, and I think they're working in Nuxt to fix this), you also have to jump through hoops to pre-convert your markdown to html because otherwise you're converting on the client side. Found this out when I looked at the bundles and there were a bunch of emojis in it, turns out it was my markdown converter getting imported into the bundle. Now, let's suppose this wasn't that difficult, there was some function were you could pre-convert this data and it wasn't including the libraries to do it in the bundle. Except this is THE pain point of SSGs for blogs. The majority of the problems are here and you would essentially end up writing a static generator inside this function (when I realized this I gave up on Nuxt and similar framework renderers). I would have gained very little for that headache, and how big I would be willing to have the bundle size is a LOT smaller. And while, yes, bundle sizes are usually small and don't matter for the cases I outlined in the beginning, this is not true, like I said, of a content heavy blog. I don't have a nice a proper benchmark, but I do have some notes from when I tried nuxt. These are the sizes of a single test index page (plain text + a couple lines of css for basic styling) and the full loading times with unthrottled and fully throttled (though this is a bit deceiving* ): Nuxt w/o Content ~170kb - 250ms - 7.8s Nuxt w/ 10 Blog Posts ~500kb - 70ms - 14s Nuxt w/ 10 Blog Posts w/ All Scripts Removed ~110kb - 8ms - 4.3s You can see how my text is about ~100kb, framework is ~170kb, but the script it produces is 500kb because, I actually forgot, it's reproducing my content 3 times (did not bother to investigate why, just twice is bad enough imo)! * Now, yes technically the throttled load times don't matter because the static html is loaded (they all paint, even throttled at around the same time), so there's no waiting for the page to load, but who knows what the rest of the background loading does to interactivity (on a proper site, with images, buttons, etc, this was just text) on slow phones with slow internet. Why would I do this to my users? It's not like it's even pre-loading anything useful (this test was with a single page, no other existing pages), imagine if it were. > Using JS to render a new page is going to be faster than loading the entire static page. Yes, the lack of a refresh is nice, I'll give you that and it's true that subsequent navigations are faster, but in my case I don't think it's worth the overhead. Plus, so much content changes per page, I could always just fetch the next page and replace the body manually without the overhead of a framework. Don't get me wrong though, I would probably use a framework based SSG for say, a code blog, where it has more pros than cons. So I can see where most developers are coming from when they use them / design them.
atmoz 8y ago
What do you mean by adding images? Do you want to login via the browser and manage a media library? Are creating RSS templates [1] considered as "handling the complexity", or is that too much hassle? [1]: https://github.com/atmoz/atmoz-net/blob/master/.ply/feed.template https://github.com/atmoz/atmoz-net/blob/master/.ply/feed.tem...
- cmroanirgo 8y ago
  > What do you mean by adding images? For me, lack of nice (ie. controllable, but 'automagic') image handling is a huge bug bear in nearly every cms out there, static or otherwise. Perhaps I didn't find the right platform/plugin... In my case, I wanted to have 1 source image that was resized at various (author controllable) sizes and then saved them, while also generating supporting HTML (eg. 'srcset' to automatically swap between them). I ended up with a couple of ways to template it in [0]: For a thumbnail: <img src="{{image #thumb{w:50} }}"> For srcset: <img src="{{image}}" srcset="{{image #srcset{w:'256,512,800',orig:'1152'} }}" sizes=...> [0] https://github.com/ergo-cms/plugin-thumbnail https://github.com/ergo-cms/plugin-thumbnail. (Note the entire project itself stalled some time ago, and wouldn't recommend it here -- even though I do dogfood it) EDIT: reorder wording
  - thirdsun 8y ago
    I think this is a task that is handled by an optional CMS, not the static site generator itself, though I agree it would be handy to have a little helper snippet to do that during build. There are CMS solutions for static sites which will probably handle images, and all kinds of content, easily. Check out Dato[1], Siteleaf[2] or Netlify CMS[3]. Alternatively you can use external tools to prepare your images and simply rely on the standard HTML srcset within your static site generator of choice. I use RetroBatch[4] to prepare my images. [1] https://www.datocms.com/ https://www.datocms.com/ [2] https://www.siteleaf.com https://www.siteleaf.com [3] https://www.netlifycms.org/ https://www.netlifycms.org/ [4] https://flyingmeat.com/retrobatch/ https://flyingmeat.com/retrobatch/
  - atmoz 8y ago
    I see, so image resizing on build. I like the idea. That should be doable. I may consider implementing something like that in my own SSG some day.
rogerbinns 8y ago
> none of them are really designed to handle that much complexity Be careful what you wish for! I had used Nikola, but every release kept handling more complexity. Which meant an ever increasing number of options, but most annoyingly a large number of warnings due to config options that had changed, been obsoleted, interacted differently etc. Nikola is one of those tools that generates an initial config file for you, which exacerbates the problem since you probably don't use all the config items, and then have this option warning mess due to those items. Yes you can install a version pinned version of the tool, but then have to do the same with all the dependencies. Before you know it, Wordpress seems easy and trivial.
- alan_n 8y ago
  Never tried Nikola, but I'm painfully aware of the problem. I think I've tried and migrated through enough SSGs at this point though that I think I can write something better. The main problem IS the config imo. That and plugins (and their configs). Don't know if you've tried Vue's new CLI, but my idea is if I don't want to or can't abstract away the complexity without imposing limits on flexibility, the next best thing is to "visually" abstract it by making the configuration happen in a UI like Vue's CLI. This will get rid of errors because you would be able to stop the user choosing conflicting actions, and things that cannot be abstracted in a config, usually CAN be in a UI. Main one, for example, and the one I'm focusing on is routing. There's tons there that can be abstracted in a UI there. You can also add hints, links, etc in a UI. There would be no need to be looking at the docs all the time. In the future hopefully I can expand the idea further to include an editor, but I have to keep my goals reasonable or I will never finish. As for how many features and how complex, after migrating my site so many times I feel the following should be included (because if they're third party plugins they end up un-maintained): RSS, Sitemaps, Image Resizing, and maybe Embeds PS: I tried wordpress with a static generator plugin once, it was horrifying, never going back. Even ignoring the static generator plugin, it still did barely better than the previous SSG I'd been using. The plugins were all of wildly varying quality. In particular I was constantly fighting the one that resized my images.
brillout 8y ago
I know this lock-in pain quite too well. That's why I've started a web framework with the goal of being entirely flexible. I've made all code ejectable, minimized/simplified the glue code (so that you eject beautiful code) and made the stack flexible (you can remove/add a frontend/backend/ORM/database to your app easily and at any time). That should give you the flexibility you are looking for. It's called Reframe (https://github.com/reframejs/reframe https://github.com/reframejs/reframe). (The default starter "react-frontend" creates a static site.)
- alan_n 8y ago
  This looks interesting for making SPAs and more app-like things, but then I haven't found things like Nuxt to be inflexible for that (never actually created a SPA with it though, just attempted to use it as an SSG). Doesn't look like it would work for me though for the same reasons Nuxt doesn't. It's not really lock-in pain that's my problem btw, but more config fatigue and missing features, and if the SSG has a plugin system to counteract this, then there's the problem that most don't work well or age quickly (imo this is usually because the plugin system is not strict enough), and regardless of whether they work, you end up with this patchwork of plugins, with a bunch of hacks and glue code to make them interact nicely, it's awful.
epage 8y ago
> But once you get into adding images, rss, or any sort of complex layout and routing, none of them are really designed to handle that much complexity. Could you give examples of these pain points? As a maintainer of one of the SSGs on the list, I'm wanting to better understand users pain points, even if they aren't my users.
- alan_n 8y ago
  Well just off the top of my head, here's what I remember from most to least painful. Some of these are not 100% fixable though or I'm not sure the best way to fix them yet without adding more complexity or some sort of editor to my SSG. - Layout & Routing - Having "special" pages is usually impossible with "structured" SSGs (like Hugo). For example, say you want a page that's your portfolio and only contains posts with x and y tags and you want it to look different than normal pages. Also see rss below. Or for example archives, where "/year" = "/year/index.html" but also "/year/month/" = "/year/month/index.html" and "/year/month/postname" = "year/month/postname.index.html". - Sitemaps - I have videos on my blog and usually you want to add those to your sitemap properly. To do this I had to patch together a plugin to fetch the video info from the youtube api. - Smart tags/embeds - Similarly I had to do something similar to fetch embeds from just the link like "[instagram]link[/instagram]". You have to do this with every provider you want, ugh, awful. Also means fighting the markdown converter you chose sometimes. - Images - I run a blog with lots of images and different views of those images, so I need to have them all in at least thumbnail size and the regular size, sometimes the original as well, but not always. I used to batch convert new ones by hand but that was a pain. I eventually patched together a metalsmith plugin for this to automate the conversion of originals to "regular" (max post width) and thumbnails but it needs improvement. Edit: And I forgot, actually adding the correct attributes needed for lazy-loading, etc, to the image tags was also a pain. - RSS, Most do not come equipped to handle rss, and available plugins are too simple (for routing that is, like having an rss feed per tag, per archive page, etc). - Layout Language - Depends on the SSG, most allow choosing or these days are super flexible (e.g. Nuxt) but those that don't and just use one template language, argh! Template languages like handlebars are just not flexible enough. This was the easiest to fix though. I just use ejs now, basically allows any valid javascript, love it.

by alan_n 8y ago

ianwalter 8y ago

Have you tried Gatsby? I have tried many SSGs and really like Vue as well, but I tried Nuxt out a while ago and it wasn't flexible enough for my liking. I'm not sure I 100% understand your issue, but I haven't found anything that I can't do in Gatsby so far and I'm blown away by the sources/graphql feature.

alan_n 8y ago
No, I don't really like react, and it's probably bound to have the same problem as Nuxt. The issue is that the site, at least with Nuxt, is pre-rendered to HTML, right, but it still needs to have Vue (because usually you would want to then use it to then refresh the content). The thing is this, apart from having to have Vue and a bunch of JS, it also adds your posts inline to the script (because Vue can't just extract the content from the DOM (well, it technically can, but this would be a mess for varying reasons). This is a bunch of bloat since I want the site to be static to be fast. You can delete the script tags after the rendering (using something like cheerio), but this gets rid of all your scripts (which I still need some small ones to for example, lazy load images). I could keep them separate and then insert them back in but then I lose the pleasure of using Vue. It's kind of impossible to solve as far as I see.
- ianwalter 8y ago
  Ok I see what you're saying but I don't see why it's a big deal. The JavaScript is non-blocking right? So the static page is still displayed just as quickly. Gatsby does code splitting so the bundles are pretty small and keep in mind it takes advantage of preload/prefetch. Using JS to render a new page is going to be faster than loading the entire static page. Maybe you should try testing out real world performance vs something like Jekyll.
  - alan_n 8y ago
    Technically, yes it's non-blocking, that's not the problem. For something like, for example, a store front, I agree, these framework renderers are great because/when: - You have little data that needs to be pre-loaded (e.g. item name, price, description, thumbnail, etc). - If you do have a lot of data, then your website is of the type that users revisit a lot (e.g. a documentation site). - The data is modular (it's not one chunk of data) and can benefit from using a framework to organize it. - Very little content changes from page to page (in a store front, the majority of the layout is re-used). - There's a lot of DOM manipulation you want to do (e.g. checkout, cart, etc) This is not true of content heavy blogs: - It's a lot of data which must at a minimum be duplicated twice. - The data is not modular. In one markdown file you have to process the text, images, videos, embeds, etc., all at once. So, for example, you can't take advantage of your framework to generate image tags. - There's almost zero DOM manipulation needed (at most I might have a portfolio page which needs some js, a lazy-load script, and some share buttons). - A lot of content changes from page to page (it overshadows any content they might have in common). - The content is not exactly of the type users would revisit often. With Nuxt (again don't know with Gatsby, and I think they're working in Nuxt to fix this), you also have to jump through hoops to pre-convert your markdown to html because otherwise you're converting on the client side. Found this out when I looked at the bundles and there were a bunch of emojis in it, turns out it was my markdown converter getting imported into the bundle. Now, let's suppose this wasn't that difficult, there was some function were you could pre-convert this data and it wasn't including the libraries to do it in the bundle. Except this is THE pain point of SSGs for blogs. The majority of the problems are here and you would essentially end up writing a static generator inside this function (when I realized this I gave up on Nuxt and similar framework renderers). I would have gained very little for that headache, and how big I would be willing to have the bundle size is a LOT smaller. And while, yes, bundle sizes are usually small and don't matter for the cases I outlined in the beginning, this is not true, like I said, of a content heavy blog. I don't have a nice a proper benchmark, but I do have some notes from when I tried nuxt. These are the sizes of a single test index page (plain text + a couple lines of css for basic styling) and the full loading times with unthrottled and fully throttled (though this is a bit deceiving* ): Nuxt w/o Content ~170kb - 250ms - 7.8s Nuxt w/ 10 Blog Posts ~500kb - 70ms - 14s Nuxt w/ 10 Blog Posts w/ All Scripts Removed ~110kb - 8ms - 4.3s You can see how my text is about ~100kb, framework is ~170kb, but the script it produces is 500kb because, I actually forgot, it's reproducing my content 3 times (did not bother to investigate why, just twice is bad enough imo)! * Now, yes technically the throttled load times don't matter because the static html is loaded (they all paint, even throttled at around the same time), so there's no waiting for the page to load, but who knows what the rest of the background loading does to interactivity (on a proper site, with images, buttons, etc, this was just text) on slow phones with slow internet. Why would I do this to my users? It's not like it's even pre-loading anything useful (this test was with a single page, no other existing pages), imagine if it were. > Using JS to render a new page is going to be faster than loading the entire static page. Yes, the lack of a refresh is nice, I'll give you that and it's true that subsequent navigations are faster, but in my case I don't think it's worth the overhead. Plus, so much content changes per page, I could always just fetch the next page and replace the body manually without the overhead of a framework. Don't get me wrong though, I would probably use a framework based SSG for say, a code blog, where it has more pros than cons. So I can see where most developers are coming from when they use them / design them.

atmoz 8y ago

What do you mean by adding images? Do you want to login via the browser and manage a media library? Are creating RSS templates [1] considered as "handling the complexity", or is that too much hassle? [1]: https://github.com/atmoz/atmoz-net/blob/master/.ply/feed.template https://github.com/atmoz/atmoz-net/blob/master/.ply/feed.tem...

cmroanirgo 8y ago
> What do you mean by adding images? For me, lack of nice (ie. controllable, but 'automagic') image handling is a huge bug bear in nearly every cms out there, static or otherwise. Perhaps I didn't find the right platform/plugin... In my case, I wanted to have 1 source image that was resized at various (author controllable) sizes and then saved them, while also generating supporting HTML (eg. 'srcset' to automatically swap between them). I ended up with a couple of ways to template it in [0]: For a thumbnail: <img src="{{image #thumb{w:50} }}"> For srcset: <img src="{{image}}" srcset="{{image #srcset{w:'256,512,800',orig:'1152'} }}" sizes=...> [0] https://github.com/ergo-cms/plugin-thumbnail https://github.com/ergo-cms/plugin-thumbnail. (Note the entire project itself stalled some time ago, and wouldn't recommend it here -- even though I do dogfood it) EDIT: reorder wording
- thirdsun 8y ago
  I think this is a task that is handled by an optional CMS, not the static site generator itself, though I agree it would be handy to have a little helper snippet to do that during build. There are CMS solutions for static sites which will probably handle images, and all kinds of content, easily. Check out Dato[1], Siteleaf[2] or Netlify CMS[3]. Alternatively you can use external tools to prepare your images and simply rely on the standard HTML srcset within your static site generator of choice. I use RetroBatch[4] to prepare my images. [1] https://www.datocms.com/ https://www.datocms.com/ [2] https://www.siteleaf.com https://www.siteleaf.com [3] https://www.netlifycms.org/ https://www.netlifycms.org/ [4] https://flyingmeat.com/retrobatch/ https://flyingmeat.com/retrobatch/
- atmoz 8y ago
  I see, so image resizing on build. I like the idea. That should be doable. I may consider implementing something like that in my own SSG some day.

rogerbinns 8y ago

> none of them are really designed to handle that much complexity Be careful what you wish for! I had used Nikola, but every release kept handling more complexity. Which meant an ever increasing number of options, but most annoyingly a large number of warnings due to config options that had changed, been obsoleted, interacted differently etc. Nikola is one of those tools that generates an initial config file for you, which exacerbates the problem since you probably don't use all the config items, and then have this option warning mess due to those items. Yes you can install a version pinned version of the tool, but then have to do the same with all the dependencies. Before you know it, Wordpress seems easy and trivial.

alan_n 8y ago
Never tried Nikola, but I'm painfully aware of the problem. I think I've tried and migrated through enough SSGs at this point though that I think I can write something better. The main problem IS the config imo. That and plugins (and their configs). Don't know if you've tried Vue's new CLI, but my idea is if I don't want to or can't abstract away the complexity without imposing limits on flexibility, the next best thing is to "visually" abstract it by making the configuration happen in a UI like Vue's CLI. This will get rid of errors because you would be able to stop the user choosing conflicting actions, and things that cannot be abstracted in a config, usually CAN be in a UI. Main one, for example, and the one I'm focusing on is routing. There's tons there that can be abstracted in a UI there. You can also add hints, links, etc in a UI. There would be no need to be looking at the docs all the time. In the future hopefully I can expand the idea further to include an editor, but I have to keep my goals reasonable or I will never finish. As for how many features and how complex, after migrating my site so many times I feel the following should be included (because if they're third party plugins they end up un-maintained): RSS, Sitemaps, Image Resizing, and maybe Embeds PS: I tried wordpress with a static generator plugin once, it was horrifying, never going back. Even ignoring the static generator plugin, it still did barely better than the previous SSG I'd been using. The plugins were all of wildly varying quality. In particular I was constantly fighting the one that resized my images.

brillout 8y ago

I know this lock-in pain quite too well. That's why I've started a web framework with the goal of being entirely flexible. I've made all code ejectable, minimized/simplified the glue code (so that you eject beautiful code) and made the stack flexible (you can remove/add a frontend/backend/ORM/database to your app easily and at any time). That should give you the flexibility you are looking for. It's called Reframe (https://github.com/reframejs/reframe https://github.com/reframejs/reframe). (The default starter "react-frontend" creates a static site.)

alan_n 8y ago
This looks interesting for making SPAs and more app-like things, but then I haven't found things like Nuxt to be inflexible for that (never actually created a SPA with it though, just attempted to use it as an SSG). Doesn't look like it would work for me though for the same reasons Nuxt doesn't. It's not really lock-in pain that's my problem btw, but more config fatigue and missing features, and if the SSG has a plugin system to counteract this, then there's the problem that most don't work well or age quickly (imo this is usually because the plugin system is not strict enough), and regardless of whether they work, you end up with this patchwork of plugins, with a bunch of hacks and glue code to make them interact nicely, it's awful.

epage 8y ago

> But once you get into adding images, rss, or any sort of complex layout and routing, none of them are really designed to handle that much complexity. Could you give examples of these pain points? As a maintainer of one of the SSGs on the list, I'm wanting to better understand users pain points, even if they aren't my users.

alan_n 8y ago
Well just off the top of my head, here's what I remember from most to least painful. Some of these are not 100% fixable though or I'm not sure the best way to fix them yet without adding more complexity or some sort of editor to my SSG. - Layout & Routing - Having "special" pages is usually impossible with "structured" SSGs (like Hugo). For example, say you want a page that's your portfolio and only contains posts with x and y tags and you want it to look different than normal pages. Also see rss below. Or for example archives, where "/year" = "/year/index.html" but also "/year/month/" = "/year/month/index.html" and "/year/month/postname" = "year/month/postname.index.html". - Sitemaps - I have videos on my blog and usually you want to add those to your sitemap properly. To do this I had to patch together a plugin to fetch the video info from the youtube api. - Smart tags/embeds - Similarly I had to do something similar to fetch embeds from just the link like "[instagram]link[/instagram]". You have to do this with every provider you want, ugh, awful. Also means fighting the markdown converter you chose sometimes. - Images - I run a blog with lots of images and different views of those images, so I need to have them all in at least thumbnail size and the regular size, sometimes the original as well, but not always. I used to batch convert new ones by hand but that was a pain. I eventually patched together a metalsmith plugin for this to automate the conversion of originals to "regular" (max post width) and thumbnails but it needs improvement. Edit: And I forgot, actually adding the correct attributes needed for lazy-loading, etc, to the image tags was also a pain. - RSS, Most do not come equipped to handle rss, and available plugins are too simple (for routing that is, like having an rss feed per tag, per archive page, etc). - Layout Language - Depends on the SSG, most allow choosing or these days are super flexible (e.g. Nuxt) but those that don't and just use one template language, argh! Template languages like handlebars are just not flexible enough. This was the easiest to fix though. I just use ejs now, basically allows any valid javascript, love it.