4 ms·

I'm already self hosting my own cookieless analytics, and before, I hosted Drupal (LEMP stack) and Apache Solr on separate servers. I'm used to self-hosting, an

I'm already self hosting my own cookieless analytics, and before, I hosted Drupal (LEMP stack) and Apache Solr on separate servers. I'm used to self-hosting, and any comment solution I use will be self-hosted as well (see: https://github.com/geerlingguy/jeffgeerling-com/issues/167 https://github.com/geerlingguy/jeffgeerling-com/issues/167) The code surface with SSG + 1 or 2 small self-hosted OSS tools is much, much smaller than it ever was running Drupal or another CMS.

jtbayly 9mo ago
But how in the world will you shove a decent search into a static site? I really want to know because there is a Drupal 7 site that I need to migrate to something but I need good search on it (I’m using solr now). Edit: I should have specified that I need more functionality than just word searching. I need filtering (ie faceted search) too. I’ve used a SSG that uses a JavaScript index and appreciate it, but that’s not going to cut it for this project.
- seanwilson 9mo ago
  See https://gohugo.io/tools/search/ https://gohugo.io/tools/search/. Not sure how well they scale to thousands of posts, but they work by statically generating multiple static search index files at build time that are queried via client JavaScript when hosted. The search UX is actually really good because they tend to respond instantly as you type your query and allow complex queries.
  - jtbayly 9mo ago
    Do you happen to know if any of those support faceted search (ie searching and filtering by date, category, etc)?
    - seanwilson 9mo ago
      I've used https://lunrjs.com/guides/getting_started.html https://lunrjs.com/guides/getting_started.html briefly and it has lots of options for things like different fields, complex queries, fuzzy searching and wildcards. Didn't notice anything specific about dates but you could always add date as a field then filter out a date range manually at the end. I'm sure there's better libraries now as well.
      - 1dom 9mo ago
        We've gone from SSGs for ease, speed and reduced resources, to talking about implementing search with multiple megabyte client side indexes and hundereds of thousands of prerendered search result pages. When does this become 1 step forward with the SSG and 2 steps back with search solutions like this?
        seanwilson 9mo ago
        You don't pre-render the search pages. You generate some search index files on the build step (something like a map of keywords to matching post URLs), and then client side JavaScript requests the search index files it needs on demand and generates the search results on the page. For a modest blog, I think the compressed index can be a few 100K. A single large image can be bigger than that. Nothing is perfect, but the above is really simple to host, is low maintenance, and easy to secure.
        1dom 9mo ago
        This is just server side search with more steps, since the index needs to be selectively split and returned, and then search results page generated from the index. Sorry, I don't mean to come across as disagreeable. You're right, nothing is perfect, and this is obviously a workable and usable solution. My issue is if we analyse it beyond "it looks like it works", it starts to look like a slightly worse solution than what we already had. Nothing wrong with moving backwards in some direction, as long as we can clearly point to some benefit we're getting elsewhere. My issues with SSGs is most of the benefits they offer end up undermined if you want any interactivity. This is a good example of that, as you end up compromising on build time and page load time compared to an SSR search solution.
        seanwilson 9mo ago
        > it starts to look like a slightly worse solution than what we already had You don't see how the server based solution is an order of magnitude more effort to maintain, monitor, optimize, and secure compared to hosting static files? Especially if search is a minor feature of your site, keeping the hosting simple can be a very reasonable tradeoff. Lots of blogs/sites also do fine with only category and tag filtering (most SSGs have this built in) without text search, and users can use a search engine if they need more.
- stavros 9mo ago
  The usual way is to create an index on generation time and serve it statically. JS just uses the index to do the search. It's a big file, so I'm not saying it's a great solution for everyone, but it works reasonably well. Of course, for my site I just redirect the user to a search engine plus `site:stavros.io`.
- anotherevan 9mo ago
  > there is a Drupal 7 site that I need to migrate to something You may be interested in Backdrop, which is a maintained fork of Drupal 7. https://backdropcms.org/ https://backdropcms.org/ (No experience with it personally. Only know about it from a friend who uses it.)
1dom 9mo ago
Yes, SSG pipeline + 1 or 2 small self-hosted OSS tools is way simpler than Drupal. But all you've done in bought on all the pain and compromise of having to think from an SSG perspective, and that created problems which you've already identified you'll figure out in future I'm suggesting 2 or 3 small self-hosted OSS tools, where one is a small hand crafted server that basically takes a markdown file, renders it, and serves it as plain HTML with a header/footer. This is more homogenous, fewer unique parts/processes, and doesn't have the constraint of dealing with an SSG. I remember my own personal pain from 2010 - 2016ish of managing Drupal and Joomla. I did exactly the same as you in 2016 and went all in on SSGs and in 2024, I realised all of the above. I feel like I wasted years of potential development time reinventing basic personal website features to try and work with an SSG and you literally have a ticket to do just that: https://github.com/geerlingguy/jeffgeerling-com/issues/167 https://github.com/geerlingguy/jeffgeerling-com/issues/167. 1 of your 3 solutions involves letting someone else host your comments:( A custom framework/server is the end destination for all nerdy personal websites - I can't wait to see what you make when you realise this:) edit/p.s. I love you and all your work. Sorry for sounding disagreeable, I'm excited to see what you learn from you SSG journey, I hope you prove me wrong!
- geerlingguy 9mo ago
  Definitely not disagreeable, more just "there are two right answers" ;) For me, an unstated reason for SSG is being able to scale to millions of requests per hour without scaling up my server to match. Serving static HTML is insanely easy, even on a cheap $5-10/month VPS. Serving anything dynamic at all is an order of magnitude harder. Though... I've been using Cloudflare since 2022, after I started getting regular targeted DDoSes (was fun initially, seeing someone target different parts of Drupal, until I just locked down everything except for the final comment pages). The site will randomly get 1-2 million requests in a few minutes, and now Cloudflare eats those quickly, instead of my VPS getting locked up. Ideally, I'll be able to host without Cloudflare in front at some point, but every month, because of one or two attacks, the site's using 25-35 TB of bandwidth (at least according to CF).
  - 1dom 9mo ago

jtbayly 9mo ago

But how in the world will you shove a decent search into a static site? I really want to know because there is a Drupal 7 site that I need to migrate to something but I need good search on it (I’m using solr now). Edit: I should have specified that I need more functionality than just word searching. I need filtering (ie faceted search) too. I’ve used a SSG that uses a JavaScript index and appreciate it, but that’s not going to cut it for this project.

seanwilson 9mo ago
See https://gohugo.io/tools/search/ https://gohugo.io/tools/search/. Not sure how well they scale to thousands of posts, but they work by statically generating multiple static search index files at build time that are queried via client JavaScript when hosted. The search UX is actually really good because they tend to respond instantly as you type your query and allow complex queries.
- jtbayly 9mo ago
  Do you happen to know if any of those support faceted search (ie searching and filtering by date, category, etc)?
  - seanwilson 9mo ago
    I've used https://lunrjs.com/guides/getting_started.html https://lunrjs.com/guides/getting_started.html briefly and it has lots of options for things like different fields, complex queries, fuzzy searching and wildcards. Didn't notice anything specific about dates but you could always add date as a field then filter out a date range manually at the end. I'm sure there's better libraries now as well.
    - 1dom 9mo ago
      We've gone from SSGs for ease, speed and reduced resources, to talking about implementing search with multiple megabyte client side indexes and hundereds of thousands of prerendered search result pages. When does this become 1 step forward with the SSG and 2 steps back with search solutions like this?
      - seanwilson 9mo ago
        You don't pre-render the search pages. You generate some search index files on the build step (something like a map of keywords to matching post URLs), and then client side JavaScript requests the search index files it needs on demand and generates the search results on the page. For a modest blog, I think the compressed index can be a few 100K. A single large image can be bigger than that. Nothing is perfect, but the above is really simple to host, is low maintenance, and easy to secure.
        1dom 9mo ago
        This is just server side search with more steps, since the index needs to be selectively split and returned, and then search results page generated from the index. Sorry, I don't mean to come across as disagreeable. You're right, nothing is perfect, and this is obviously a workable and usable solution. My issue is if we analyse it beyond "it looks like it works", it starts to look like a slightly worse solution than what we already had. Nothing wrong with moving backwards in some direction, as long as we can clearly point to some benefit we're getting elsewhere. My issues with SSGs is most of the benefits they offer end up undermined if you want any interactivity. This is a good example of that, as you end up compromising on build time and page load time compared to an SSR search solution.
        seanwilson 9mo ago
        > it starts to look like a slightly worse solution than what we already had You don't see how the server based solution is an order of magnitude more effort to maintain, monitor, optimize, and secure compared to hosting static files? Especially if search is a minor feature of your site, keeping the hosting simple can be a very reasonable tradeoff. Lots of blogs/sites also do fine with only category and tag filtering (most SSGs have this built in) without text search, and users can use a search engine if they need more.
stavros 9mo ago
The usual way is to create an index on generation time and serve it statically. JS just uses the index to do the search. It's a big file, so I'm not saying it's a great solution for everyone, but it works reasonably well. Of course, for my site I just redirect the user to a search engine plus `site:stavros.io`.
anotherevan 9mo ago
> there is a Drupal 7 site that I need to migrate to something You may be interested in Backdrop, which is a maintained fork of Drupal 7. https://backdropcms.org/ https://backdropcms.org/ (No experience with it personally. Only know about it from a friend who uses it.)

1dom 9mo ago

Yes, SSG pipeline + 1 or 2 small self-hosted OSS tools is way simpler than Drupal. But all you've done in bought on all the pain and compromise of having to think from an SSG perspective, and that created problems which you've already identified you'll figure out in future I'm suggesting 2 or 3 small self-hosted OSS tools, where one is a small hand crafted server that basically takes a markdown file, renders it, and serves it as plain HTML with a header/footer. This is more homogenous, fewer unique parts/processes, and doesn't have the constraint of dealing with an SSG. I remember my own personal pain from 2010 - 2016ish of managing Drupal and Joomla. I did exactly the same as you in 2016 and went all in on SSGs and in 2024, I realised all of the above. I feel like I wasted years of potential development time reinventing basic personal website features to try and work with an SSG and you literally have a ticket to do just that: https://github.com/geerlingguy/jeffgeerling-com/issues/167 https://github.com/geerlingguy/jeffgeerling-com/issues/167. 1 of your 3 solutions involves letting someone else host your comments:( A custom framework/server is the end destination for all nerdy personal websites - I can't wait to see what you make when you realise this:) edit/p.s. I love you and all your work. Sorry for sounding disagreeable, I'm excited to see what you learn from you SSG journey, I hope you prove me wrong!

geerlingguy 9mo ago
Definitely not disagreeable, more just "there are two right answers" ;) For me, an unstated reason for SSG is being able to scale to millions of requests per hour without scaling up my server to match. Serving static HTML is insanely easy, even on a cheap $5-10/month VPS. Serving anything dynamic at all is an order of magnitude harder. Though... I've been using Cloudflare since 2022, after I started getting regular targeted DDoSes (was fun initially, seeing someone target different parts of Drupal, until I just locked down everything except for the final comment pages). The site will randomly get 1-2 million requests in a few minutes, and now Cloudflare eats those quickly, instead of my VPS getting locked up. Ideally, I'll be able to host without Cloudflare in front at some point, but every month, because of one or two attacks, the site's using 25-35 TB of bandwidth (at least according to CF).
- 1dom 9mo ago